MASTERING CREATIVITY IN DATA SCIENCE
Data analytics and AI is just as much as art an it is a science. That makes it a creative task. Thankfully we are all creative beings.
Success in working with and using data requires nurturing this creativity. Unleashing our creativity to build a pathway from problems and challenges to answering questions, providing direction and making lives better.
I find most problems in data science and analytics are unique. They thus require unique creative solutions to solve the issues at hand. I have to think what available data I have (and its limitations), how I can manipulate it, how to avoid biased analysis, how to tailor it for implementation and allow it to be responsive to changes. In doing all these striving to avoid unnecessary complexity; the great artists of course create beauty in simplicity. And critically how to communicate such work a way that will inspire robust yet creative solutions for the given audience. None of these tasks flourish with linear thought.
It is important to note that unsupervised AIs are not yet in a position to do this. They certainly create and can be creative in the confined environments (think Move 37 made by Alpha Go or AI Art Generators). But this creativity comes from what they have learnt and its connections, not the full abstract multi-faceted creativity humans are capable of in our complex world. We have the edge for now.
How do we find this creativity, especially for many of us where it may not be our natural way of thinking? Although a lot of this is individual, there are certainly some common factors. You can't beat heathy living and a good sleep. Stepping away for our working environment is helpful, few great thoughts are inspired staring at a spreadsheet (as beautiful as they can be!).
I find I need to break out of our habitual thinking methods. I am helped by physical exercise – running or time in nature (yes, maybe a bit sad I solve a coding challenge whilst marvelling at the mountains). I also find visual art a creative prompt for me - and who can't be inspired by a well thought out and beautiful data visualisation.
AI can still be an effective tool. Large Language Models (LLMs) for instance can be very useful. They are not necessarily creative but can inspire creativity by formulating ideas in a new way (e.g., try "10 ideas on blogs about XXX" in ChatGPT - not used on this occasion I should caveat!). Nonetheless, no one wants our film habits defined by "10 ideas for movie plots" (yes, support the Hollywood writers!).
The place where I think creativity flourishes most is in the team. The best way to get a successful project rolling is getting a diverse mix of creative, skilled and experienced minds together and working through scenarios and solutions. Many failed projects would be avoided if this was done effectively first.
I believe creativity is not something that effortlessly flourishes for most people. It is a skill that needs to be strengthened and given space to perform. But it is essential for all those working with data or using its outputs – the great breakthroughs don’t tend to come by faster, stronger and more efficient, but in seeing things a whole new way.
How do others nurture creativity in their work? Any successes anyone would like to share?