Growth vs Disillusionment
The BLS* recently published a report "Big data adds up to opportunities in math careers" which is full of exciting news for those of us in the career field. They project growth of careers around mathematical sciences by 27.9% over the decade from 2016-26. We found ourselves already two years into that projected decade and I would say by the seat of the pants that feels right. And while the article focuses primarily on the mathematically focused jobs, all of the supporting roles around data, technology, and the domain integration will certainly grow with it.
Lurking under the surface of the exciting continued grown of Data Science is what I believe is also a growing problem; the tenure of data scientists at companies continues to decline, down to an average of only 2 years. This trend has been accelerating over the last 5 years, and shows no sign of an inflection point. So how do we make sure that data scientists are getting the jobs they want to have and companies are getting the return on their investment? While not a complete list, here are some of the keys to turning this trend in a positive direction.
- Setting realistic expectations - Companies have outlandish expectations and requirements for what they're looking for in a data science hire, and also make promises about the nature of the work that they cannot fulfill. Companies must be more thoughtful and realistic with what they're looking for when hiring, and look to create teams with the appropriately balanced skill sets - not a group of unicorns. They also need to understand the real use cases that are likely, and embed the data science teams into the real problems in the business.
- Create and foster an environment of continuous learning - Data Scientists are driven by curiosity and the opportunity to try new things. This isn't an attempt to just pad their resumes with the latest techniques and technologies, so we need to make sure to give them access to problems and tools that enable them to continuously grow. This needs to be tied to the real needs of the business, but there has to be opportunity for freedom and exploration.
- Invest in training Data Science leaders - Becoming a new manager is hard, in any career field and any industry. Becoming a data and analytics practitioner doesn't do much to prepare one for leadership. All to often we expect too much of new managers and leaders, without any investment in their leadership skills. The lack of good data science leaders is directly reflected in the shrinking tenure of data scientists, and as the growth of the field accelerates that problem will become more acute. We need to develop structured training and give new leaders space to step away from the technical and learn the people management.
- Create support structures - Along with better management, data science departments need to be part of a holistic data and analytics practice, with the appropriate support structures. Different companies have different needs, but at a minimum this means: working data science platforms, ease of access to appropriate data, dedicated (and skilled) engineering support, and a clear tie to the business initiatives and problems they can help to solve.
We should all be excited about the projected growth of the field, and to be part of a revolution in how companies (and the world) think about information and data. This excitement needs to be tempered with a realistic awareness of the challenges already facing data science and a willingness to focus on addressing them.
* Bureau of Labor and Statistics - An acronym I feel silly spelling out, since we've all used the heck out of its data, but just in case