The analysis of Big Data is becoming increasingly important in today’s business world, yet the role of data scientists in evaluating that data is more controversial than you might expect.
While it seems intuitive that data scientists can be a useful asset to any company looking to analyze their own data, a CIO article from earlier this year argues that there is a big misconception about what the job actually entails on a day-to-day basis.
“Hiring a Data Scientist? You’re Doing it all Wrong”
The article asserts that data scientists spend much more time organizing data than actually analyzing it. This is especially true for new hires who may enter a company with unrealistic expectations about the state of the data they are being brought in to leverage.
The author warns new hires to inquire into the quality and organization of a company’s data before accepting a job to ensure that they will be tasked with the responsibilities of a data scientist and not a “data janitor.” Conversely, she cautions employers looking to hire data scientists to understand the state of their data and to establish reasonable expectations for what the prospective employee can do with it.
These are valid points, as many companies looking to hire data scientists don’t actually have the architecture, resources, or capabilities to use them effectively. Furthermore, some company cultures don’t allow for changes to be made based on the solutions their data scientists can provide, letting their analysis go to waste.
With this in mind, utilizing data scientists to the best of their abilities is undoubtedly going to become a major challenge for organizations that rely on Big Data in the years ahead. Currently, the role of “data organizer” and “data analyst” are not mutually exclusive in many institutions. And organizing the data in a clear and cohesive way is commonly expected to be one of a data scientist’s primary responsibilities. If we want to maximize the potential of the data scientists we employ, this needs to change.
“People are wasting time and money hiring data scientists if their data isn’t ready to work with.” Says Aberdeen’s Matt Grant. “Every organization needs to have processes and people in place that ensure that data is accurate and accessible. Do you want your data scientists to do something interesting and innovative? Or do you want them to worry about data governance and management?”
Doing the Dirty Work
Big Data is inherently messy. It is a conglomeration of data points taken from different sources at different times, and it is a given that it will consist of both structured and unstructured data.
While many data scientists may accept that organizing data is part of the job, doing so at the expense of the much more important analysis component will only delay the time it takes to draw actionable conclusions from large datasets. Somebody, of course, has to spend time putting it all together and making sure that the data is accurate and readable. But should it be the same person you’re paying $100k+ a year to provide invaluable analysis that will inform major business decisions and influence the direction of the company? Probably not.
Although the full scope of a data scientist’s responsibilities may be up for debate and will certainly vary from business to business, it is hard to deny the value that they provide when used correctly. A recent Aberdeen research report shows that companies employing data scientists see a 15% year-over-year increase in organic revenue, in addition to many other benefits and advantages.
Drawing practical conclusions from large quantities of data is the data scientist’s ultimate goal, but getting the data into a workable state is an essential first step. Some organizations are better equipped than others to bring in data scientists and let them do what they do best, but hiring organizations should strive to be cognizant of what the role actually entails and understand how to truly get the most out of data scientists before bringing one on board.