A new study by the Economist Intelligence Unit has just been released that shows how big data is moving from its infancy to “data adolescence,” in which companies are increasingly meeting the challenges of a data-driven world.
The report, called “Big Data Evolution,” details the ways in which companies’ attitudes and activities have changed over the past four years with regards to big data — collecting it, storing it, analyzing it, and using it to make business decisions about strategy.
Data is becoming a corporate asset
The report shows that, since 2011, substantially more companies are treating their data as what it actually is: a strategic corporate asset. The initial excitement about the possibilities presented by big data is morphing into a more strategic approach, defining which data initiatives will have the biggest and most immediate impact.
I refer to this as asking the right questions. Companies are getting a bit savvier, and on the whole, are not asking for more data, but rather the right data to help solve specific problems and address certain issues.
Because of this greater understanding, executives are more likely to report they are making good, fact-based decisions about their data and their business.
In addition, data strategy has been elevated to the C-level, usually centralized with a CIO/CTO or a newly-appointed Chief Data Officer (CDO). Outside that position, executives across the board are more likely to be in charge of their departments’ particular data initiatives and instrumental in putting those resources to use.
Click Image To Enlarge
Click Image To Enlarge
Another important finding of the survey points to a strong correlation between good data management practices and financial success.
Companies with a well defined data policy, are much more likely to report that they are financially competitive with their peers and rivals. In addition, they’re more likely to report that their data initiatives are successful and effective at resolving real business problems.
The reason for this could be that data initiatives are moving out of the realm of theoretical possibility and well into reality, demonstrating the ability to focus on real business problems and provide practical solutions.
Less about volume and velocity than value
One final encouraging trend the report finds is that the “bigness” of big data is starting to wear off. Companies are less focused on the quantity of data they can collect and the speed at which they can access it, and more focused on the value the data can provide for their business.
I strongly believe that the right data is much more important and valuable than simply collecting more data, and the report bears that out.
As technology continues to improve, the “bigness” of big data will become less and less of a factor. Companies are becoming more comfortable with the idea that they will need to scale up to allow the value of data initiatives to reach all sectors of the business, and so they are becoming more comfortable with approximation, agility and experimentation.
From my point of view, all of these are positive signs that big data is moving out of its infant stages and is well on the way to data maturity.
Bernard Marr is a best-selling author, keynote speaker and business consultant in big data, analytics and enterprise performance. His new books are 'Big Data' 'Key Business Analytics'
COMMENTARY: The tone of corporate conversations about big data continues to shift from initial excitement to expecting long-term business impact.
Over the past four years, executives have not only become better educated about the technology behind big data, but have fully embraced the relevance of data to their corporate strategy and competitive success. It could be said that most companies are experiencing their “data adolescence”, increasingly rising to the challenge of executing and delivering against the promise and potential of big data.
What are the hallmarks of this current stage of evolution, and what does the path to “data adulthood” look like from here?
In February 2015, the Economist Intelligence Unit (EIU) conducted a global survey of 550 senior executives sponsored by SAS, to follow up on our 2011 and 2012 executive surveys. By comparing the results, we were able to examine the evolution of companies’ views, capabilities and practices regarding big data as a corporate asset, and explore the future implications as companies continue to mature as strategic data managers.
Additionally, EIU conducted six in-depth interviews with leading corporate big data thought leaders and practitioners. Two of these interviews revisited specific big data–related issues these companies faced beginning in 2011.
Key highlights of the research include the following:
- Since 2011, a significantly larger proportion of companies have come to regard and manage data as a strategic corporate asset. The ranks of companies with well-defined data-management strategies that focus on identifying and analysing the most valuable data (referred to here as “strategic data managers”) have swollen impressively since 2011. No longer indiscriminate data collectors or wasters, companies are entering a period when the initial excitement over the possibilities presented by big data gives way to the need to prioritise and develop on data initiatives with the biggest payoff. More companies have ventured further into this stage of their data evolution, and their executives are more likely to feel that they are better at making good, factbased business use of their information.
- Strategic data management is correlated with strong financial performance. Our survey points to a clear correlation between managing data strategically and achieving financial success. Companies with a well-defined data strategy are much more likely to report that they financially outperform their competitors. In addition, they are more likely to be successful in executing their data initiatives and effectively applying their data and analytics to resolve real and relevant business problems.
- Data-strategy ownership has been elevated and centralised, while engagement and demand from the business is at an all-time high. Across industries, data strategy has been elevated and centralised to the C-level, most often with the CIO/ CTO or the newly minted chief data officer (CDO) role. At the same time, senior executives across functions and business units are increasingly in the driver’s seat of their data initiatives, and not just relying on IT leadership to design and execute them.
- Data initiatives have moved from theoretical possibilities to focus on solving real and pressing business problems. Companies approach data initiatives today with a clear focus on their purpose—putting business value first. They are much more likely to start by articulating and finding a consensus on the high-priority business problems the organisation will solve by leveraging its data assets. Financial resources available for big data initiatives remain scarce, so there is a pronounced need to prioritise which initiatives to invest in, as well as how to demonstrate the financial return on these investments.
- Technical challenges associated with quality, quantity and security persist. Even top performers continue to struggle with a number of technical aspects of big data. These foundational aspects of data management still drown out the more advanced, higher-value-add aspects of data management, such as governance, compliance and converting data into actionable insights.
- The future of big data is less about volume and velocity, and more about the value that the business can extract from it. Going forward, companies will have to shift their attention away from the “bigness” of big data and focus on its business value. Data and analytics will be increasingly applied to predict future outcomes and automate decisions and actions. Most importantly, many companies will have to continue to evolve their structure and culture to scale up successful data pilots across the entire organisation. This means becoming more comfortable with approximation, agility and experimentation, and reinventing themselves into a new kind of information-driven, data-centric business—closer to data adulthood.
CIO's Now Consider Big Data Analytics A Game-Changer
Greg Taffet, CIO of U.S. Gas & Electric, when The Economist Intelligence Unit interviewed him back in 2011, said.
“It is going to be a game changer.”
He was referring to fast-moving, real-time “big data”—which, at that time, was a novel buzz word.
In EIU's first comprehensive study in 2011 of how companies perceive and handle big data as a corporate asset, just 9% of survey respondents said data had completely changed the way they do business, while 39% believed data had become an important tool that drives strategic decisions at their organisation. But more than half of executives saw data in less critically important terms.
Click Image To Enlarge
As you can see from the above EUI survey, 58% (First two colums for 2015 survey: 14% + 44%) of respondents now see data as a game-changing asset, or at least, an important decision-making tool.
In the EUI 2011 study, four categories of companies were identified based on their level of sophistication of their thinking and strategy vis-à-vis corporate data:
- Strategic data managers: companies that have well-defined data-management strategies that focus resources on collecting and analysing the most valuable data;
- Aspiring data managers: companies that understand the value of data and are marshalling resources to take better advantage of them;
- Data collectors: companies that collect a large amount of data but do not consistently maximise their value; and
- Data wasters: companies that collect data, yet severely underuse them.
EUI plotted the four corporate categories and compared 2011 vs 2015 in the following chart:
Click Image To Enlarge
The above chart (Figure 2) shows that, in the last four years, companies have advanced along the evolutionary curve and, compared with 2011, many more now have developed a well defined data strategy. The ranks of strategic data managers have swollen impressively (18% vs 33%), and actually showed the only growth among the four categories, while the number of data collectors and wasters is shrinking (28% to 20%).
Further evidence that companies are moving beyond strategy development and are tackling the adoption, or implementation, stage of data evolution is the fact that executives today put more of their valuable data to good use (see Figure 3).
Click Image To Enlarge
Alan Feeley, managing director of global shared services at Siemens, a global engineering firm, points out.
“Data and analytics are no longer opportunistic. They are now formal research areas for our company.”
The CIO and CTO Have Now Taken Ownership of Big Data
The ownership of data strategy and the sponsorship of data initiatives have evolved throughout the organisation. Responsibility for the organisation’s data strategy has been elevated and centralised to the C-level, but at the same time, the pull and energy are increasingly coming from the lower levels of the corporate pyramid. Over half of companies surveyed make sure that data are available to employees who need them, and offer the appropriate technology and training programmes. Data strategy has become “everybody’s business”—senior executives across functions and business units are increasingly in the driver’s seat of their data initiatives, instead of relying on the CIO or CTO to design and execute them in a top-down manner.
The vertical migration to centralised leadership of data strategy and strong ownership from the C-suite is an emerging best practice today. Ram Chandrashekar, executive vice-president of operational excellence and IT and president of Asia Pacific and Middle East region at ManpowerGroup, a global human-resources consulting company, says.
“Clearly, a top-down data strategy driven and articulated by the CEO is a critical success factor.”
The EUI Survey data support his observation.
Over the past four years, ownership of corporate data strategy has migrated upwards from executives at the business-unit level to C-suite members—particularly, the CIO. In 2011, 23% of respondents said their CIO is primarily responsible for all data initiatives. This proportion jumped to 30% in 2012, and continued to rise to 39% in 2015 (Figure 7 below).
Click Image To Enlarge
A recent appearance in our 2015 survey is the increasingly popular chief data officer (CDO) role. This C-level position was virtually unknown in 2011—limited mostly to government and heavily regulated industries such as banking and insurance following the 2008 financial crisis. In our 2015 survey, some 9% of respondents pointed to their CDO as the custodian of the corporate data strategy and capabilities. Emergence of this role comes at a good time, especially as business executives from across the functional spectrum have become much more technology-literate and involved in the design and execution of their data strategy and initiatives.
Emergence of the Chief Data Officer (CDO)
Increased involvement from the business comes with the challenge of co-ordinating agendas, aligning priorities and communicating effectively with all stakeholders. Mr Chandrashekar of ManpowerGroup says.
“There is strong alignment and articulation at the C-level. People on the frontline, such as sales and operational staff, are also data-driven. The disconnect often happens in the middle, and the challenge is to make the data flow from top to bottom. Engaging the business is critical—data strategy cannot be seen as just a central initiative.”
And few today excel at engaging the business. In the EUI 2015 survey, when asked to rate their company’s competence across different datarelated corporate capabilities, respondents expressed the least confidence in their ability to engage employees across the organisation to use data in day-to-day decision-making (only 26% rated their company as “very competent”, while 22% saw themselves as “not at all competent”). High-quality, consistent engagement across layers of the organisation and among horizontal functional lines is in high demand, and in short supply.
Click Image To Enlarge
Enter the CDO. Mr Krishnamurthy of Cognizant Technology Solutions says.
“The CDO has emerged as the embodiment of ‘integrated leadership’.”
He points out that the best-designed CDO roles are focused on three top-level priorities:
- Ensuring availability and integrity of data across the organisation;
- Driving adoption—from small-scale pilots to company-wide rollouts; and
- Driving the monetisation of new data capabilities.
Most importantly, the role is about organisational engagement, brokering between agendas and balancing priorities among big data initiatives. Thus, finding the right senior talent to fill the CDO role can be tricky, as Edd Dumbill, vice-president of marketing and strategy at Silicon Valley Data Science, a big data consulting firm, points out:
“They have to know technology, they have to know the business, and they have to be a political wiz.”
Foundational and talent challenges
Companies have made great strides in embracing data as a strategic asset, making the necessary technology investments, and even beginning to evolve their corporate structure. Centralised leadership allows for better co-ordination in strategy and execution of initiatives. And executives, both on the business side and in IT, are much more focused on deploying their limited resources on top-`priority data projects that extract tangible business value from these investments.
However, significant challenges still plague most companies—and that’s true even for companies with the financial resources. The most daunting challenges companies face relate to three. highly technical and operational aspects of big data—quality, quantity and security (see Figure 8). These are fundamental aspects of data management. Yet companies are far from having resolved them completely and with full confidence, leading to a lack of progress to more advanced, value-added aspects of data management.
In the last four years, the problems posed by the overwhelming amount of data companies can access and collect have only been exacerbated further. In 2011, one in eight companies said they had so much data that they struggled to make sense of them—in 2015 this was nearly one in four companies. And today, more than half of executives (54%) say they probably leverage only half of their valuable data (Figure 3).
Given the sheer volumes, ensuring the integrity and quality of data, and arriving at the proverbial “single source of truth”, are still major problems. And thus, the ultimate challenge of extracting meaningful and actionable business knowledge from data is still a significant one for most companies, even slightly more so for companies that say that they are strong financial performers as they may be more ambitious with their data strategy. But only 16% of companies these cite extracting business insights as a top challenge—for reported poor financial performers, this was 24%. Despite strong or poor financial performance, 33% of all survey respondents continue to struggle with managing the vast amount of data and 41% struggle with maintaining quality (Figure 9).
Click Image To Enlarge
On the organisational front, companies have made strides in both creating the right structures and roles, as well as recruiting key talent to enable them to formulate and begin executing their data strategy. However, the talent market in the data and analytics field is still very tight.
This is especially still true in the market for data strategists—executives who are expected to speak the languages of both technology and data science, as well as understand the business, the markets and the customers (see section Paving the way for the CDO). These rare and invaluable executives—the “effective engagers”, as Ms Merkel of Zurich Insurance calls them—are in short supply and high demand. As Mr Feeley of Siemens puts it,
“There’s a war for talent, particularly for people who combine data expertise with domain knowledge.”
In a blog post dated December 8, 2015, in which I wrote about the market for data scientists, I mentioned the serious talent shortage in big data and analytics.
Courtesy of an article dated November 30, 2015 appearing in Forbes and the whitepaper prepared by The Economist Intelligence Unit titled, "Big Data Evolution"
Follow Me: 
Twitter: turk5555 
Facebook: 797743955 
LinkedIn: turk5555
Recent Comments