Intuition in the Age of Big Data

How far ahead are Canadian organizations in their use of data analytics? The Smith School of Business Executive Survey released recently shows that leaders of Canada's organizations agree Big Data is changing the business landscape. They also seem to be split on how they should be using data/analytics in decision making.

Of those surveyed, 52 percent said they rely too much on data and analytics when making decisions and not enough on their intuition, while 41 percent indicated they rely on their intuition and experience more and not enough on data and analytics.

"We've seen an incredible uptick in the amount of data available to business leaders over the past decade," says Salman Mufti, associate dean and executive director of Queen's Executive Education at Smith School of Business. "However, leaders are still learning how to best use that data to improve the efficiency and profitability of their organizations."

The survey of 250 senior leaders in Canada reveals that Big Data is now changing how business leaders make decisions. Executives are now relying more on data analytics to guide strategic business moves, saying that their decisions tend to yield better results when based on data analytics, instead of experience or intuition.

Key findings include:

• 81 percent of the leaders at least somewhat agree that "Big Data" is changing how leaders make major business decisions and that important decisions are now based more on data and analytics and less on intuition and experience.
• Of those surveyed, 56 percent find that decisions based on data analytics, as opposed to experience and intuition, tend to deliver better results; 28 percent say it makes no difference.

"What business leaders should recognize is that data analytics and intuition are not mutually exclusive," says Mufti. "Even when we use data, intuition and experience are still critical in developing novel strategies to meet business challenges. The additional data available simply allows us to test our assumptions, recognize new patterns when creating strategy and make course corrections when strategies are not working out."

Using Analytics and Gut

So when do executives rely on data analytics and when do they tend to go with their gut? According to the Smith School of Business Executive Survey, executives agree that business leaders must go with their gut and intuition when time is of the essence. They also tend to rely on intuition when making people-oriented decisions and when dealing with volatile situations – such as crises – where there is more focus on interpersonal and relational assessment than just data analysis.

• About 78 percent of respondents at least somewhat agree that a business leader must go with his or her gut and intuition when an important decision needs to be made quickly.
• Data analytics is behind the majority of leaders' strategic decision-making (71 percent), while leaders tend to use intuition to make decisions involving their teams (66 percent) and crisis management (65 percent).

"Sometimes you can have too much or conflicting information, and it doesn't always provide a clear direction on what to do," says Mufti. "I think what the Smith School of Business Executive Survey results are showing is that business leaders are concerned about data paralysis and realize that sometimes – for certain types of management challenges – quick decisions are critical to an organization's success."

Barriers to Using Data Analytics

There is still some ambivalence among Canada's organizational leaders when it comes to fully embracing data and analytics as an essential part of their decision-making process. The Smith School of Business Executive Survey highlights the following top five barriers to using more data and analytics in making important decisions, as identified by the leaders surveyed:

• Availability and accessibility of accurate or reliable data (25 percent)
• Lack of tools to collect and analyze data (21 percent)
• Lack of staff with expertise to collect and analyze data (18 percent)
• Lack of time to collect and analyze data (14 percent)
• Reliance on intuition/data interpretation (9 percent)

"What we are learning from this study is how leaders feel about their and their organizations' decision-making approaches," says Mufti. "But what benchmarks should we apply to measure success? How do we marry technological and managerial excellence to achieve top performance? What's an optimal mix of analytical and intuitive approaches in making important decisions – say, in a strategic turnaround, restructuring or a merger and acquisition scenario? These are the questions we encourage leaders to think about in the context of their organizations and the challenges they are facing."