key success factor

Big data needs trust and organizational support

Why is big data not as successful as it should be – in the retail sector, for example? What is the role of trust, management, and the organizational set-up in big data adoption?

Those of you who have read my previous posts can tell that I’m a big supporter of the idea of big data. With technology as an enabler, I truly believe in the simple idea that in-depth analysis of “all” relevant details of the past and present can provide a very insightful basis for decision-making. So far, most people would probably agree with this very basic statement.

In order to effectively use big data in supply chain management, organizations and their supply chains need to be agile enough to react to big data’s findings. This applies especially when analyses call for short-term (re-)actions due to sudden changes in forecasted demand or imminent risks, for example. I recently came across an article that inspired me to dig a bit deeper in order to identify pre-conditions for such organizational agility.

Retail stock issues: big data to the rescue, or not?

The 2015 research study by the IHL group sponsored by Dynamic Action details the impact of overstock and out-of-stock situations in retail. In total, these two issues account for an overall loss of 7.3% of retail revenues globally, which equates to 1.06 trillion US$ per year. Without having counted the zeros in this number – this is a lot!

But wait, isn’t retail at the forefront of early adopters of big data? And shouldn’t big data – in theory – be the right approach to help exactly with issues like these? In fact, the retail industry is in a perfect position to access all relevant data to determine just the right amount of stock required in all facilities at all times.

Losses based on out-of-stock and overstock issues, however, have grown by around 40% since 2012, which is when this kind of research first started. So have big data projects helped at all to improve the situation? It actually seems big data’s impact has been rather limited up to now. There are many other reasons for the reported increase of losses in this study, of course. A rapidly growing part of global retail revenues is generated in developing countries with little IT support, for example.

The study provides further details regarding underlying issues: better aligned systems, forecasts, and insights could help recapture 50-70% of these losses. One example for this according to Dynamic Action is a great disconnect between marketing and sales. Improving internal collaboration, however, is not exclusively based on better data, technology, or algorithms…

All on-board: big data needs support to make it work

In my opinion, the root cause for many of these issues lies less in the use of big data as such and more in the overall organization and management of companies. And certainly this doesn’t only apply to the retail sector. It is a common challenge in all industries.

My conclusion is simple: to make effective use of big data, all involved parties – within and beyond an organization – must not only provide the data required centrally but as importantly, they must follow any decisions derived from the corresponding data analyses consistently and without delay.

If this is not the case, the required “full picture” cannot be established and the appropriate agility to effectively react in a coordinated way cannot be achieved. So what does this mean to an organization? And what is needed from the involved stakeholders?

Big data challenges: analysts versus business experts

The most challenging change that is required lies probably in the necessary shift of decision-making power from individual business units to central analytic experts. This power shift starts with business units sharing relevant internal data with a central analysis unit. Even though this may seem like a simple step, it’s in fact a great hurdle, as such data is often preferred to be kept within the realms of the responsible business unit.

In addition, business units are often not overly enthusiastic about adopting this approach, because it also means that decision-making power shifts from self-governed business units to “somewhere else” based on data analysis without business context. It’s a sensitive issue that can result in real resistance. This difficulty is further intensified by the fact that many companies are actually developing in the opposite direction: decision-making power is being shifted from the center to the periphery of organizations in order to cope with the ever-increasing market-environment complexity.

It’s quite hard for business units – where professional expertise and knowledge about the business resides – to understand and accept the need to move into a new subordinate role and leave decisions about “their business” to central analytic experts. After all, analytic decisions tend to be opaque to peripheral teams because they are based on complex algorithms fed by huge amounts of data from many different sources and as such, they are out of the control of single business units.

At the end of the day, it all comes down to trust

In conclusion, this means that the effective use of big data depends on an immense amount of trust across (and beyond!) entire organizations. Most companies are far from having established such a level of trust. But without trust, big data can only deliver benefits in a very limited way. It sounds a bit like the famous “chicken-and-egg” problem. The right way to overcome this paradox depends on the individual set-up of an organization, but typically involves a step-by-step approach including training and awareness initiatives.

Well, I will remain a big believer in big data despite all the challenges we need to tackle to make it a success. The basic principle, the potential, and its promise are just too attractive to be ignored.

How are you experiencing this challenge in your company? Let me know what your thoughts are on this on LinkedIn. I look forward to hearing from you!