Why 'research' from the big consultancies should be ignored

With no independent peer review required, big consultancies are free to cynically dress up marketing as "research” to sell us nonsense. It’s shocking. Here are just a few choice examples.

Wellbeing became a hot trend around 2017 when Deloitte launched workplace wellbeing surveys. Executives got very uncomfortable when many employees reported that they weren’t feeling ‘very well’ in these surveys. Large organisations appointed Chief Wellbeing Officers, recruited wellbeing teams and hired costly wellbeing consultants who suddenly appeared on the scene from nowhere. A veritable avalanche of activity ensued, with some organisations creating over eighty different wellbeing interventions - everything from ‘duvet days’ to workplace yoga, guided meditation apps, one-on-one mindfulness coaching, sound wave healing, cupping, and dog petting.

Photo by Will Myers on Unsplash

Millions of employees received mildly patronising missives about taking action to improve their mental health, no doubt causing a rise in anxiety amongst those who were previously fine. Annual reports and senior leadership conferences were full of ‘wellbeing strategy’ presentations highlighting what a great and caring employer the organisation had suddenly become.

There was only one slightly inconvenient problem with all of this. It turns out that there’s no evidence that any of these interventions increase employee wellbeing. Multiple research studies have come to the same conclusions. Oxford University’s Wellbeing Centre published research in 2023 on the effect of mental health interventions across 143 UK companies with 27,932 employees(1). The report’s author, Dr William Fleming, says:

“The analysis estimates the effect of a range of common initiatives, including mindfulness, resilience training, stress management and wellbeing apps. No evidence is found to demonstrate that these strategies improve worker mental health across multiple employee mental health measures. This suggests that recent critical literature is right to be concerned and that convenient wellbeing strategies should be given less financial and institutional support.”

Dr Fleming specifically calls out Deloitte’s “research” and their declaration of an overwhelmingly positive business case for investment in wellbeing, saying that “their ad hoc selection process and inability to identify valid studies undermines return-on-investment estimates that are widely and uncritically shared in public media and academic literature.”

Let me paraphrase the above in non-academic language: It’s complete bullshit.

The “research” from Deloitte was beautifully crafted to exploit our dislike of uncertainty and drive for sense-making. It’s little wonder that it quickly became a top executive focus globally. Let’s deconstruct it.

Step one: Trigger our aversion to uncertainty. Our research shows that many employees should be considered at risk of high stress, burnout, underperformance, and leaving. The future of your organisation is uncertain.

Step 2. Create the illusion of sense-making: It’s all the organisation’s fault (i.e., your fault as leaders). You’ve allowed this to happen through your uncaring approach as an employer with long hours, demanding managers, pressured tasks, etc. What were you thinking?

Step 3. Promise control over the future: We can help you to fix your wellbeing through this range of interventions. Here’s the cast-iron business case for the investment. Trust us; we know what we are doing. This will cost you £X but will return almost £5X.

Despite having their research methodology so publicly slammed, as recently as May 2024, Deloitte still claimed that: “For every £1 spent on supporting the mental health and wellbeing of their workforce, employers get (on average) about £4.70 back in increased productivity.”(2) Shameless.

Accenture’s 2022 report, The CHRO as a Growth Executive(3), claims to have found that “by activating the growth combination of data, technology and people, companies stand to gain a premium of up to 11% on top-line productivity” compared to just 4% where companies “implement data and tech solutions that fail to put people at the centre”. How did they identify these ‘leading companies’ that ‘put people at the centre’? The answer lies in the small print at the back of the report:

“Leading companies were identified by applying text analytics to earnings call transcripts…strategic focus on talent creation and digital core topics was determined through a set of keywords and phrases.”

So, the ‘leading’ companies were chosen based on how many management buzzwords about ‘talent creation’ and ‘digital core topics’ the management team threw out in the PR-spun earnings calls with market analysts. This is not only incredibly lazy but also just plain silly.

Accenture also released a report in October 2024 titled The Productivity Payoff: Unlock Competitiveness with Gen AI. In the report, the authors mash up bizarrely different data sets to conclude that a new “Productivity Equation” has been validated. The equation is:

Cost Inputs x Effectiveness Factor x Generative AI Multiplier

The report concludes: “Our analysis indicates that taking a holistic approach to productivity that encompasses actions across the three dimensions could increase the productivity growth of the median company up to 16% annually”, going on to suggest that “For an average-productivity performance company, this could translate to a boost of 2.8x its EBIT over a 10-year period.” The three data sets mashed together to reach these conclusions were:

1. Survey data from 2,000 executives asked about “their strategies for unlocking productivity and where they invest gains.” The survey results were “integrated with historical productivity trends to discern effective management practices.”

2. An AI-generated analysis of 63,000 earnings calls to: “understand how leaders communicated to investors about productivity.”

3. A Generative AI Labor Productivity Model “incorporating the latest findings from AI research by leading academia and AI labs, assessing the potential of generative AI to enhance both the efficiency and quality of work across various tasks.”

So, let’s deconstruct this odd mashup. Firstly, self-reported survey data (notoriously unreliable) was examined for correlation with historical productivity trends. Once again, it appears that correlation was assumed to be causation*, given the conclusion that this allowed the authors to “discern effective management practices”. This is entirely invalid from a credible research perspective. Secondly, using AI to analyse earnings calls is once more silly and lazy. All it might reveal is trends in buzzwords used on the calls, which are essentially a form of marketing. No meaningful conclusions can be drawn about actual practices in these companies. Thirdly, the assessment of the potential benefits of Generative AI was based on reviewing 25 academic experiments with no data from the actual deployment of Generative AI in the workplace. There’s no evidence in the report that the results of these experiments are repeatable in real-world situations. Therefore, the actual equation in this report is:

Mere Correlation x Buzzword Frequency x Unvalidated AI Gains

And that equals another load of meaningless drivel, frankly. Accenture should be ashamed of publishing this. If they submitted this research to any serious academic journal, the resulting hoots of laughter would resonate around academia for some time.

Another fundamental issue with consultants’ research is that much of it is done in functional silos. This gives rise to ‘research’ that claims isolated functional interventions caused performance gains. Here are a few examples:

• Boston Consulting Group claims that more diverse management teams deliver a 9% higher EBIT margin than less diverse teams(4).

• Accenture claims that putting people at the centre of data and technology implementations improves “top-line productivity” by 11%.

• Gallup research suggests that moving from the bottom to the top quartile on engagement delivers a median 23% uplift in profit(5).

• Deloitte claims that its 2022 Digital Frontier Study(6) showed that organisations in the study with “tech savvy” boards saw, on average, 8% better year-on-year stock performance than those with “non-tech savvy boards”.

Are we meant to believe these gains are entirely independent - the single root cause of significant performance improvement? This is hardly credible in a complex world. If you could increase profits from, let’s say, £100m per year to £125m by adopting these recommendations, you could then invest the excess £25m in wellbeing, which, according to Deloitte, would return, on average, £117.5m. Happy days - we’ve more than doubled our profits…I think not. It’s completely ridiculous.

Executives should view any research published by management consultants, executive coaching organisations, software vendors, and other conflicted parties with extreme scepticism. In many cases, basic research good practice is missing. Studies often focus not on random samples but on organisations cherry-picked by the authors to confirm starting hypotheses. Sample sizes are frequently too small to be statistically significant. The questionable-cause logic fallacy is endemic. Context is often completely ignored, with research findings frequently morphing into lists of the “10 top takeaways to implement in your organisation today”. Don’t be fooled; implementing these recommendations is invariably pointless. It will just add more silly ‘stuff’ to any already nonsense-strewn work landscape. Your organisation has likely already fallen for far too much of this twaddle.

Excerpts from Magnetic Nonsense: A Short History of Bullshit at Work and How to Make it Go Away

Need any help sorting out nonsense in your organisation? www.disruptionspace.co

∗ Remember the number one mantra of the scientific method: Correlation does not imply causation!

1. Estimating effects of individual-level workplace mental wellbeing interventions:Cross-sectional evidence from the UK, William Fleming, Wellbeing Research Centre, University of Oxford, April 2023

2. Deloitte.com: Poor mental health costs UK employers £51 billion a year for employees, 17th May 2024

3. https://www.accenture.com/content/dam/accenture/final/capabilities/strategy-and-consulting/talent-and-organization/document/Accenture-CHRO-Growth-Executive.pdf

4. https://www.bcg.com/publications/2018/how-diverse-leadership-teams-boost-innovation

5. Gallup: The Relationship Between Engagement at Work and Organizational Outcomes Q12® Meta-Analysis: 11th Edition

6. https://www2.deloitte.com/us/en/insights/topics/leadership/digital-transformation-topics-for-corporate-technology-leadership.html