Why Data Projects Fail to Deliver Real-Life Impact: 5 Critical Elements to Watch Out for as an Analytics Manager

A simple guide to understand the macro-elements that can negatively impact your work

Ever found yourself deep in a data project, only to realize it’s going nowhere? It's a more common feeling than you might think: 

• VentureBeat reported that 87% of data science projects don’t make it into production 

• Gartner predicted in 2018 that by 2022 85% of AI projects would deliver erroneous outcomes. In 2016, they estimated that 60% of big data projects fail.

Two weeks ago we discussed how to do quality data analyses, but producing a high-quality analysis is really only half the battle. A lot of impressive work never actually make it to real life and end up being “displays of data acumen” (at best). So how do you cross the gap between quality work and impactful work? 

The very first step is to understand the rules of the game - and have a good visibility over the macro-elements that’ll determine whether your project will soar or sink.

PESTEL - but for Analytics 

If you have ever interacted with a few consulting folks (or if you yourself have a consulting background), you might have heard of the term “PESTEL”. It stands for “Political, Economic , Social, Technological, Environmental, Legal”. This framework is used to understand the macro-environmental factors affecting an organization and to form a better perspective of the strengths, weaknesses, opportunities and threats for a business. 

To some extent, the same principle can apply to assessing the potential success of your data projects, but with a twist (frameworks, after all, are tools meant to be adapted, not adopted wholesale). For our variant, we have Data Availability, Skillset, Timeframe, Organizational Readiness, and Political Environment. Each of these factors is like a puzzle piece in the big picture of your data project’s success. Understanding and aligning these elements is like tuning an engine: get it right, and your project will hum along beautifully; get it wrong, and you're in for a bumpy ride.

Data Availability

That is a tautology - but for any data project, you need data. The availability and accessibility of relevant data are fundamental. If you find that the necessary data is unavailable, or if it proves impossible to obtain, your project will face significant challenges. It’s important not to concede defeat immediately upon encountering this obstacle though - you should explore other options to either acquire the data or identify a viable proxy (and persistence in this phase is key - I saw countless of projects being abandoned at this phase even though a suitable solution existed). But, if after a very thorough investigation you conclude that the data is truly unattainable and no suitable proxy exists, then it's definitely a valid (and even sound) decision to reconsider the feasibility of the project.

Example: imagine you're planning a study to analyze consumer behavior in a niche market, but you discover that specific consumer data for this segment is not collected by any existing sources. Before abandoning the project, you might explore alternative data sources like social media trends, related market studies, or even conduct a targeted survey to gather approximate data. If all these efforts fail to yield useful data, it would then be justifiable to halt the project

Skillsets

Now that you have the data - do you have the right skillsets to investigate it? It's not just about having a handle on technical skills like SQL or Python; it's equally about possessing the specific knowledge required for the type of analysis you're undertaking. This becomes particularly crucial when the project's requirements fall outside your usual area of expertise. For example, if your forte is in constructing data pipelines, but the project at hand is centered around sophisticated forecasting, this misalignment in skills can become a significant barrier. Depending on the distance between your team’s current skills and the ones they need to acquire, you might consider upskilling the team - which can also be very rewarding in the long term - provided it aligns with the project timeline. It's about striking the right balance: seizing opportunities for development while also being realistic about the project's timeline and priorities.

Example: You manage a healthcare research team experienced in patient data analysis, and you are asked to undertake a project that requires them to apply epidemiological modeling to predict the spread of a disease. While they are skilled in handling patient data, the specific demands of epidemiological forecasting—a different realm of expertise—might pose a significant challenge. 

Timeframe

When it comes to time, there are two elements to understand:

• If you don’t leave enough time for a project to be completed, the quality of the project can be highly impacted.

• After a certain duration, you hit a point of diminishing returns, where adding more time doesn't necessarily equate to the same additional level of quality.

This video (the viral spiderman drawing) is a great representation of this phenomenon. The leap in quality between a 10-second and a 1-minute drawing is remarkable, showcasing a significant improvement with just 50 additional seconds. But, when comparing the 1-minute drawing to one that took 10 minutes, while the latter is undeniably better, the degree of improvement is less pronounced despite the 5x increase in time.

Example: You work for a retail company that wants to analyze customer purchasing patterns to optimize its stock levels for the upcoming holiday season. If your data team is given one week to conduct the analysis, they can deliver basic insights, identifying general trends and top-selling items. However, if they're given a month, the quality of the analysis significantly improves, allowing for a more nuanced understanding of customer preferences, regional variations, and potential stock issues. Yet, extending this time to three months might only yield marginally more detailed insights, while delaying crucial decision-making and potentially missing market opportunities.

Organizational Readiness

Organizational readiness is about how prepared and willing a company is to make the most out of data insights. It's not just about having the data or the analysis; it's about having the right structure and processes in place to act on those insights. In a previous article, I discussed the importance of making your study 'digestible' to increase the adoption of insights. However, there's an extent to which this facilitation is beyond your control.

Example: Suppose you discover that a particular store isn't performing well, primarily due to its less-than-ideal location. You propose that relocating just a few blocks could significantly boost earnings. To prove this point, you collaborate with an operations team to set up a temporary 'pop-up' shop in the proposed new location. This experiment runs long enough to negate any novelty effect, conclusively demonstrating the potential for increased revenue. Yet, here's where organizational readiness comes into play: the company is tied into a five-year lease at the current underperforming location, with financial subsidies and no suitable alternative space readily available in the desired area.

Political Environment

Everybody’s favorite one: navigating the political landscape within an organization <3. It is unfortunately a crucial step for the success of a data analysis project. You need the alignment of your stakeholders with the project's goals, but also on the roles and responsibilities linked to the project. At times, you’ll get competing interests among teams or a lack of consensus on project ownership - these are high risk situations for your project that you need to navigate prior to actually working on the project (if you don’t want several teams working in silos and doing the exact same thing). 

Example: You’re in a multinational corporation where two regional teams are tasked with analyzing market trends for a new product launch. However, due to historical rivalries and lack of clear leadership direction, these teams operate in silos. Each team uses different methodologies and data sources, leading to conflicting conclusions. Such a scenario not only breeds mistrust in the data but also creates confusion at the executive level regarding which insights to trust and act upon. This dissonance can ultimately lead to the dismissal of valuable findings, underscoring the critical impact of political harmony in leveraging data effectively.

Conclusion

The key elements we've discussed - Data, Skills, Time, Organizational Readiness, and Politics - are the gears that drive the success of any data project. Without the right data, even the most skilled team can't build insights. But skills matter too; they turn data into meaningful analysis. Time is your canvas - too little and your picture is incomplete, too much and you risk losing focus. Organizational Readiness is about ensuring your findings don’t just sit on a shelf gathering dust; they need to be actionable. And let’s not forget Politics - the art of navigating your organization to make sure your work sees the light of day.

In the end, it’s about understanding the dynamics at play within your organization to steer your projects toward success, i.e. to not just produce insights but to also drive change.