In this article, we explain what to consider when carrying out successful Data Analytics projects. We review the current paradigm to understand how the data area works in companies today. Additionally, we reflect on the reasons for the failure of data projects.
The new paradigm and defining the data analytics strategy
70 years ago, for every difficulty, a technology was created to solve it. Today, the situation has changed, and there are multiple tools to solve a problem. Every industry and every part of the value chain have a specific technological application to embed in their processes.
This places us in front of a new paradigm marked by an excess of technological availability, which means there are many tools to integrate into each organizational process. This situation can be overwhelming; however, contrary to expectations, the challenge is not technological but deeply cultural.
In this new paradigm, data is at the center of any strategy, and these new ways of doing things compel us to start thinking differently.
What role does data analytics play in business today?
In theory, everything is clear, however, in practice, things don’t flow as smoothly and problems arise. A good data analytics project has to do with the effective use that people give it. We may have the best dashboard, the fastest model, the most detailed report, but if nobody uses it or makes decisions based on it, we will have failed.
According to the Harvard Business Review, the main cause of failure in data projects lies in a cultural dimension; people (often even top management) don’t fully understand it and, therefore, don’t adopt it.
Disciplines related to data & AI are new and have experienced exponential growth in recent years. This has brought about some changes:
- The area has gone from being important to critical in organizations. Today, more and more data models are at the heart of the business. Therefore, any failure in one of those pipelines can leave part of the company inoperative or can impact profitability.
- Faced with growing demand for data from the business, we started using new tools and technologies that allowed us to process that large amount of information in a more user-friendly way and enabled us to solve problems more effectively.
- The change has generated more and more challenges: more requests, more data, more models. In the best-case scenario, they will ask for changes all the time. Which is very good because it means, as we said above, that people use what we do. The worst-case scenario is to mistakenly think that if we implement something new and nobody asks for anything, it’s because the product is well done. The truth is they don’t ask because they don’t use it, not because it’s the ideal product.
The first step in any data analytics project
Working in data analytics involves obtaining explicit information from implicit information; it’s about using the data we have to generate information that we don’t have. Therefore, the purpose —and the first step— of any data analytics strategy is to define what we are going to use the data for.
Some questions we could ask ourselves to determine this are:
- What can I do differently with my data?
- What data am I not capturing?
- How can my value chain be enriched with my data?
- Using the data I have, can I improve a sale? Can I make a better offer? Can I predict a price?
What are the characteristics of data analytics projects?
Data analytics projects are based on an architecture composed of different components that, as a whole, provide people with quick and easy access to information. This is key to making better business decisions.
Data analytics solutions have the following characteristics:
- They are not a system or a product: A data project is something that cuts across the entire organization; it’s not necessarily about a technology or a product. Carrying out this type of project is much more than implementing reporting software or an artificial intelligence platform; it’s more about shaping a conglomerate of different elements involving multiple areas of the organization.
- Users must be directly involved: The use they make of the solution will be the true measure of success. Data analytics must work hand in hand with people who know the data, know how the business and industry work, and will be responsible for using the data and making decisions based on it. They are the protagonists and must be fully involved from the beginning.
- The linguistic and conceptual barrier must be reduced: There are gaps between technical and business areas, so data analytics must work to break them down. We don’t have to speak in technical terms; we have to understand the business. After all, to develop a pipeline or a predictive model, it is crucial to understand how the industry for which we are doing it works, what is important, and what value we bring to the business.
- Data quality problems become increasingly visible: People and the uses given to data are growing. This amplifies any data quality problem that may exist. We can develop the best predictive model, have the most efficient architecture, but if the data does not reflect a faithful reality, anything we do will not serve or, worse, could lead to decisions based on incorrect data.
- Dynamic nature: Data — like reality, the market, and technology — change constantly. This shortens development times because the business demands that we deliver value constantly, iteratively, and quickly.
- We need standards: The world of data is becoming more and more attractive to more business areas. So, we can’t develop as we please; we must have standards that allow us to scale.
Why do data analytics projects fail?
As we’ve mentioned, developing a data analytics project is much more than just installing software or an application. It involves actively involving different people and business areas that may not always handle technical language or know the details of data engineering work (something they don’t need to know as it’s not their field).
When the level of demand increases, data quality problems become more visible, and the chances of failure are greater. While the reasons for this vary — many of which relate to the cultural dimension of the organization — on a practical level, we could say that these are the two most common causes of data analytics project failures.
1) Analysis paralysis: This situation occurs when we decide not to start the project until we’re 100% sure of what we’re going to do. To kick-start it, we wait to have everything surveyed, all documentation signed, etc. Reality shows that this is more of a hindrance than an enabler because it leads us to do nothing, and once we decide to start, things have changed, and what we did has likely become obsolete.
2) Extinction by action: This is precisely the opposite of the previous point. We start quickly and act as things are defined. We act without having a clear goal or objective. So, whatever we do, it never really becomes anything, it cannot grow or evolve, we don’t know where we’re going, and it ends in a big failure.
How to make successful data analytics projects?
🔨 Build them in parts: The secret of any data project is to do it in parts; don’t try to cover everything from the start because that way, sooner or later, we’ll fail.
👩💻 Start with something simple: The first steps of a data project will depend on the capabilities and experience of the team. It is advisable to start with something they are prepared for; they can start with something simple from which they can continue to grow. The important thing is to ensure that the team can understand the technology, the business, the data, the environment, the users, etc.
🧩 The relationships between the parts should set the pace. When we define data projects, we should assemble a sort of puzzle. For example, if we make a “sales” Data Mart, the second one should not be “human resources” because both areas are not related. Ideally, we should choose a second Data Mart that is close to the first one; for example, if we start with “sales,” we should continue with something like “marketing” or “logistics,” for example.
🛑 Stop and rethink everything. Whenever necessary, take the time to analyze and think if we are on the right track, discuss with users, the team, and internal clients what we are doing and where we are going.
The important thing is not to lose sight that we are the people who work with the data. We have to build a team, not only with those who work with us but also with the clients; this will generate mutual trust. Both parties will have the assurance that each will give everything for the project to succeed. By focusing on people, we will be able to generate commitment, inspiration, build more empathetic bonds, and, above all, set and achieve common goals. After all, no matter how advanced the technologies we work with, we were, are, and will be people who work with data.
—
The original version of this article was written in Spanish and translated into English by ChatGPT.
