Eugene Khazin, Principal at Prime TSR, wrote a detailed blog post about why most big data analytics projects fail and followed it up with a podcast interview on the Data Engineering Podcast with host Tobias Macey.
If you’re in the beginning stages or in the process of implementing analytics at your company, this podcast is a must-listen.
Here are some of the key takeaways.
- Most analytics projects fail simply because the data isn’t in the right shape to analyze or prepare for operational uses.
- Get data engineers involved early on in projects. Don’t wait before it’s too late.
- Without a proper data strategy & data platform, companies will struggle to get proper analytics. Analytics requires the right data in the right format.
- Insurance companies who build a proper data platform with data from various sources can use predictive analytics to prevent fraud.
Detailed show notes are below, and here are a few interesting excerpts from the show:
Show Notes
05:22 — Two questions a company should ask itself before venturing into analytics projects.
06:59 — A case study on how a large hospital client used analytics to reduce ER visits. The real goal was to reduce ER visits by 7%.
10:08 — Why most analytics projects fail.
13:08 — Strategies to increase the overall success rate of projects that data engineers work on.
15:05 — How data engineers can engage with business early on in the project to show their capabilities institutionally, organizationally, and from a technology/infrastructure perspective.
16:52 — How data engineers can be better prepared for success on projects, and some areas of improvement for people working in the data engineering space.
19:32 — An insurance company case study of an initial failure that turned into a success—how insurance companies can use predictive analytics to anticipate fraud.
22:52 — Common patterns that lead to project failure.
29:10 — The future of cloud-native technologies and how companies have to adjust to the more modern methods of implementing analytics.