How Companies Can Prevent Big Data Analytics Failures

Eugene Khazin

Principal and Co-Founder, Prime TSR

Eugene Khazin, Principal at Prime TSR, wrote a detailed blog post about why most big data analytics projects fail and followed it up with a podcast interview on the Data Engineering Podcast with host Tobias Macey.

If you’re in the beginning stages or in the process of implementing analytics at your company, this podcast is a must-listen

Here are some of the key takeaways. 

  1. Most analytics projects fail simply because the data isn’t in the right shape to analyze or prepare for operational uses.
  2. Get data engineers involved early on in projects. Don’t wait before it’s too late. 
  3. Without a proper data strategy & data platform, companies will struggle to get proper analytics. Analytics requires the right data in the right format. 
  4. Insurance companies who build a proper data platform with data from various sources can use predictive analytics to prevent fraud.

Detailed show notes are below, and here are a few interesting excerpts from the show:

Show Notes

05:22 — Two questions a company should ask itself before venturing into analytics projects.

06:59 — A case study on how a large hospital client used analytics to reduce ER visits. The real goal was to reduce ER visits by 7%.

10:08 — Why most analytics projects fail.

13:08 — Strategies to increase the overall success rate of projects that data engineers work on.

15:05 — How data engineers can engage with business early on in the project to show their capabilities institutionally, organizationally, and from a technology/infrastructure perspective.

16:52 — How data engineers can be better prepared for success on projects, and some areas of improvement for people working in the data engineering space.

19:32 — An insurance company case study of an initial failure that turned into a success—how insurance companies can use predictive analytics to anticipate fraud.

22:52 — Common patterns that lead to project failure.

29:10 — The future of cloud-native technologies and how companies have to adjust to the more modern methods of implementing analytics.