Big Data Analytics: Break free of silos

01st August 2015
Big Data Analytics: Break free of silos

Moshe Kranc, Chief Data Officer, Ness Software Engineering Services (SES) suggests why  enterprises must embrace big data solutions only after a personal tryout
In my position as Chief Data Officer for Ness Software Engineering Services, I talk with many companies about their Big Data challenges and needs. Some of these companies have chosen to analyze their Big Data to enable use cases such as online personalization or creating a 360-degree view of their users. Others have been forced into this area by one of their competitors, who announced that, thanks to their successful Big Data initiative, they now possess a competitive advantage over all their competitors; e.g., their products require significantly fewer service calls because they can monitor the performance of the equipment remotely.

If there is one common thread that links 95% of these customers, it is FUD – fear, uncertainty, doubt. The field is full of competing or overlapping products, each of which claims to be the right solution for Big Data. The Big Boys (Oracle, IBM, SAP, Teradata, etc.) all want to steer you towards their (sometimes costly) offering. The upstarts (Cloudera, HortonWorks, DataBricks, DataStax, etc.) all talk about use cases with dramatic savings, but you may not be familiar with the use cases in which their product isn’t designed to work as well. And, depending on your vertical, there are dozens of one stop shops who offer to take your data and come back with vertical-specific insights.

Left to evaluate this cacophony of conflicting voices is your organization. It’s hardly a fair battle – you may not know the hard questions to ask. That company that promises 20,000 writes per second? That’s only true if only one process is writing the data – the performance tanks when there are multiple writers. That product that claims to support full SQL? They consider little things like JOINs, transactions and nested queries to be out of scope.

No wonder we read so often about Big Data failures. “Product X let us down,” announces the CTO of yet another large financial institution that made the mistake of believing the vendor hype. The real culprit is not Product X – the real culprit is the silo approach that drove the company to make a choice without the benefit of outside advice or experience. The only way to cut through the hype around a product is to try it yourself, and/or talk to someone you trust who is using it.

The best advice I can give to a company contemplating a Big Data initiative for the first time: Get help. Perhaps you can hire experts from outside your organization who have already built such a system. Perhaps you can build up a network of friends who have made the beginner’s mistakes and can share their insights with you. Another option is to partner with a company like Ness Software Engineering Services that has managed a broad range of Big Data projects and technologies, and has a proven track record of success. The silo approach, going it alone like the sheriff facing off against the outlaws in a Hollywood Western film, may work in the movies, but not in Big Data.

Moshe Kranc , has over 25 years’ experience in management roles and another 10 years working in high-tech. Most recently, Moshe led the Big Data Centre of Excellence in Tel Aviv for Barclays Bank, which provided guidance and Proof of Concepts (PoCs) for Big Data technologies and projects across the bank. Moshe was also part of the Emmy award-winning team that designed the scrambling system for DIRECTV. 
August 2 2015