Doing Data Science: Connecting Small Dots in Large Number

I was attending a conference on data science earlier today, and thought that it will be relevant to write on this topic. It's a reality that methods of data management and application are becoming increasingly popular in different walks of life. In business, it builds commercial viability in this ever-competitive world. Take the case of retail portal of Amazon, video website like YouTube, and many more. Their success relies on the right use of data for product offers, operational processes and decision-making.

It's not an innovation of a year or two. The famous usage of data that perhaps determined the fate of the World War II was the first recognisable data science effort, aided by machine.

The field of study has evolved into a commercial paradigm demanding popular recognition and understanding. This also poses challenge for mushrooming data consultants to have minimum level of understanding of mathematics, statistical methods, and computer algorithms and tools to prove any real suitability for the purpose.

But the central theme is simple.

How do we find if a data model is good enough, or if it is working fine for a particular scenario?

Well, it must be a mathematical model, no subjective narration. We may write a set of equations - popular, or not so popular.

Hmmm, can we adopt this for our future purpose?

Existing data fitted into it. We may claim that it could go well in future too. It would be a process to be tested with parameters put against time and for a period that will establish its authenticity.

The remaining aspect is all details. It's all about connecting dots with huge data that we generate.