Data analysis is not new. Even before computers are used, information gained in business travel or other activities is reviewed in order to make the process more efficient and more profitable. Of course this is a relatively small scale business given the limitations imposed by resources and labor; The analysis should be manual and sluggish by modern standards, but still useful. Voting, for example, has been done since the beginning of the 19th century, almost 200 years ago. The first national survey took place in 1916 and involved the publication of Literary Digest which sent millions of postcards and counted the yields. As a result, they correctly predicted Woodrow Wilson's election as president.
Since then, the volume of data has grown exponentially. The advent of internet and faster computing means that vast amounts of information can now be harvested and used to optimize business processes. The problem is that conventional methods are not at all suitable for deciphering all numbers and understanding it. The amount of information is phenomenal, and in that information there are insights that can be very useful. Once patterns are identified, they can be used to customize business practices, create targeted campaigns and remove ineffective ones. However, as with large amounts of storage, special software is required to understand all of this data in a useful way.
Due to the nature of BigData, specialist companies have grown up around it to manage the volume and complexity of the information involved.
Like many other large data companies, IBM is building its offer at Hadoop - so fast, affordable and open source. This allows companies to capture, manage and analyze structured and unstructured data with its BigInsights product. It's also available in the cloud (BigInsights on Cloud) to deliver the benefits of outsourced storage and processing, providing Hadoop as a service. The InfoSphere stream is designed to allow the retrieval and analysis of data in realtime for Internet-of-Things applications. IBM's analysis enables robust data collection and visualization with excellent flexibility and storage. You can also find lots of white documentation and documents that can be downloaded on their site.
Another well-known name in the field of IT, HP brings a lot of experience to the big data. As well as offering their own platforms, they run workshops to assess organizational needs. Then, 'when you are ready to transform your infrastructure, HP can help you develop an IT architecture that provides the capacity to manage your volume, speed, variety, pride, and data values.' The platform itself is based on Hadoop. HP sees to add value beyond the provision of the software itself, and will consult you to help you strategize to help you maximize the data you collect - and how to do it very efficiently.