IT organizations are increasingly looking to cloud computing as the structure to support their big data projects. While enterprises often keep their most sensitive data in-house, huge volumes of public data such as social media may be stored externally.
Most of the public data sets such as Facebook, Twitter, Pinterest, financial markets data, weather data, genome datasets and industry-specific data live in the cloud and it becomes more cost-effective for the enterprise to analyze this data where it resides, in the cloud itself. With the increase in the amount of unstructured data available, more value can be extracted from big data when structured data sets are merged with the unstructured data and analyzed to gain competitive advantage.
Drivers for adoption of big data in the cloud
Cost reduction:Cloud computing offers a cost-effective way to support big data technologies and the advanced analytics applications that can uncover the hidden insights in the data and drive business value. Big data environments require clusters of servers to support the tools that process the large volumes, high velocity, and varied formats of big data. IT organizations should look to cloud computing as the infrastructure for their big data initiative because the cloud’s pay-per-use model will reduce their costs.
Reduce overhead: Various hardware components and system integration are required for any big data solution implementation. With cloud computing, infrastructure management can be automated, reducing complexity and improving the IT team’s productivity.
Rapid provisioning/time to market: Provisioning servers in the cloud is as easy as buying something on the Internet. Big data environments can be scaled up or down easily based on the processing requirements. Faster provisioning is important for big data applications because the value of the data is rapidly reduced as time goes by.
Flexibility/scalability: Big data analysis, especially in the life sciences industry, requires huge compute power for a short time. For this type of analysis, servers need to be provisioned in minutes. This kind of scalability and flexibility can easily be achieved in the cloud.