The Future of Public Cloud Storage for Big Data


Insight Platform-as-a-Service (PaaS) for big data in the public cloud is set to replace on-premise storage. According to Forrester data, cloud storage for big data is the most sought after technology. In fact, global spending on big data solutions through cloud subscriptions is estimated to grow more than seven times faster than on-premise subscriptions. Here are some interesting points.

Public Cloud Innovations to Burry On-premise Options

Just as happened with Moore’s law when silicon chips met transistors, public cloud and big data are creating exponential effects. A recent research predicts that public cloud prices for big data processing and storage will decrease by half every few years while processing power will double. Public cloud skeptics predict that costs of big data storage in the public cloud will be the same or increase slightly in 10 years. however, this not true as the costs are expected to decrease significantly.

On the other hand, the costs of upgrading big data software will increase. in a few years, Hadoop data lake will need to be upgraded. Right now, data scientists are preferring multiple versions of Spark, indicating the beginning of on-premise headaches. This can only get worse as Google, Amazon Web Services (AWS) and Microsoft pursue a serverless strategy.

Technologies such as quantum computing and artificial intelligence will most likely be consumed through the public cloud. For example, Google and Quantum vendor D-Wave intend to let customers use quantum computing in Tensor Flow. This means that firms that are not ready will find it hard to remain competitive.

Insight PaaS is Advantageous Compared to Other Cloud Solutions

To help its clients in their data journey to the cloud, ZDNet completed a Forrester Wave Evaluation of eight big data vendors: Amazon Web Services, 1010Data, GoodData, Databricks, Microsoft, Google, IBM and Qubole. It was found that all of them offer insight PaaS. Forrester defines insight PaaS as an integrated set of data analytics, management and insight application development and management components offered as a platform that is not owned or controlled by an enterprise. The advantages of insight PaaS over other big data solutions that advertise cloud deployment options are immense. Most importantly, insight PaaS vendors have gone beyond their roots to offer insight application development platforms among other big, exponential advantages discussed below.

Firstly, Insight PaaS are better than on-premise at managing and accessing large, complex data sets. A good example is Google’s BigQuery which lets developers query petabytes in milliseconds. All you need to do is make a few decisions about schema design and cluster size.

Secondly, PaaS are better at updating and evolving applications that deliver insight at the moment of action. For instance, GoodData’s insight application life-cycle management features allow customers to build and optimize solutions without extensive query tuning or code debugging.

Finally, insight PaaS are better than on-premise in updating and upgrading complex technology. A perfect example is Databricks which makes current and older versions of Spark accessible by URL at the same time. This means that existing applications can continue using older versions until they are ready to migrate, at which time, they simply point to the new version without the need for server upgrade.