Time to Go Cloud: Hype to Reality
A plum of happiness spreads over the face of an IT consultant when an inquisitive customer enquires about the cloud computing. A sense of faith is bestowed upon the consultant by customer to get the right solution, and perhaps, the today's solution. In the same breath, the consultant embarks upon the difficult task of providing IT solution with right platforms and Apps drawn from the mushrooming industry with a large number of cloud-based IT providers, some claiming 'go cloud' to anything they offer while others having complicated proprietary solutions. Both are bad for customer, and consequently dent the credibility of the consultant.
On the other hand, the hype of cloud computing has the capability to drive the decision-making at board meetings. A single-person enterprise finds it lucrative to settle with a few cloud providers to meet their different business needs to avoid upfront investment in IT. Even common people hook to the cloud for news, entertainment, education and research.
Yes, we have discussed the hype in the Chapter 1. In fact, the hype surrounding cloud computing has hindered its rightful adoption and exploitation. In this chapter, we shall review the famous Hype Cycle, and predictions and happenings related to the cloud computing paradigm. We shall draw parallels to a historic phenomenon of automobile industry, and shall then review the changes to IT landscape in the past several decades to understand the realistic shift towards adopting this novel computing paradigm.
The Famous Hype Cycle
Hype Cycle (figure 3.1) is a graph of the visibility of a technology in the course of time.
It is a general phenomenon that a technology, when appears in market, generates a greater hype or unusual attention than many times justified, taking the visibility of the technology to its peak, known as the peak of inflated expectations, from the initial appearance or point of technology trigger. However, as time proceeds, the technology gets feedback from users; and further innovations take place due to efforts by providers and makers of the technology to meet users’ demands thereby bringing the technology to a practical stage of rightful adoption and appropriate usage. This is known as the plateau of productivity.
However, the interesting aspects of this transition of technology from high visibility to a realistic regime of successful adoption and productivity lie prior to the final realization. In the course of occurrence of such transition, the visibility dives down initially exerting pressure on providers and makers of the technology to innovate rapidly. This phase is known as the trough of disillusionment after the expectation from users falls as rapidly as it went up as innovations struggle to match. Then industry innovates; and it gradually pushes the acceptance of the technology among users slowly, taking the visibility through the slope of enlightenment and reaching the plateau of productivity finally.
Cloud Computing is going through such a hype cycle. As per “Gartner’s Hype Cycle for emerging technologies published in 2011”, cloud computing was nearing the trough of disillusionment (figure 3.1); and the next two to five years will see its consolidation among users bringing it into the mainstream adoption and full exploitation. In fact, cloud computing is set to traverse the similar path like any other technology due to the mismatch in innovations required at the backdrop of unusual expectations generated among users.
Here, we shall not argue about the exact timeline as formulated in the Gartner’s predictions. We shall rather view the scenario holistically with competing factors of users’ demands and innovations that will gradually guide the rightful adoption of this technology.
User Demands and Innovations
It is not a solitary situation for cloud computing, and not even for IT, where user demands drives innovations. In fact, the complex interplay of these two competing forces determines the fate of a technology. An optimistic scenario occurs when innovations by providers match the demands from users. On the other hand, adoptability suffers when reverse thing happens. While discussing the Hype Cycle in the previous section, which describes the entire lifecycle of a successful technology, we have seen the crest and trough of visibility of the technology, and later a productive phase of its adoption by target users.
Now let us look into these two competitive forces, user demands and innovations, in detail (figure 3.2) that continually balance the industry scenario with respect to the adoption and use of a particular technology.
It is a complex process, after all, as the verdict is the result of a collective phenomenon. Consumers settle for economics always, in fact, for the best possible option. When the technology is nascent, the cost of a product using the technology is high, and thus a premium price tag is not surprising. However, widespread adoption can only happen with cost falling back into an affordable limit. And, the limit is determined by different economic indices.
Apart from economics, other factors like Quality of Service (QoS) and availability are critical in determining the fate of a product using the technology in question. QoS is a broad term encompassing several different aspects of a service including usability, and guarantee of service, etc. In layman’s term, usability pertains to how easy it is to use a particular technology or a product using this technology, and its different features. Guarantee of service requires Service Level Agreement (SLA) over a mere advertised claim of service provider about meeting users’ demands in applicable situations. On the other hand, availability of a product is realized if users can access and use the product whenever and wherever its demand arises.
The competitiveness of a technology or of a product using the technology is ascertained based on how well it serves the above parameters. To understand the adoptability of a product, it is essential to get feedback from users when they use the technology. And, feedback acts as the driver for the future innovations by the providers and makers. On the other hand, innovations rely on parameters like technology, standard compliance, and agility of the product to different possible usage scenarios.
To elaborate the above concepts and to consolidate our thoughts further, let us look beyond IT, and scan through the historic happenings in automobile industry since late 19th century which has seen an evolution spanned over a century (figure 3.3).
During the late 19th century, thousands of automobile design and manufacturing units were operating. However, the process of designing and manufacturing was mostly manual. The production cycle was long and costly. As the technology and methods were yet to mature, imparting skills to large workforce and upgrading their skills continually was a difficult and costly proposition too. Only a few automobiles could be produced from a manufacturing plant every year. Consequently, the costs of automobiles attracted high price tags; and were thus accessible to ruling establishments and economically privileged population.
As expectations from business opportunities went up, entrepreneurs started new ventures along the supply line starting from getting raw materials, actual production and building of manufacturing tools to delivering the automobiles to target users. Also, engineers, technicians, creative people and even common workers got attracted towards this industry for higher remunerations. This resulted in larger manpower pool, and created a sense of chaos in the absence of standard process of production across different companies. The situation led to disappointments when the expectations did not match with the ground realities.
Here, it is noteworthy to mention about a major event. This was when World War I started in Europe. The requirements of automobiles to supply weapons and support materials to soldiers at battlefront increased. More tanks and other vehicles were needed, and were expected to be available on demand, and, of course, to be very usable. Giving a mundane example, let us evaluate a scenario when bolt of a vehicle needed replacement in the enemy territory. It was a choice between contacting the original vendor, and adopting a solution where the accessories are standardized and available elsewhere irrespective of the vendor who manufactured those. Innovations in this front led to adopting a standardization process that was later provided impetus to ISO standards.
The need for large number of vehicles triggered the requirement for quicker production process. This resulted in innovations in automobile engineering, and in the process of production. Today, multiple vehicles come out of a modern plant every minute; and consequently, there is significant reduction in cost and enhancement in quality. Moreover, common usages and fast changing lifestyles of users brought the subtle parameter of agility into the design, manufacturing, delivery and re-engaging users’ feedback into the next cycle of production process.
The above example vindicates our proposition about requirement for lower cost, standardization of product, improved quality and faster production process. It also illustrates the approach to technology development leading to fruitful acceptance by users.
Can we apply the same paradigm into our understanding of the changing scenarios in IT? The answer is ‘yes’. To substantiate this, let us look again into history, though not as early as the previous example.
Phenomenon Repeats
Let us review the evolution of IT briefly, pattern of its adoption by users and the consequent influence on innovations. Well, we must note that feedback cycle is faster today; and the technology must keep pace to this shrinking cycle to mature, to be productive, and above all, to be adopted successfully among its target users.
Information Technology started presenting a complex landscape due to its impact not only on every sphere of business, but also due to the radical changes in human behavior and the multitude of dependencies on different other technologies. Though these complexities have challenged the entire human fraternity, the quicker adoption and ubiquitous usages have propelled the need for better economic proposition and high level of simplicity in its usage.
Before 1990, application of IT was largely limited to using LAN-based software system where a huge (in fact, it was huge in size) and all-powerful machine was running the software, and was meeting all computing needs of the enterprise. Consumers of IT used to take help of dumb terminals to access the software system on this large central machine. That was mainframe era.
In 1990's, personal computers (PC) became popular; and those dumb terminals were replaced by these new powerful machines sitting near the users. Thus started the PC era, when numerous desktop software applications were written and used in a big way. Of course, LAN computing continued to be the carrier of enterprise-wide IT.
Towards the end of last century, widespread internet usage showed new ways to use software applications. Many web and mobile applications were developed. New scripting languages, database systems backed by open source movement played a significant role in keeping the momentum of innovations up in IT. Methods of collaboration and information management saw transformations with tactically planned deployment of geographically distributed IT resources that would reside partly within the LAN environment, and the remaining on web servers. In such situations, web applications mostly acted as extensions to the legacy applications, and were usually treated light-weight. However, the apparently separated islands of LAN ecosystem found channels of the Internet to work in tandem with one another. The barrier was broken.
With this new opportunity, many new possibilities emerged. This period saw faster innovations in different fronts. The entire cycle of IT fulfillment incited many new developments. Datacenters adopted energy efficient technologies, and invested in the automation of system administration and monitoring. Users could afford servers at lower cost. Open source scripting languages like PHP and database systems like MySQL gained immense popularity among application developers giving rise to a large community, and numerous innovative and popular applications. This enabled greater competition, thereby reducing the cost of application development.
IT became ubiquitous, and expectations went high regarding economics, QoS and availability. On the other hand, the critical issues and shortcomings became more prominent with increased adoptability of IT, and the availability of applications for almost every usage scenario. Business users (and enterprises) continued (or even increased) their dependence on developers to integrate various solutions, and to manage it on-premise. The cost of hiring developers subsequently increased; and it was unsustainable as an individual developer or freelancer may not be as reliable as an IT service company due to the possibility of the changes in their priorities or careers, or even location change apart from many other extreme possibilities. On the other hand, the option of hiring enterprises could only come at a premium price as service companies had to manage all these risks. It was no win-win situation though both end-users and the service providers accepted the situation and marched ahead. And, everybody waited for the next major innovation.
Onsite software and user-specific custom software development became untenable due to the higher cost of development and management. This phenomenon, in fact, triggered the adoption of SaaS model. Even though this provided limited scope for customization of applications, consumers were not bothered about initial (upfront) cost of software, its upgrade cost, or even high maintenance cost and risks. They could rather use software applications with minimal subscription fees while the service provider owned these. This was a win-win situation where provider did not have to maintain multiple copies of application at multiple locations (without direct control over the system as those were sitting on customer's premises), thus reducing the overall cost of management of application life-cycle.
Different software applications with their own architecture, and data storage and manipulation mechanism offered an uphill task for inter-communications. This triggered the need for a standard way for applications to talk to one another, i.e., to access data from one another. The result is the popular acceptance of standard protocols for Application Programming Interface (API).
The above development was important as it encouraged thinking of software applications as a conglomerate of disparate systems communicating with each other through standard API – the concept of Web Service was born. Web service evolved as an encapsulation of data and access methods adhering to standard protocols like SOAP, XML-RPC and ReST, and exchanging data with one another through standard formats like XML and JSON. We shall learn more about these standards in Chapter 4 when we define and discuss software technologies in today’s context.
Gradually, we entered into the era of cloud computing. It is not a single step development from the earlier stage; rather, it was multi-fold. However, things which remained constant through different periods are the mutually engaging parameters like user demands and innovations.
Now, we need to understand the inevitability and essence of cloud computing with respect to these fundamental parameters, user demands and innovations, in the following section.
Economics of Accessing Big Data
We shall confine our discussion to three prominent demands from users, and the innovations driven by these: big data, universal access, and economies of scale (figure 3.5). These can be called as the major drivers of cloud computing today. On the other hand, the paradigm of cloud computing can be called as the result of economics of accessing big data universally.
With the increasing usage of web services, websites could be scalable easily with least redundancy of development and maintenance efforts, and without promoting a monolithic architecture. As Web 2.0 wave spread, content piled up at websites. This created a challenge to manage data of large scale. Data size of Terabyte then became usual; and went up to Petabyte, and Exabyte, scales easily. In the first chapter of this book, we have discussed about large video content, such as movies and surveillance camera outputs, being handled in day-to-day life by us – these are also about data of these scales. To give another example, Walmart handles more than a million customer transactions every hour; and this data, having an approximate size of 2.5 Petabytes, is equivalent of 167 times the total information contained in all the books in the US Library of Congress! The data of such a magnitude are known as big data.
The name ‘big data’ cannot be confused with just large size or volume of data; rather, it has other quantitative attributes like velocity (the speed of data storage and retrieval) and variety (the varieties of data that are created and used). The traditional Relational Database Management Systems (RDBMS) and associated tools, which were being used to manage data since long, fail to provide adequate means to handle big data. Thus an alternate data management system, known as noSQL (not only SQL), came into the picture. We shall differ the formal definition of big data and a technical discussion on this topic until the next chapter.
On the other hand, we are increasingly going mobile in our personal and professional lives. In such situations, we expect to acquire the capability to access our data and any public information from anywhere, at any time and with any device having a network connectivity. This expectation of having a universal access to big data signifies the changing mindset, and drives innovations. We shall discuss on technologies related to user devices and cloud clients in Chapter 4.
Moreover, the availability of big data and accessing these universally have their influences on analytics, the faculty of developing optimal recommendations based on insights into data. Analytics use statistical models, and analysis against existing or simulated future data or both to arrive at decision making. This approach is popularly known as pattern-based strategy. Each cycle of analytics is based on seeking a pattern from the available data, modeling the impact, and finally, adapting according to the patterns.
A few years back, this was a separate process of drawing data from slower tape drives, and processing these through complex programs at periodic intervals to seek patterns or trends. It was like answering questions like what happened and why it happened – an era of Old Analytics.
However, we now expect data to be available anytime, in real time. We cannot afford to store our data in slower devices, and thus, to reconcile with our traditional queries while dealing with analytics. We are more interested to know what is happening now, and what can happen next – a shift from the thinking of Old Analytics. These renewed expectations have paved the way for pattern-based strategy to go real-time, in business or elsewhere.
Real-time analytics demands the access and processing of big data for any meaningful purpose, and requires large-scale computation and highly available storage that must be economic too. The option of deploying a single big server with larger computing power or a single disk with larger storage to match this scale (known as vertical scaling or scaling up) becomes untenable as systems would quickly reach their limits of technical-feasibility. The alternative approach is to have a distributed environment where usual low-end servers and storage devices are linked horizontally (horizontal scaling, or scale-out) to create a mammoth computing system. Well, we are talking about cloud computing!
Today, people emphasize on reducing the time gap between a plan and its materialization or Time to Value (TTV), and the Total Cost of Ownership (TCO) while making a decision on IT investment. Thus, the subscription-based usage model has become popular as it facilitates instant use of IT resources, eliminates the capital expenditures (CapEx), and curtails operational expenditures (OpEx).
We have seen the inevitability of cloud computing in the changed scenario involving big data. Basically, this can be understood as achieving the economy of scale for providing universal access to big data that we incessantly generate these days.
Growth in Cloud Adoption
With a strong advocacy for cloud computing based on ground realities of today, it would be justified to know about the adoption of this computing paradigm in practice.
(Ref: Article by Balakrishna Narasimhan and Ryan Nichols in March 2011 Issue of Computer, IEEE Computer Society. The Horizontal Axis Denotes the Percentage of Respondents)
The above illustration has been used to denote the trend in a symbolic way. In fact, there are many surveys with distinct trends that establish faster adoption, even substantial increase in the rate of adoption, and above all, a very positive scenario. Let us now scan through some important data from different surveys that provide insights into the adoption of cloud computing in various spheres:
- As a report of Redshift Research suggests, 54% of business and 27% of government prefer private cloud solution. Public cloud adoption would be 9% and 17% respectively; similarly, hybrid cloud adoption would be 5% and 7% respectively.
- CDW 2011 Cloud Computing Tracking Poll brings out figures for IT adoption in the USA as 21% for SMEs, 37% for large businesses, 29% for federal government, 23% for state and local governments, 30% for healthcare, 27% for schools and 34% for higher education.
- Asia-Pacific Business and Technology Report informs that the cloud computing market in Japan will grow to U.S.$29.2 billion by 2015.
- SMB Cloud Adoption Study Dec 2010 by Microsoft points that 39% of SMEs would be paying for one or more cloud services within three years.
We shall discuss more on the migration to cloud, and adoption scenarios in Chapter 5. Let us now proceed to summarize the success path of cloud computing and plan our next discussion topics.
Following the Path of Success
Through the current chapter, we explored the hype and realities around the paradigm of cloud computing from the perspective of collective interplay between demands from the vast group of IT users and innovations across the mushrooming industry supporting the growth. We discussed the present scenario from the point of view of big data, universal access requirements and economies of scale.
While it is a happy situation all along with optimistic examples and analyses, it is imperative to learn the key technologies that are responsible for this renewed perspective. We shall discuss these in the next chapter before we again return to applications of this new paradigm for remaining part of the book.