tag:blogger.com,1999:blog-54417929117081402712024-03-13T03:50:21.395-07:00. _ . . _ . . _ . .Stratus StrategemDavid Bernsteinhttp://www.blogger.com/profile/01143001563622774579noreply@blogger.comBlogger7125tag:blogger.com,1999:blog-5441792911708140271.post-14913625304643936572013-04-12T12:56:00.001-07:002013-04-16T07:39:25.222-07:00"Intercloud" - Not all the same! Federation versus Multicloud<h2>
<span style="background-color: white; color: blue; font-family: Times, 'Times New Roman', serif;">Types of Intercloud: Federation and Multi-Cloud</span></h2>
<div style="background-color: white;">
<span style="color: blue; font-family: Times, Times New Roman, serif;">Lately, more and more people are talking about cloud interoperability.</span><br />
<span style="color: blue; font-family: Times, Times New Roman, serif;"><br /></span><span style="color: blue; font-family: Times, Times New Roman, serif;">Unfortunately it's become almost a "marketing war" - who can gain the most momentum with "their" approach, the soonest. There's two camps, and it's really confusing to understand - they are actually different!</span><br />
<h2>
<span style="color: blue; font-family: Times, Times New Roman, serif;">Executive Summary</span><span style="color: blue; font-family: Times, 'Times New Roman', serif;"> </span></h2>
<h3>
<span style="color: blue; font-family: Times, 'Times New Roman', serif;">Multi-Cloud are User API's</span></h3>
<div>
<span style="color: blue; font-family: Times, Times New Roman, serif;">One approach which has the support of OGF and also roughly equivalent is the approach the FP7 Helix Nebula project has taken is called the <b>"Multi-Cloud" approach. </b>This is ideal in situations where there is a user (like a Grid or HPC user) wanting to access several clouds to fulfill his/her computing requirements. Generally this is <b>for academic and research computing </b>constituencies as this technology architecture is "from the User into the Network" type of "explicit demand for resources" where the user is very specifically controlling the computing they want. This also can work for companies who absolutely must access different public clouds and have the IT staff to operate a specific gateway box, or can write code, to do so.</span><br />
<span style="color: blue; font-family: Times, Times New Roman, serif;"><br /></span><span style="color: blue; font-family: Times, Times New Roman, serif;">Basically, Multi-cloud "leaves it to the user", even if there is a layer that doesn't make it seem so.</span></div>
<div>
<h3>
<span style="color: blue; font-family: Times, Times New Roman, serif;">Federation is like the Internet</span></h3>
</div>
<div>
<span style="color: blue; font-family: Times, Times New Roman, serif;">Another approach which has the support of several Tier 1 Telcos, several commercial Labs, different FP7 projects including those from universities in Naples, Amsterdam, and Helsinki, is called the <b>"Intercloud" approach</b>, which is the subject of the work of the IEEE P2302 Working group as well as the IEEE Intercloud Testbed. This is ideal for large<b> public/commercial Mobile and Internet scenarios, or Enterprise</b> cloud deployments in conjunction with Telco MPLS/VPN. This technology architecture is "from the Network to the User" with "implicit demand for resources" where the user is unaware of what is happening behind the scenes. Think of it as similar technology to Mobile Roaming, or the Public Internet ability for any browser to access any web site on the Internet. </span><br />
<span style="color: blue; font-family: Times, Times New Roman, serif;"><br /></span><span style="color: blue; font-family: Times, Times New Roman, serif;">Basically, Federation "makes it invisible", just like the Internet or Phone network.</span></div>
<h2>
<span style="color: blue; font-family: Times, Times New Roman, serif;">A Closer Look</span></h2>
<div>
<span style="color: blue; font-family: Times, Times New Roman, serif;">Professor Raj Buyya from Melbourne University has produced some great explanations for this in a recently published paper. I extract from his work to <span style="text-align: justify;">dive into what really differentiates between these - </span><b style="text-align: justify;">Federation</b><span style="text-align: justify;"> and </span><b style="text-align: justify;">Multi-Cloud</b><span style="text-align: justify;">.</span></span></div>
<div>
<div class="MsoNormal" style="text-align: justify;">
<span style="color: blue; font-family: Times, Times New Roman, serif;"><o:p></o:p></span></div>
<div class="MsoNormal" style="text-align: justify;">
</div>
<ul>
<li><span style="color: blue; font-family: Times, Times New Roman, serif;"><b style="text-indent: -0.25in;">A Federation</b><span style="text-indent: -0.25in;"> is achieved when a set of cloud providers voluntarily interconnect their infrastructures in order to allow sharing of resources among each other.</span></span></li>
<li><span style="color: blue; font-family: Times, Times New Roman, serif;"><b style="text-indent: -0.25in;">Multi-Cloud</b><span style="text-indent: -0.25in;"> denotes the usage of multiple, independent clouds by a client or a service. Unlike a federation, a multi-cloud environment does not imply volunteer interconnection and sharing of providers’ infrastructures. Clients or their representatives are directly responsible for managing resource provisioning and scheduling.</span></span></li>
</ul>
<span style="color: blue; font-family: Times, Times New Roman, serif;"><span style="text-align: justify;">Both federations and multi-clouds are types of Inter-Clouds!</span></span></div>
<h3>
<span style="color: blue; font-family: Times, Times New Roman, serif;">Federations are like the Internet, or the Phone System</span></h3>
<div>
<span style="color: blue; font-family: Times, Times New Roman, serif;">In a Federation, the clouds have decided to join together and create mechanisms which are largely transparent to the users. <b>Connections between clouds are made underneath via special protocols from cloud to cloud.</b> Actually, it's quite hard to view the independent clouds as independent any more! For examples of Federations, think of the Internet, where any browser can access any website - this is enabled by DNS, routing protocols, peering/exchange agreements - set up by the IP transit providers in advance, and transparent to the users. In the phone network, standards for interconnections of phone companies utilize SS7 networking, standardized numbering plans, and origination/termination agreements to result in a system where any phone can dial any other phone worldwide. The mobile phone system adds a roaming layer on top of this providing and even more comprehensive notion of Federation.</span></div>
<h3>
<span style="color: blue; font-family: Times, Times New Roman, serif;">Multi-clouds are like Social Networks, or like Calling Cards</span></h3>
<div>
<span style="color: blue; font-family: Times, Times New Roman, serif;">In a Multi-cloud, the underlying separate clouds are still quite visible as separate clouds. C<b>onnections between clouds are made via over the top via user APIs.</b> In other words the user has placed a mechanism - a box or a software API - in front of the multiple clouds (unbeknownst to them) which makes enables that user to view and use them all at once. It's like a Social Network of today. When you participate in one social network, it's completely separate from another social network. You might be members of many social networks, but they are "walled gardens" and don't have any substantial interoperability across them. If you want a "merged" friends or contact list, you must use a utility perhaps found in your email program, or an program deigned to "aggregate" social networks. It will use the different API's of the social networks to access each one, using your credentials on each, and provide a layer merging together the most important (say "contacts") features of each Social network. Another example is the Calling Card. In the phone network, you may choose not to use the Federation capabilities, perhaps because they are too expensive (direct dial long distance and mobile roaming can be expensive!) for example. In this case you can use a Calling Card, where you manually use the phone network at hand, say through a "toll free" mechanism, to connect to your Calling card, and then through that system, you manually direct it to dial the end phone. In this way you are using the "user API's" of the phone system (phone numbers) to construct an over the top end to end connection.</span></div>
<div class="MsoNormal" style="text-align: justify;">
</div>
<h2>
<span style="color: blue; font-family: Times, Times New Roman, serif;">Hybrid Cloud is not Intercloud<o:p></o:p></span></h2>
<div class="MsoNormal">
<span style="color: blue; font-family: Times, Times New Roman, serif;">Another term used is <b>Hybrid Cloud</b>. It has been defined as a
composition of two or more different cloud infrastructures - e.g a private and
a public cloud. Thus a hybrid cloud is a type of a Multi-Cloud that connects
miscellaneous clouds in terms of their deployment models. Often hybrid clouds
are used for cloud bursting - the usage of external cloud resources when local
ones are insufficient.</span></div>
<div class="MsoNormal" style="text-align: justify;">
</div>
<h2>
<span style="color: blue; font-family: Times, Times New Roman, serif;">Intercloud Brokers/Exchanges<o:p></o:p></span></h2>
<div class="MsoNormal">
<span style="color: blue; font-family: Times, Times New Roman, serif;">The term Inter-Cloud <b>broker</b> or <b>exchange</b> has been used with different meanings. In most cases it
means a service that acts on behalf of the client in order to provision
resources and deploy application components. A Cloud broker or exchange is an
automated entity with the following responsibilities:</span></div>
<div class="MsoNormal">
</div>
<ul>
<li><span style="text-indent: -0.25in;"><span style="color: blue; font-family: Times, Times New Roman, serif;">Automatic resource provisioning and management
across multiple clouds for a given application. This would include allocation
and de-allocation of resources (e.g. VMs and storage).</span></span></li>
<li><span style="text-indent: -0.25in;"><span style="color: blue; font-family: Times, Times New Roman, serif;">Automatic deployment of application components
in the provisioned resources.</span></span></li>
<li><span style="text-indent: -0.25in;"><span style="color: blue; font-family: Times, Times New Roman, serif;">Scheduling and load balancing of the incoming
requests to the allocated resources.</span></span></li>
</ul>
<h2>
<span style="text-indent: -24px;"><span style="color: blue; font-family: Times, Times New Roman, serif;">Intercloud Architectural and Topological Taxonomy</span></span></h2>
<div>
<span style="text-indent: -24px;"><span style="color: blue; font-family: Times, Times New Roman, serif;">Now, let's follow Prof. Buyya's scientific classification methodology to better understand all this.</span></span></div>
<div>
<span style="text-indent: -24px;"><span style="color: blue; font-family: Times, Times New Roman, serif;"><br /></span></span></div>
<span style="text-indent: -24px;"><span style="color: blue; font-family: Times, Times New Roman, serif;">We can broadly classify Inter-Clouds as:</span></span><br />
<ul>
<li><span style="color: blue; font-family: Times, Times New Roman, serif;"><b style="text-indent: -0.25in;">Volunteer
federation</b><span style="text-indent: -0.25in;"> - when a group of cloud providers voluntarily collaborate with
each other to exchange resources. This type of Inter-cloud is mostly viable for
governmental clouds, private cloud portfolios, or a public cloud system.</span></span></li>
<li><span style="color: blue; font-family: Times, Times New Roman, serif;"><b style="text-indent: -0.25in;">Independent</b><span style="text-indent: -0.25in;">
- when multiple clouds are used in aggregation by an application or its
broker/exchange. This approach is essentially independent of the cloud provider
and can be used to utilize resources from both governmental and private
clouds. Another term used for this is Multi-Cloud.</span></span></li>
</ul>
<span style="color: blue; font-family: Times, 'Times New Roman', serif;">From an architectural perspective </span><i style="color: blue; font-family: Times, 'Times New Roman', serif;">Volunteer federations</i><span style="color: blue; font-family: Times, 'Times New Roman', serif;"> can be further
classified as:</span><br />
<div class="MsoNormal">
</div>
<ul>
<li><span style="color: blue; font-family: Times, Times New Roman, serif;"><b style="text-indent: -0.25in;">Peer-to-Peer</b><span style="text-indent: -0.25in;">
- in the architectures from this group clouds communicate and negotiate
directly with each other without mediators.</span></span></li>
<li><span style="color: blue; font-family: Times, Times New Roman, serif;"><b style="text-indent: -0.25in;">Centralized</b><span style="text-indent: -0.25in;">
- in every instance of this group of architectures there is a central entity
that either performs or facilitates resource allocation. Usually this central
entity acts as a repository where available cloud resources are registered, but
may also have other responsibilities like acting as a market place for
resources.</span></span></li>
</ul>
<div class="MsoNormal">
<span style="color: blue; font-family: Times, Times New Roman, serif;">From an architectural perspective
<i>Independent Multicloud</i> developments
can be further classified as:</span></div>
<div class="MsoNormal">
</div>
<ul>
<li><span style="color: blue; font-family: Times, Times New Roman, serif;"><b style="text-indent: -0.25in;">Services</b><span style="text-indent: -0.25in;">
- application provisioning is done by a service which can be hosted either
externally or in-house by the cloud clients. Most such services include broker
components in themselves. Typically application developers specify an SLA or a
set of provisioning rules and the service performs the deployment and execution
in the background, in a way respecting these predefined attributes.</span></span></li>
<li><span style="color: blue; font-family: Times, Times New Roman, serif;"><b style="text-indent: -0.25in;">Libraries</b><span style="text-indent: -0.25in;">
- often custom application brokers that directly take care of provisioning and
scheduling application components across clouds are needed. Typically such
approaches make use of inter-cloud libraries that facilitate the usage of
multiple clouds in a uniform way.</span></span></li>
</ul>
<span style="color: blue; font-family: Times, 'Times New Roman', serif;">The whole taxonomy is depicted below, showing example projects falling into each category:</span><br />
<div class="MsoNormal">
<span style="color: blue; font-family: Times, Times New Roman, serif;"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-TXzQEY1hAuE/UWhjsRsU7HI/AAAAAAAAAA4/KYZBSmrPdrA/s1600/Architectural+Classification+of+Interclouds.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><span style="color: blue; font-family: Times, Times New Roman, serif;"><img border="0" height="225" src="http://2.bp.blogspot.com/-TXzQEY1hAuE/UWhjsRsU7HI/AAAAAAAAAA4/KYZBSmrPdrA/s400/Architectural+Classification+of+Interclouds.png" width="400" /></span></a></div>
<div class="separator" style="clear: both; text-align: center;">
<span style="color: blue; font-family: Times, Times New Roman, serif;"><br /></span></div>
<div class="MsoNormal">
<span style="color: blue; font-family: Times, Times New Roman, serif;"><br /></span></div>
<div class="MsoNormal">
</div>
<div class="MsoNormal">
<span style="color: blue; font-family: Times, Times New Roman, serif;">And we can consider the topology
of the different Inter-Cloud architectures as follows:<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="color: blue; font-family: Times, Times New Roman, serif;"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-3pA5OqA1Dd8/UWhj4S6JXhI/AAAAAAAAABA/tMzvEjLwA1I/s1600/Topologies+of+Different+Intercloud+Architectures.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><span style="color: blue; font-family: Times, Times New Roman, serif;"><img border="0" height="318" src="http://2.bp.blogspot.com/-3pA5OqA1Dd8/UWhj4S6JXhI/AAAAAAAAABA/tMzvEjLwA1I/s400/Topologies+of+Different+Intercloud+Architectures.png" width="400" /></span></a></div>
<div class="separator" style="clear: both; text-align: center;">
<span style="color: blue; font-family: Times, Times New Roman, serif;"><br /></span></div>
<h2>
<span style="color: blue; font-family: Times, Times New Roman, serif;">How to Choose the Right Intercloud Architecture</span></h2>
<div>
<div class="MsoNormal">
<span style="color: blue; font-family: Times, Times New Roman, serif;">To answer this, lets look to a formal definition of
Inter-cloud computing (from the GICTF):<o:p></o:p></span></div>
<div class="MsoNormal" style="margin: 0in 0.5in 0.0001pt;">
<br /></div>
<div class="MsoNormal" style="margin: 0in 0.5in 0.0001pt;">
<span style="color: blue; font-family: Times, Times New Roman, serif;">“A cloud model that,
for the purpose of guaranteeing service quality, such as the performance and
availability of each service, allows on-demand reassignment of resources and
transfer of workload through a [sic] interworking of cloud systems of different
cloud providers based on coordination of each consumers requirements for
service quality with each providers SLA and use of standard interfaces.”</span></div>
<div class="MsoNormal" style="margin: 0in 0.5in 0.0001pt;">
<span style="color: blue; font-family: Times, Times New Roman, serif;"><br /></span></div>
<div class="MsoNormal" style="margin: 0in 0.5in 0.0001pt;">
</div>
<span style="color: blue; font-family: Times, Times New Roman, serif;">Which to choose?</span></div>
<div>
<span style="color: blue; font-family: Times, Times New Roman, serif;"><br /></span></div>
<div>
<span style="color: blue; font-family: Times, Times New Roman, serif;">The <b>"<span class="il" style="background-color: #ffffcc;">Multi</span>-<span class="il" style="background-color: #ffffcc;">Cloud</span>" approach, </b>which is the subject of the work of the OGF, several academic projects, and also proprietary "CloudSwitch" like boxes for enterprises, is ideal in situations where there is a user (like a Grid or HPC user) wanting to access several clouds to fulfill his/her computing requirements. Generally this is <b>for academic and research computing </b>constituencies as this technology architecture is "from the User into the Network" type of "explicit demand for resources" where the user is very specifically controlling the computing they want.</span></div>
<div>
<span style="color: blue; font-family: Times, Times New Roman, serif;"><br /></span></div>
<div>
<span style="color: blue; font-family: Times, Times New Roman, serif;">The <b>"Intercloud" approach</b>, which is the subject of the work of the IEEE P2302 Working group as well as the IEEE Intercloud Testbed, is ideal for large<b> public/commercial Mobile and Internet scenarios, or Enterprise</b> <span class="il" style="background-color: #ffffcc;">cloud</span> deployments in conjunction with Telco MPLS/VPN. This technology architecture is "from the Network to the User" with "implicit demand for resources" where the user is unaware of what is happening behind the scenes. Think of it as similar technology to Mobile Roaming, or the Public Internet ability for any browser to access any web site on the Internet. </span></div>
<div class="MsoListParagraphCxSpFirst" style="text-align: justify; text-indent: -0.25in;">
<!--[if !supportLists]--><span style="color: #222222;"><o:p></o:p></span></div>
<div class="MsoListParagraphCxSpLast" style="color: #222222; text-align: justify; text-indent: -0.25in;">
<o:p></o:p></div>
</div>
David Bernsteinhttp://www.blogger.com/profile/01143001563622774579noreply@blogger.com13tag:blogger.com,1999:blog-5441792911708140271.post-54978981767849498952012-11-17T12:40:00.000-08:002012-11-17T12:40:03.714-08:00Carrier Cloud Strategies: Advice from GigaOM Pro<br />
<div style="background-color: white; color: #3e444c; font-family: DINLight, Helvetica; font-size: 11pt; line-height: 16pt; margin-bottom: 12pt; padding: 0px;">
Last month, Jo Maitland at GigaOM Pro referenced the work Cloudscaling is doing in a report titled, “<a href="http://pro.gigaom.com/2012/07/how-carriers-can-catch-up-in-the-cloud-race/" style="-webkit-transition: all 0.3s ease; color: #b21f24; margin: 0px; padding: 0px; text-decoration: initial;">How carriers can catch up in the cloud race</a>.” (subscription required, and worth it) Her work provides practical advice for Communication Service Providers (CSPs) asking this very question.</div>
<div style="background-color: white; color: #3e444c; font-family: DINLight, Helvetica; font-size: 11pt; line-height: 16pt; margin-bottom: 12pt; padding: 0px;">
How did the CSPs let themselves get in the situation of needing to catch up in the first place?</div>
<div style="background-color: white; color: #3e444c; font-family: DINLight, Helvetica; font-size: 11pt; line-height: 16pt; margin-bottom: 12pt; padding: 0px;">
Looking back, most carriers had hosting and co-location offerings, which provided a level of security, reliability, and performance which was usually a notch above the same type of offering from the low-cost, internet centric public datacenter provider. CSPs believed that there would be an upper crust of enterprises which would value the premier service being offered and stick with the CSP for their hosting needs. In the earlier days of the Internet, the network connection to the enterprise was expensive, not as fast as the LAN, and was almost always provided by the CSP. The CSP likely connected the multiple offices of the enterprise together with an MPLS VPN, and so for both bandwidth reasons and for the ability to conveniently have the outsourced datacenter within your MPLS VPN network, the CSP was a logical choice for most.</div>
<div style="background-color: white; color: #3e444c; font-family: DINLight, Helvetica; font-size: 11pt; line-height: 16pt; margin-bottom: 12pt; padding: 0px;">
Much has changed since then. Today, most mainstream datacenters have the same advanced security, reliability, and performance capabilities CSPs offer. As Jo points out, “The larger players have been able to buy their way into the market: We saw Verizon acquire Terremark”. One could argue, that the multi-carrier network of the “carrier neutral” hosting and co-location companies is actually better connectivity than any one CSP could offer. Some believe this strategic multi-carrier connectivity is the real reason Verizon acquired Terremark, with its “NAP of the Americas.” Finally, performance improvements and cost reduction in encryption-capable routers are allowing customer-configurable, IPSec-based VPN capability over multiple carrier networks an attractive alternative to the carrier-configured and controlled MPLS-based VPN.</div>
<div style="background-color: white; color: #3e444c; font-family: DINLight, Helvetica; font-size: 11pt; line-height: 16pt; margin-bottom: 12pt; padding: 0px;">
Jo goes on to point out, CSPs find themselves looking at a $40-50 billion market led by such offerings as AWS, GCE and Rackspace Cloud. It’s no wonder that many CSPs are beginning to build out cloud offerings themselves. As Jo puts it, “with mixed successes: Telstra, SingTel, French operator SFR, British Telecom, KT Corporation, Orange, AT&T, KDDI Corporation and Chunghwa Telecom, among many others, have all taken a shot at creating a cloud-services business.”</div>
<div style="background-color: white; color: #3e444c; font-family: DINLight, Helvetica; font-size: 11pt; line-height: 16pt; margin-bottom: 12pt; padding: 0px;">
So, why have carriers experienced mixed success? Because building cloud infrastructure at scale is not easy. Solving this problem is Cloudscaling’s raison d’être.</div>
<div style="background-color: white; color: #3e444c; font-family: DINLight, Helvetica; font-size: 11pt; line-height: 16pt; margin-bottom: 12pt; padding: 0px;">
Jo’s report outlines three areas in which carrier clouds have fallen short: cost competitiveness, acumen in selling cloud services, and bad technology bets. She then points out that, in an effort to address those shortcomings, many carriers “are now turning to commercial products or well-supported open-source systems to take another crack at it.” That’s what we’re seeing, too.</div>
<div style="background-color: white; color: #3e444c; font-family: DINLight, Helvetica; font-size: 11pt; line-height: 16pt; margin-bottom: 12pt; padding: 0px;">
Cost competitiveness is partially about equipment cost and software licensing, but equally important is operational efficiency. Achieving operational efficiency starts with cloud design, and that’s where the discomfort starts for many carriers. One need look no further than comparing the architecture of EC2 or GCE with that of Terremark or Savvis. They’re completely different. Our approach is to help CSPs understand how cloud design and architecture profoundly impact their long-term ability to deliver services cost competitively.</div>
<div style="background-color: white; color: #3e444c; font-family: DINLight, Helvetica; font-size: 11pt; line-height: 16pt; margin-bottom: 12pt; padding: 0px;">
The operational efficiency of cloud virtualization and automation provides for ultimately powerful pricing advantages over virtually any other model in hosting and co-location.</div>
<div style="background-color: white; color: #3e444c; font-family: DINLight, Helvetica; font-size: 11pt; line-height: 16pt; margin-bottom: 12pt; padding: 0px;">
Cloud developers and application technicians know how they want to acquire clouds services, what they should pay for them, and how they should work. Amazon has been quite successful in listening to the community and delivering a constant stream of innovation. CSPs usually set out to offer their cloud services in their home regions, where Amazon is not strong. We suggest a go- to-market model drafting on this runaway success, delivering what the early adopters want, but in the countries and using the special capabilities that the CSP has.</div>
<div style="background-color: white; color: #3e444c; font-family: DINLight, Helvetica; font-size: 11pt; line-height: 16pt; margin-bottom: 12pt; padding: 0px;">
And finally, it’s hard to argue with “open cloud” technology, as epitomized by projects like <a href="http://www.openstack.org/" style="-webkit-transition: all 0.3s ease; color: #b21f24; margin: 0px; padding: 0px; text-decoration: initial;">OpenStack</a>, <a href="http://www.openflow.org/" style="-webkit-transition: all 0.3s ease; color: #b21f24; margin: 0px; padding: 0px; text-decoration: initial;">OpenFlow</a> and <a href="http://opencompute.org/" style="-webkit-transition: all 0.3s ease; color: #b21f24; margin: 0px; padding: 0px; text-decoration: initial;">Open Compute</a>. At this point in the maturity cycle of cloud technology, there is strong momentum and investment in these directions. The early pioneers on a different technology base are going to be marginalized into niche market segments (at best) and OpenStack, like the Linux OS, or IP networking, will become the standard “open cloud” technology direction that will dominate the industry. It’s not a difficult prediction to make at this point, but if a CSP isn’t going that way yet, it’s time to try a new approach.</div>
<div style="background-color: white; color: #3e444c; font-family: DINLight, Helvetica; font-size: 11pt; line-height: 16pt; margin-bottom: 12pt; padding: 0px;">
<em style="margin: 0px; padding: 0px;">(Excerpted with permission from <a href="http://pro.gigaom.com/" style="-webkit-transition: all 0.3s ease; color: #b21f24; margin: 0px; padding: 0px; text-decoration: initial;">GigaOM Pro</a>.)</em></div>
David Bernsteinhttp://www.blogger.com/profile/01143001563622774579noreply@blogger.com4tag:blogger.com,1999:blog-5441792911708140271.post-64300632010676166132010-03-30T23:13:00.000-07:002010-03-30T23:14:49.872-07:00Thinking about Intercloud Topology, and Using XMPP as a transport in Intercloud Protocols<span style="font-size:100%;"><span style="font-family: arial;">I have been thinking about the topology for the Intercloud. There are Intercloud Exchanges (analogous to Internet Exchanges and Peering Points) where clouds can interoperate, and there is an Intercloud Root, containing services such as Naming Authority, Trust Authority, Directory Services, and other “root” capabilities. It is envisioned that the Intercloud root is of course physically not a single entity, a global replicating and hierarchical system similar to DNS. All elements in the Intercloud topology contain some gateway capability analogous to an Internet Router, implementing Intercloud protocols in order to participate in Intercloud interoperability. Let's call these Intercloud Gateways.<br /><br />The Intercloud Root and Intercloud Exchanges would facilitate and mediate the initial Intercloud negotiating process among Clouds and all these elements. It is this Presence and Messaging capability that's got me thinking, just what protocol would we use for that?<br /><br />If you think about it, cloud instances must be able to dialog with each other. One cloud must be able to find one or more other clouds, which for a particular interoperability scenario is ready, willing, and able to accept an interoperability transaction with and furthermore, exchanging whatever subscription or usage related information which might have been needed as a pre-cursor to the transaction. Thus, an Intercloud Protocol for presence and messaging needs to exist which can support the 1-to-1, 1-to-many, and many-to-many Cloud to Cloud use cases.<br /><br />Extensible Messaging and Presence Protocol (XMPP) is exactly such a protocol. XMPP is a set of open XML technologies for presence and real-time communication developed by the Jabber open-source community in 1999, formalized by the IETF in 2002-2004, continuously extended through the standards process of the XMPP Standards Foundation. XMPP supports presence, structured conversation, lightweight middleware, content syndication, and generalized routing of XML data. XMPP root services would be located in the Intercloud Root in the topology explained above.<br /><br />XMPP defines protocols for communicating between groups of entities which register with an XMPP server. Registration is dynamic and provides the basis for Presence. In a large implementation, such as the global Intercloud envisioned herein, XMPP servers are connected together. This is identical to the way service providers connect XMPP servers together already supporting cross-domain Instant Messaging. In this way, XMPP facilitates both presence and many-to-many messaging across service provider domains. XMPP messages are extensible, and can be used to carry messages of different types. For example, an XMPP Message can carry Instant Messaging (IM) type traffic. We could be using a Cloud extension to XMPP.<br /><br />Deepak Vij and I were thinking about all this, and we put some detail into a whole design for XMPP as a core transport in Intercloud. We wrote a paper and <a href="http://www.cloudstrategypartners.com/6.html">posted it on the CSP Site</a>. Check it out. It seems, XMPP might be just what the doctor ordered for Intercloud.</span></span>David Bernsteinhttp://www.blogger.com/profile/01143001563622774579noreply@blogger.com3tag:blogger.com,1999:blog-5441792911708140271.post-84819156549429740692009-12-29T12:28:00.000-08:002009-12-29T13:21:53.023-08:00Intercloud is comingI've been running around the world talking about Intercloud for well over a year now. I'm thankful that companies such as Cisco, and now Huawei, have been supportive in this adventure. Just as interoperability between the global Carriers of IP traffic, the result of which we now call the "Internet", the interoperability of Cloud Computingis an important development which needs to occur, and that's what I call "the Intercloud".<br /><br />I am happy to report that the concept of the Intercloud is finally picking up steam!<br /><br />First off, it was great to see that somebody wrote a Wikipedia entry for Intercloud (no it wasn't me!), which you can see <a href="http://en.wikipedia.org/wiki/Intercloud">here</a>. You will notice that the first official reference to the work Intercloud in the Wikipedia entry, is the paper I wrote with my team at Cisco. If you want to read the whole paper, and also see related presentations and so on, please check it out at my consulting site <a href="http://www.cloudstrategypartners.com/6.html">here</a>.<br /><br />The end of the year was quite active for me on the Intercloud front, as I was fortunate enough to keynote address on the subject at the 6th IFIP International Conference on Network and Parallel Computing (NPC 2009), October 19-21, 2009, in Gold Coast, Australia; and then another keynote at IARIA's First International Conferences on Advanced Service Computing SERVICE COMPUTATION 2009, November 15-20, 2009, in Athens/Glyfada, Greece. I gave similar talks at both events, you can download the talk <a href="http://www.cloudstrategypartners.com/resources/Intro+to+Intercloud+V6.pdf">here</a>.<br /><br />But the big end of the year wrap-up, was the mecca of all the Cloud Standards people getting together in Long Beach, at a Cloud Standards Workshop hosted by OMG. The event agenda can be seen <a href="http://www.omg.org/news/meetings/tc/ca/special-events/Cloud_Interop_Roadmaps.htm">here</a>. What a great line-up of many of the grooups working on different angles of the Cloud Standards problem.<br /><br />Two very important groups have stepped forward, <span style="font-weight: bold;">especially focused on the Intercloud problem, </span>are of particular interest to me (as an Intercloud guy).<br /><br />The first is a group I've been working with for some time, called the <a href="http://opencloudconsortium.org/">Open Cloud Consortium</a>. They finally announced an <a href="http://opencloudconsortium.org/working-groups/">Intercloud Workgroup</a>! This is an idea we've been working on for some time, where Open Cloud Testbed could be used to host some Root Intercloud services. This is especially exciting as the testbed is the same physical infrastructure as the <a href="http://www.nlr.net/">National Lambda Rail project</a> and has ties with the next generation Internet. I hope to be able to publish more info on this as soon as it is available.<br /><br />The next group of interest is the newly announced <a href="http://www.gictf.jp/index_e.html">Global Inter-Cloud Technology Task Force</a>, started in Tokyo, and expanding globally. They have picked up the work on Intercloud and have gotten support from carriers in the Asia-Pac region to really take this forward. They describe some of the charter and work <a href="http://www.cloudstrategypartners.com/resources/GICTF.pdf">here</a>. I'll bee teaming with them also to progress the Intercloud in Asia-Pac.<br /><br />So the year closes with great progress on the Intercloud. I hope 2010 shows as much progress.David Bernsteinhttp://www.blogger.com/profile/01143001563622774579noreply@blogger.com1tag:blogger.com,1999:blog-5441792911708140271.post-88000509039410261552009-06-16T19:24:00.000-07:002009-06-16T19:26:03.970-07:00IP Addressing is Broken for VM Mobile capable Clouds<span class="Apple-style-span" style="font-family: arial; font-size: 12px; "><p>I have been thinking that IP addressing is sort of broken when<br />it comes to Clouds. A lot of people in my former company have been thinking<br />about this (needless to say). I am going to paraphrase some thoughts and<br />write-ups we've had in this space. Although extending Layer 2 as widely<br />as possible solves a lot of problems, it doesn't solve the general<br />"private to public" or "public to public" problem. You always get back<br />to routing in the capital-I Internet.<br /></p><p>It all starts with the fact that, in a highly virtualized environment,<br />IP address space explodes. Everything has multiple IP addresses; servers<br />have IP addresses for management, for the physical NICs, for all of the<br />virtual machines and the virtual NIC therein, and if any virtual<br />appliances are installed they have multiple IP addresses as well.<br /></p><p>Several areas are of concern here, on the one hand, the IPv4 address<br />space simply starts to run out. Consider an environment inside the Cloud<br />which has 1M actual servers. As explained above, assuming a 16 core<br />server, each server could have 32 VM's, and each VM could have a handful<br />of IP addresses associated with it (virtual NICs, etc). That could<br />easily explode to a Cloud with well over 32M IP addresses. Even using<br />Network Address Translation (NAT), the 24-bit Class A reserved Private<br />Network Range provides a total address space of only 16M unique IP<br />addresses!<br /></p><p>For this reason many Cloud operators are considering switching to IPv6<br />which provides for a much larger local address space in the trillions of<br />unique IP addresses. Switching to IPv6 is quite an undertaking, and some<br />believe that switching from one static addressing scheme to another<br />static addressing scheme (eg IPv4 to IPv6) might not be the right<br />approach in a large highly virtualized environment such as Cloud<br />Computing. If one is reconsidering addressing, one should consider the<br />Mobility aspects of VMs in Cloud.<br /></p><p>VM Mobility provides for new challenges in any static addressing scheme.<br />When one moves a running VM from one location to another, the IP address<br />goes with the running VM and any application runtimes hosted by the VM.<br />IP addresses (of either traditional type) embody both Location and<br />Identity in the IP address, eg, routers and switches use the form of the<br />IP address not only to identify uniquely the endpoint, but by virtual of<br />decoding the address, infer the Location of the endpoint (and how to<br />reach that endpoint using switching and routing protocols). So while an<br />addressing scheme is being reconsidered, let's consider two schemes<br />which embody Mobility.<br /></p><p>You might think that Mobile IPv4 <<a target="_blank" rel="nofollow" href="http://www.ietf.org/rfc/rfc3344.txt" style="color: rgb(0, 0, 204); ">http://www.ietf.org/rfc/rfc3344.txt</a>><br />and Mobile IPv6 <<a target="_blank" rel="nofollow" href="http://www.ietf.org/rfc/rfc3775.txt" style="color: rgb(0, 0, 204); ">http://www.ietf.org/rfc/rfc3775.txt</a>> mechanisms can be<br />used in this case. Because IP addresses in either case are still<br />provider-supplied and follow top level address allocations, we still<br />find VM mobility issues when a VM attempts more general mobility from<br />one Cloud provider to another for example.<br /></p><p>In an attempt to completely generalize the addressing solution, a<br />completely dynamic scheme where Location and Identification have been<br />separated has been developed. This new scheme is called Location<br />Identity Separation Protocol<br /><<a target="_blank" rel="nofollow" href="http://tools.ietf.org/html/draft-farinacci-lisp-10" style="color: rgb(0, 0, 204); ">http://tools.ietf.org/html/draft-farinacci-lisp-10</a>> (LISP). LISP based<br />systems can interwork with both IPv4 and IPv6 based networks, through<br />protocol support on edge routers. However, internal to a Cloud, which<br />may in itself span several geographies, LISP addressing may be used.<br /></p><p>The basic idea behind the Loc/ID split is that the current Internet<br />routing and addressing architecture combines two functions: Routing<br />Locators (RLOCs), which describe how a device is attached to the<br />network, and Endpoint Identifiers (EIDs), which define "who" the device<br />is, in a single numbering space, the IP address. Proponents of the<br />Loc/ID split argue that this "overloading" of functions places the<br />constraints on end-system use of addresses that we detailed. Splitting<br />these functions apart by using different numbering spaces for EIDs and<br />RLOCs yields several advantages, including improved scalability of the<br />routing system through greater aggregation of RLOCs. To achieve this<br />aggregation, we must allocate RLOCs in a way that is congruent with the<br />topology of the network. EIDs, on the other hand, are typically<br />allocated along organizational boundaries.<br /></p><p>Because the network topology and organizational hierarchies are rarely<br />congruent, it is difficult (if not impossible) to make a single<br />numbering space efficiently serve both purposes without imposing<br />unacceptable constraints (such as requiring renumbering upon provider<br />changes) on the use of that space. LISP, as a specific instance of the<br />Loc/ID split, aims to decouple location and identity. This decoupling<br />will facilitate improved aggregation of the RLOC space, implement<br />persistent identity in the EID space, and hopefully increase the<br />security and efficiency of network mobility.<br /></p><p>Although LISP isn't in routers yet, it is alive <<a target="_blank" rel="nofollow" href="http://www.lisp4.net/" style="color: rgb(0, 0, 204); ">http://www.lisp4.net/</a>><br />and open <<a target="_blank" rel="nofollow" href="http://gforge.info.ucl.ac.be/projects/openlisp" style="color: rgb(0, 0, 204); ">http://gforge.info.ucl.ac.be/projects/openlisp</a>> ,<br />it may be just what the doctor ordered for the IP addressing 'challenge'<br />in Clouds. </p></span>David Bernsteinhttp://www.blogger.com/profile/01143001563622774579noreply@blogger.com1tag:blogger.com,1999:blog-5441792911708140271.post-44750039238208546812009-06-16T18:36:00.001-07:002009-06-16T18:43:34.530-07:00Thinking about Cloud to Cloud Interoperability Use Cases<p class="nospacing"><span class="Apple-style-span" style="font-family:Arial;"></span></p><span class="Apple-style-span" style="font-family:Arial;"><p class="nospacing">Cloud computing is a term applied to large, hosted datacenters, usually geographically distributed, which offer various computational services on a “utility” basis. Most typically the configuration and provisioning of these datacenters, as far as the services for the subscribers go, is highly automated, to the point of the service being delivered within seconds of the subscriber request. Additionally, the datacenters typically use hypervisor based virtualization as a technique to deliver these services. The concept of a cloud operated by one service provider or enterprise interoperating with a clouds operated by another is a powerful idea. So far that is limited to use cases where code running on one cloud explicitly references a service on another cloud. There is no implicit and transparent interoperability. In this article, I write about use cases for interoperability, and an architecture for Intercloud standards.</p><p class="nospacing">Of course from within one cloud, explicit instructions can be issued over the Internet to another cloud. For example, code executing within Google AppEngine can also reference storage residing on AWS. However there are no implicit ways that clouds resources and services can be exported or caused to interoperate.</p><p class="nospacing">In this Blog I come up with two main use cases for Cloud Interoperability; the first is a use case involving a physical metaphor (servers, disks, network segments, etc). The second is a use case involving an abstract metaphor (blob storage functions, message queue, email functions, multicast functions, etc). We look at cloud interoperability challenges using use cases illustrating the two major personality types of clouds.</p><p class="nospacing"><b>Virtual Machine Instantiation and Mobility</b></p><p class="nospacing"> </p><p class="nospacing">One of the most basic resources which cloud computing delivers is the Virtual Machine, which is a physical metaphor type of resource. One way or another, a subscriber requests the provisioning of a particularly configured virtual machine with certain quantities of resources such as memory processor speeds and quantities.. The format of this request varies widely by cloud computing platform and also is somewhat specific to the type of hypervisor (the virtualization layer of the operating system inside the cloud computing platform). In a few seconds they receive pointers and credentials with which to access it. The pointers are usually the MAC and IP addresses and sometimes a DNS name given to the VM. The credentials are usually a pair of RSA keys (a public key and a private key, which one uses in the API to speak with the VM). Most often, the VM presents an x86 PC machine architecture. On that VM, one boots a system image yielding a running system, and uses it in a similar manner as one would use a running system in your own datacenter.</p><p class="nospacing"> </p><p class="nospacing">VM Mobility is that feature in a particular hypervisor which allows a running system to be moved from one VM to another VM. As far as the running system is concerned it does not need to be reconfigured, all of the elements such as MAC and IP address and DNS name stay the same; any of the ways storage may be referenced (such as a World Wide Name in a SAN) stay the same. Whatever needs to happen to make this work is not the concern of the running system.</p><p class="nospacing"> </p><p class="nospacing">VM Mobility has been implemented with several hypervisors but there are limitations. Usually these limitations are a result of the “scope” of applicability of the network and storage addressing. Typically, VM Mobility is restricted to a Layer 3 subnet and a Layer 2 domain (for VLANs) because the underlying network will support the VM operating outside of the local scope of those addresses. Needless to say, the network addressing scheme in a cloud operated by an entirely different service provider is not only a different subnet but a different class B or class A network altogether. Routers and switches simply would not know how to cope with the “rogue” running system.</p><p class="nospacing"> </p><p class="nospacing">Another aspect is that, the instantiation instructions of the VM for the running system are very specific to that cloud computing platform and the hypervisor which it uses. We would want to re-issue some of these instructions to the new cloud so that the VM it delivered onto which the VM would move, was as suitable as the first VM which was provisioned for us. If the new Cloud takes an entirely different set of instructions, this is another barrier to VM Mobility.</p><p class="nospacing"> </p><p class="nospacing">All of this assumed that in the universe of cloud computing systems out there, we were able to find another cloud, which was ready, willing, and able to accept a VM mobility transaction with me. And that I was able to have a reliable conversation with that cloud, perhaps exchanging whatever subscription or usage related information which might have been needed as a pre-cursor to the transaction, and finally that I had a reliable transport on which to move the VM itself.</p><p class="nospacing"> </p><p class="nospacing"><b>Storage Interoperability and Federation</b></p><p class="nospacing"> </p><p class="nospacing">Now let us consider an interoperability use case involving an abstract metaphor. In this case, we are running script or code in my datacenter or in the cloud, which is utilizing Cloud based storage functions. In cloud computing, storage is not like disk access, there are several parameters around the storage which are inherent to the system, and one decides if they meet your needs or not For example, object storage is typically replicated to several places in the cloud, In AWS and in Azure it is replicated three places. The storage API is not explicit in this, but implicitly, we know that a write will return as successful when one replicate of the storage has been affected, and then a “lazy” internal algorithm is used to replicate the object to two additional places. If one or two of the object replicates are lost the cloud platform will replicate it to another place or two such that it is now in three places. A user has some control over where the storage is, physically, for example, one can restrict the storage to replicate entirely in North America or in Europe.</p><p class="nospacing"> </p><p class="nospacing">There is no ability to vary from these parameters; that is what the storage system provides. One would have thought that there might be several API’s each with a different underlying characteristic, and, you could always use a “better” service implementation than the API demanded. To this end, we do envision other providers implementing say, five replicates, or a deterministic replication algorithm, or a replicated (DR) write which doesn’t return until and unless n replicates are persisted. One can create a large number of variations around “quality of storage” for Cloud.</p><p class="nospacing"> </p><p class="nospacing">In the interoperability scenario, suppose AWS is running short of storage, or wants to provide a geographic storage location for an AWS customer, where AWS does not have a datacenter, it would be sub-contracting the storage to another service provider. In either of these scenarios, AWS would need to find another cloud, which was ready, willing, and able to accept a storage subcontracting transaction with them. AWS would have to be able to have a reliable conversation with that cloud, again exchanging whatever subscription or usage related information which might have been needed as a pre-cursor to the transaction, and finally have a reliable transport on which to move the storage itself. </p><p class="nospacing"> </p><p class="nospacing">Although the addressing issues are not as severe in this case where an abstract metaphor is used, the naming, discovery, conversation setup items challenges all remain.</p></span><p></p>David Bernsteinhttp://www.blogger.com/profile/01143001563622774579noreply@blogger.com1tag:blogger.com,1999:blog-5441792911708140271.post-4276967245412812202009-06-16T18:34:00.000-07:002009-06-16T18:35:31.341-07:00What Makes a Cloud - A Cloud - and not just a Datacenter<p class="nospacing"><span style="font-family:Arial">Cloud computing has emerged recently as a label for a particular type of datacenter. It can be hosted by anyone; an enterprise, a service provider, or a government. I have been thinking, a way to define cloud computing, is to realize that a Cloud is just a special kind of datacenter. We list seven key characteristics which make a large datacenter into a cloud<o:p></o:p></span></p> <p class="nospacing" style="margin-left:.5in;text-indent:-.25in;mso-list:l0 level1 lfo1; tab-stops:list .5in"><span style="font-family:Arial; mso-fareast-font-family:Arial"><span style="mso-list:Ignore">1.<span style="font:7.0pt "Times New Roman""> </span></span></span><span style="font-family:Arial">Implement a pool of computing resources and services which are shared amongst subscribers.<u1:p></u1:p><o:p></o:p></span></p> <p class="nospacing" style="margin-left:.5in;text-indent:-.25in"><span style="font-family:Arial">2.<span style="font-size-adjust: none;font-stretch: normal"></span><span style="font-size:7.0pt;font-family:Arial"> </span></span><span style="font-family:Arial">Charge for resources and services using an “as used” metered and/or capacity based model.<u1:p></u1:p><o:p></o:p></span></p> <p class="nospacing" style="margin-left:.5in;text-indent:-.25in"><span style="font-family:Arial">3.<span style="font-size-adjust: none;font-stretch: normal"></span><span style="font-size:7.0pt;font-family:Arial"> </span></span><span style="font-family:Arial">Are usually geographically distributed, in a manner which is transparent to the subscriber (unless they explicitly ask for visibility of that).<u1:p></u1:p><o:p></o:p></span></p> <p class="nospacing" style="margin-left:.5in;text-indent:-.25in"><span style="font-family:Arial">4.<span style="font-size-adjust: none;font-stretch: normal"></span><span style="font-size:7.0pt;font-family:Arial"> </span></span><span style="font-family:Arial">Are automated in that the provisioning and configuration (and de-configuration and un-provisioning) of resources and services occur on the “self service”, usually programmatic request of the subscriber, occur in an automated way with no human operator assistance, and are delivered in one or two orders of seconds.<u1:p></u1:p><o:p></o:p></span></p> <p class="nospacing" style="margin-left:.5in;text-indent:-.25in"><span style="font-family:Arial">5.<span style="font-size-adjust: none;font-stretch: normal"></span><span style="font-size:7.0pt;font-family:Arial"> </span></span><span style="font-family:Arial">Resources and services are delivered virtually, that is, although they may appear to be physical (servers, disks, network segments, etc) they are actually virtual implementations of those on an underlying physical infrastructure which the subscriber never sees.<u1:p></u1:p><o:p></o:p></span></p> <p class="nospacing" style="margin-left:.5in;text-indent:-.25in"><span style="font-family:Arial">6.<span style="font-size-adjust: none;font-stretch: normal"></span><span style="font-size:7.0pt;font-family:Arial"> </span></span><span style="font-family:Arial">The physical infrastructure changes rarely. The virtually delivered resources and services are changing constantly.<u1:p></u1:p><o:p></o:p></span></p> <p class="nospacing" style="margin-left:.5in;text-indent:-.25in"><span style="font-family:Arial">7.<span style="font-size-adjust: none;font-stretch: normal"></span><span style="font-size:7.0pt;font-family:Arial"> </span></span><span style="font-family:Arial">Resources and services may be of a physical metaphor (servers, disks, network segments, etc) or they may be of an abstract metaphor (blob storage functions, message queue functions, email functions, multicast functions, etc). These may be intermixed.<u1:p></u1:p><o:p></o:p></span></p> <p class="nospacing"><span style="font-family:Arial">Cloud computing services as defined above are best exemplified by the Amazon Web Services (AWS) or Google AppEngine. Both of these systems exhibit all seven characteristics as detailed above. Various companies are beginning to offer similar services, such as the Microsoft Azure Service, and software companies such as VMware and open source projects such as UCSB Eucalyptus are creating software for building a cloud service. Each of these offerings embody cloud computing with a self-contained set of conventions, file formats, and programmer interfaces. If one wants to utilize that variation of cloud, one must create configurations and code specific to that cloud.</span></p>David Bernsteinhttp://www.blogger.com/profile/01143001563622774579noreply@blogger.com12