Microsoft needs SDN for Azure cloud

Microsoft needs SDN for Azure cloud

Couldn't scale without it, Azure CTO says
The Microsoft cloud, through which the company’s software products are delivered, has 22 hyper-scale regions around the world. Azure storage and compute usage is doubling every six months, and Azure lines up 90,000 new subscribers a month.

Six TED Talks that can change your career
Of the hundreds of TED talks available online, many are geared toward helping people view life in a new

Fifty-seven percent of the Fortune 500 use Azure and the number of hosts quickly grew from 100,000 to millions, said CTO Mark Russinovich during his Open Network Summit keynote address here this week. Azure needs a virtualized, partitioned and scale-out design, delivered through software, in order to keep up with that kind of growth.

“When we started to build these networks and started to see these types of requirements, the scale we were operating at, you can’t have humans provisioning things,” Russinovich said. “You’ve got to have systems that are very flexible and also delivering functionality very quickly. This meant we couldn’t go to the Web and do an Internet search for a scalable cloud controller that supports this kind of functionality. It just didn’t exist.”

Microsoft wrote all of the software code for Azure’s SDN. A description of it can be found here.
Microsoft uses virtual networks (Vnets) built from overlays and Network Functions Virtualization services running as software on commodity servers. Vnets are partitioned through Azure controllers established as a set of interconnected services, and each service is partitioned to scale and run protocols on multiple instances for high availability.

Controllers are established in regions where there could be 100,000 to 500,000 hosts. Within those regions are smaller clustered controllers which act as stateless caches for up to 1,000 hosts.
Related

Why is Microsoft killing off Internet Explorer?
Microsoft builds these controllers using an internally developed Service Fabric for Azure. Service Fabric has what Microsoft calls a microservices-based architecture that allows customers to update individual application components without having to update the entire application.

Microsoft makes the Azure Service Fabric SDK available here.
Much of the programmability of the Azure SDN is performed on the host server with hardware assist. A Virtual Filtering Platform (VFP) in Hyper-V hosts enable Azure’s data plane to act as a Hyper-V virtual network programmable switch for network agents that work on behalf of controllers for Vnet and other functions, like load balancing.

Packet processing is done at the host where a NIC with a Field Programmable Gate Array offloads network processing from the host CPU to scale the Azure data plane from 1Gbps to 40Gbps and beyond. That helps retain host CPU cycles for processing customer VMs, Microsoft says.

Remote Direct Memory Access is employed for the high-performance storage back-end to Azure.
Though SDNs and open source go hand-in-hand, there’s no open source software content in the Azure SDN. That’s because the functionality required for Azure was not offered through open source communities, Russinovich says.

“As these requirements were hitting us, there was no open source out there able to meet them,” he says. “And once you start on a path where you’re starting to build out infrastructure and system, even if there’s something else that comes along and addresses those requirements the switching cost is pretty huge. It’s not an aversion to it; it’s that we haven’t seen open source out there that really meets our needs, and there’s a switching cost that we have to take into account, which will slow us down.”

Microsoft is, however, considering contributing the Azure Service Fabric architecture to the open source community, Russinovich said. But there has to be some symbiosis.

“What’s secret sauce, what’s not; what’s the cost of contributing to open source, what’s the benefit to customers of open source, what’s the benefit to us penetrating markets,” he says. “It’s a constant evaluation.”

Some of the challenges in constructing the Azure SDN were retrofitting existing controllers into the Service Fabric, Russinovich says. That resulted in some scaling issues.
Resources

7 Critical Questions to Demystify DRaaS
“Some of the original controllers were written not using Service Fabric so they were not microservice oriented,” he says. “We immediately started to run into scale challenges with that. Existing ones are being (rewritten) onto Service Fabric.

“Another one is this evolution of the VFP and how it does packet processing. That is not something that we sat down initially and said, ‘it’s connections, not flows.’ We need to make sure that packet processing on every packet after the connection is set up needs to be highly efficient. It’s been the challenge of being able to operate efficiently, scale it up quickly, being able to deliver features into it quickly, and being able to take the load off the server so we can run VMs on it.”

What’s next for the Azure SDN? Preparing for more explosive growth of the Microsoft cloud, Russinovich says.

“It’s a constant evolution in terms of functionality and features,” he says. “You’re going to see us get more richer and powerful abstractions at the network level from a customer API perspective. We’re going to see 10X scale in a few years.”

Best Microsoft MCTS Certification, Microsoft MCITP Training at certkingdom.com

Startup proposes fiber-based Glass Core as a bold rethink of data center networking

Software Defined Networking (SDN) challenges long held conventions, and newcomer Fiber Mountain wants to use the SDN momentum to leap frog forward and redefine the fundamental approach to data center switching while we're at it. The promise: 1.5x to 2x the capacity for half the price.

How? By swapping out traditional top of rack and other data center switches with optical cross connects that are all software controlled. The resultant “Glass Core,” as the company calls it, provides “software-controlled fiber optic connectivity emulating the benefits of direct-attached connectivity from any port … to any other server, storage, switch, or router port across the entire data center, regardless of location and with near-zero latency.”

The privately funded company, headed by Founder and CEO M. H. Raza, whose career in networking includes stints at ADC Telecommunications, 3Com, Fujitsu BCS and General DataComm, announced its new approach at Interop in New York earlier this week. It’s a bold rethinking of basic data center infrastructure that you don’t see too often.

“Their value proposition changes some of the rules of the game,” says Rohit Mehra, vice resident of network infrastructure at IDC. “If they can get into some key accounts, they have a shot at gaining some mind share.”

Raza says the classic approach of networking data center servers always results in “punting everything up to the core” – from top of rack switches to end of row devices and then up to the core and back down to the destination. The layers add expense and latency, which Fiber Mountain wants to address with a family of products designed to avoid as much packet processing as possible by establishing what amounts to point-to-point fiber links between data center ports.

“I like to call it direct attached,” Raza says. “We create what we call Programmable Light Paths between a point in the network and any other point, so it is almost like a physical layer connection. I say almost because we do have an optical packet exchange in the middle that can switch light from one port to another.”

That central device is the company’s AllPath 4000-Series Optical Exchange, with 14 24-fiber MPO connectors, supporting up to 160x160 10G ports. A 10G port requires a fiber pair, and multiple 10G ports can be ganged together to support 40G or 100G requirements.

The 4000 Exchange is connected via fiber to any of the company’s top-of-rack devices, which are available in different configurations, and all of these devices run Fiber Mountain’s Alpine Orchestration System (AOS) software.

That allows the company’s homegrown AOS SDN controller, which supports OpenFlow APIs (but is otherwise proprietary), to control all of the components as one system. Delivered as a 1U appliance, the controller “knows where all the ports are, what they are connected to, and makes it possible to connect virtually any port to any other port,” Raza says. The controller “allows centralized configuration, control and topology discovery for the entire data center network,” the company reports, and allows for “administrator-definable Programmable Light Paths” between
How do the numbers work out? Raza uses a typical data center row of 10 racks of servers as the basis for comparison. The traditional approach;

Each rack typically has two top-of-rack switches for redundancy, each of which costs about $50,000 (so $100,000/rack, or $1 million per row of 10 racks).
Each row typically has two end-of-row switches that cost about $75,000 each (another $150,000)
Cabling is usually 5%-10% of the cost (10% of $1.15 million adds $115,000)
Total: $1.265 million

With the Fiber Mountain approach:
Each top-of-rack switch has capacity enough to support two racks, so a fully redundant system for a row of 10 racks is 10 switches, each of which cost $30,000. ($300,000).
The 4000 series core device set up at the end of an isle costs roughly $30,000 (and you need two, so $60,000).
Cabling is more expensive because of the fiber used, and while it wouldn’t probably be more than double the expense, for this exercise Raza says to use $300,000.

Total $660,000. About half, and that doesn’t include savings that would be realized by reducing demands on the legacy data center core now that you aren’t “punting everything up” there all the time.

What’s more, Raza says, “besides lower up front costs, we also promise great Opex savings because everything is under software control.”

No one, of course, rips out depreciated infrastructure to swap in untested gear, so how does the company stand a chance at gaining a foothold?

Incremental incursion.
Try us in one row, Raza says. Put in our top-of-rack switches and connect all the server fibers to that and the existing top-of-rack switch fibers to that, and connect our switches to one of our cores at the end of the isle. “Then, if you can get somewhere on fiber only, you can achieve that, or, if you need the legacy switch, you can shift traffic over to that,” he says.

Down the road, connect the end of isle Glass Core directly to other end of row switches, bypassing the legacy core altogether. The goal, Raza says, is to direct connect racks and start to take legacy switching out.

While he is impressed by what he sees, IDC’s Mehra says “the new paradigm comes with risks. What if it doesn’t scale? What if it doesn’t do what they promise? The question is, can they execute in the short term. I would give them six to 12 months to really prove themselves.”

Raza says he has four large New York-based companies considering the technology now, and expects his first deployment to be later this month (October 2014).

---

Best comptia A+ Training, Comptia A+ Certification at Certkingdom.com