If you have your own network that needs to communicate with your cloud resources in Azure over an internal network, you have two options.
- (site-2-site) VPN over the internet to a VPN server in your cloud
- Easy to setup
- Cheap to use
- Limited and unpredictable bandwidth
- Depending on internet quality and availability
- ExpressRoute
- Requires a connectivity provider
- Dedicated connection, not over the internet
- Bandwidth up to 10Gbps
- Dual-channel, redundant by design
- Extremely reliable and secure
This article is about the ExpressRoute option. You can establish private connectivity between Azure and your datacenter, office, or colocation environment. In many cases, this reduces your network costs, increase bandwidth throughput, and provide a more consistent network experience than Internet-based connections.
ExpressRoute: your options
Microsoft does not allow you to connect your network directly into their router. Instead, they have gathered a list of connectivity providers.
A connectivity provider is a provider that shares a datacenter with Microsoft and handles connectivity and contracts with end-customers.
As an end-customer you have, again, two options:
- Peer directly with that connectivity provider
- Requires your own hardware in the datacenter of that provider
- Requires your own connectivity from your offices directly to that hardware
- You manage your lines and hardware yourself
- For all of the above reasons probably expensive
- Use a service provider to peer with the connectivity provider
- Is the man in the middle that provides the service to connect your office via their network to their equipment in the data center where it will connect to the connectivity provider.
- You share their hardware with other customers
- You introduce an extra point of failure
- Cheaper
I’ve created an schematic overview of the above:
From Azure point of view, ExpressRoute is a single service. Although it has two channels (redundancy) you can not leave one out to save some money for example.
Routing and ExpressRoute
ExpressRoute is BGP controlled. BGP stands for Border Gateway Protocol. BGP is a protocol designed to exchange routing and reachability information between two or more systems. Both Microsoft and the router in the datacenter of the connectivity provider announce subnets to each other that they can handle. Only traffic for subnets that the other side announces, is allowed to travel over the ExpressRoute connection.
You can not manage BGP directly on Azure. Instead, Azure will automatically announce all subnets that can be reached from the VNET that the ExpressRoute is connected to. Your router needs to do the same. Azure will refuse to route traffic over the ExpressRoute where the destination is not announced via BGP even if you put manual routing tables in place. This is important because it will probably be the cause of most “it doesn’t work”-scenarios.
A special case is announcing subnets that are public, i.e. internet traffic, or even 0.0.0.0/0 as a route. If your router makes such an announcement you need to be able to route internet traffic via your router as Azure will no longer use its own outgoing internet connectivity. This is useful if you want to route internet traffic via a security device on-premise.
The ExpressRoute stack in Azure
Setting up an expressroute consists of several steps. In essence, the ExpressRoute is considered a very special type of VPN that enters a VNET via a Virtual Network Gateway.
Your first step will be to create the ExpressRoute configuration in Azure. Here you will define the connectivity provider, the requested bandwidth, metered or unlimited bandwidth, and more. In return, you will receive a service-key. This service-key, a long list of numbers, needs to be provided to your connectivity provider as part of the ordering process. With that key, they will provision your connection. Once the provisioning is completed, your ExpressRoute physical connection is up.
From there on you can setup the connection object, and link that connection to a Virtual Network Gateway in a VNET.
Congratulations, your ExpressRoute is up and running.
Hub – Spoke and FastPath
In most cases, you will connect your ExpressRoute to a single VNET, called the hub, from where peering is made to other VNETs, called the spokes. This is the reference design by Microsoft. It allows you to deploy extra network services (IDS, firewall) that can be utilized by all the spokes.
However, technically it is possible to peer multiple VNETs directly to the ExpressRoute. This can be a requirement when using FastPath.
FastPath sends network traffic directly to virtual machines in the virtual network, bypassing the gateway, for extra speed. This is not possible when a server is in a spoke, and the hub is in between. FastPath only works in the first broadcast domain.
Office 365 over ExpressRoute
By default, an ExpressRoute will not route your O365 traffic. This is not the intended purpose of the link, but it is possible to have this enabled in very specific cases. Microsoft reviews every request and authorizes only ExpressRoute for Office 365 usage when a customer’s regulatory requirement mandates direct connectivity.
Office 365 users who are not located in their ExpressRoute connected building can still connect over the Internet.
Without approval you are not authorized for creating route filters for Office 365 and any attempt to configure this will fail.
Connectivity of other cloud vendors
Cloud connectivity options are available with the “big three” cloud service providers: Amazon Web Services, Microsoft Azure, and Google.
Azure ExpressRoute is Microsoft. Amazon has named it AWS Direct Connect, and Google has its Cloud Dedicated Interconnect. All are similar technologies but have their own advantages and disadvantages.