This post focuses on BGP VxLAN EVPN and thus an understanding of BGP and VxLAN is very helpful to understand this topic. Additionally EVPN and VxLAN are considered overlay technologies which run over an underlay IP fabric. In this context the underlay fabric’s purpose is to provide reachability between VTEP’s.

Whilst outside the focus of this post, the main choices for the underlay are OSPF or IS-IS. There are pros and cons to each option, with OSPFv2 being very well understood by most engineers and simple to deploy, however it does not support IPv6 and thus OSPFv3 would be required for IPv6 support which is still not mature in vendor implementations. Alternatively IS-IS has supported both IPv4 and IPv6 for many years and is well supported by vendors but is not well understood by many engineers outside of Telcos. Note: BGP can also be used as the underlay but as it is also utilised in the overlay this can cause confusion and complexity. Utilising BGP for the underlay is fine however I would recommend doing your own research regarding the underlay protocol to use taking into account the engineer’s skills who will be deploying and supporting the fabric.

In summary VxLAN is a tunneling mechanism which takes layer 2 frames or a layer 3 packet, and encapsulates it with an IP header and routes it to a VxLAN Virtual Tunnel End Point (VTEP) for decapsulation; It effectively encapsulates an MAC address inside an IP packet.

Similar to VLANs which have a 12-bit field specifying the VLAN to which the frame belongs, for a total of 4096 VLAN tags, VxLAN header includes a 24-bit field called VxLAN Network Identifier (VNI), which allows up to 16 million layer 2 domains.

VXLAN uses by default the flood-and-learn behavior of the multicast control plane, which is fine for small deployments but does have salability limitations in large deployments. Another method is ingress Head End Replication (HER), which does not require multicast but is still a flood-and-learn data plane procedure. There are also some controller based solutions but these are outside the scope of this discussion.

To resolve the scaling limitation of the flood-and-learn approach Ethernet VPN (EVPN) control plane was created, utilizing a new address family in Multi-Protocol BGP (MP-BGP) to distribute the layer 2 and layer 3 host reachability information. Therefore Multi-Protocol Border Gateway Protocol (MP-BGP) was extended to utilise the Network Layer Reachability Information (NLRI) to carry both Layer 2 MAC and Layer 3 IP information at the same time, and this is called EVPN – Ethernet Virtual Private Network.  It also offers a range of other benefits such as reduction of data center traffic through ARP suppression.

Utilising BGP as the control plane for VxLAN enables capabilities such as MAC address learning and VRF multi-tenancy while providing optimized equal-cost multi-pathing (ECMP). The new BGP address family in Multi-Protocol BGP is utilised to exchange Network Layer Reachability Information (NLRI) via a series of route types. Of these route types, the two most applicable for this discussion are:

Type 2 – Host MAC and IP addresses (MAC-VRF)
Type 5 – IP Prefix information (IP-VRF)

Type-2 routes (RT-2) are utilised to advertise an end host’s MAC and IP address within the VLAN over an IP network. A VxLAN Network Identifier (VNI) is mapped to a VLAN and all VTEP’s (typically leaf switches) within the VNI utilise RT-2 to share and learn the end host’s MAC addresses to provide Layer 2 reachability.

Type-5 routes (RT-5) are utilised to advertise IP prefixes. A VXLAN Network Identifier (VNI) is mapped to a Virtual Routing & Forwarding (VRF) which identifies a tenant within the fabric, allowing for multitenancy and route tables to coexist.

The advertisement of the type 5 EVPN attribute will provide the NLRI between subnets and routing contexts, allowing for learning of prefixes (not MACs) that are advertised across different VRFs in the fabric. This means the fabric can provide end-to-end segmentation without being aware of the segmentation itself. For example, a VRF context can be created on a pair of Leaf switches and be extended to some other pair of Leaf switches without the devices in between being aware of the VRFs. With EVPN, only the leaf switches need to possess the VRFs which endpoints are attached to, allowing the Spine switches to simply provide transit between Leafs.

There are two models to provide inter-subnet routing with EVPN, which are asymmetric integrated routing and bridging (IRB) and symmetric IRB. The main difference between the asymmetric IRB model and symmetric IRB model is how and where the routing lookup is done, which results in differences concerning which VNI the packet travels on through the infrastructure.

The asymmetric model allows routing and bridging on the VXLAN tunnel ingress, but only bridging on the egress. This results in bi-directional VXLAN traffic traveling on different VNIs in each direction (always the destination VNI) across the routed infrastructure

The symmetric model routes and bridges on both the ingress and the egress leafs. This results in bi-directional traffic being able to travel on the same VNI, hence the symmetric name. However, a new specialty transit VNI is used for all routed VXLAN traffic, called the L3VNI. All traffic that needs to be routed will be routed onto the L3VNI, tunneled across the layer 3 Infrastructure, routed off the L3VNI to the appropriate VLAN and ultimately bridged to the destination.

Depending on the vendor hardware the topic of asymmetric or symmetric model may not be of concern as some hardware only supports one model and thus you will need to configure the fabric based on that limitation.

Generally, if you configure all VLANs/Subnets/VNIs on all leafs anyway then the asymmetric model is fine and may be simpler to configure as it doesn’t require extra VNIs.

If your VLANs/Subnets/VNIs are widely dispersed and/or provisioned on the fly, then the symmetric model is better and all routed traffic will use a transit VNI (L3VNI), while bridged traffic will use L2VNI.

NOTE: The symmetric model is what Cisco utilises and supports.

This should provide an overview of EVPN, and I’ll delve into more technical detail and configuration in subsequent posts.