Redefine the future of enterprise connectivity

MPLS VPN Setup in DCSG Using NiralOS


Telco Network at a glance

A mobile network operators’ network infrastructure is very complex comprising of 10s of thousands of devices. 5G deployment would further densify the network with the addition of indoor and outdoor small cells, outdoor macrocells for radio access and cell site routers, edge compute appliances. Typically, the current 4G network contains the following:

  • Mobile Core – Contains Packet core components like PGW, MME, HSS, DPI
  • Core Routers Ring
  • Aggregation Routers Ring
  • Access Routers Ring
  • Cell Sites – Customer access endpoints like Cell towers, Residential Gateway, Enterprise VPN

Figure 1 – 5G Network Infrastructure

For 5G, DCSG (Disaggregated Cell Site Gateway) is introduced in between Cell towers and Access Routers. Multiple DCSGs form a ring that connects to the Access Router. As per 3GPP standards, DCSG carries the X2/Xn traffic. DCSG can also carries Residential and Enterprise traffic.

Niral Networks develops disaggregated open source Network Operating system (NOS) for DSCG, Access Router and DPI (Deep Packet Inspection) called NiralOS. Niral Networks aims to democratize the wireline routing and switching systems in Telco’s networks infrastructure with its NiralOS. NiralOS is integrable with White box hardware based on Broadcom, Intel, NXP.

DCSG Traffic differentiation and engineering

Figure 2 – Traffic through DCSG

Various kinds of traffic flow through the DCSG:

  • 5G mobile traffic from subscribers connected to 5G base station (gNB)
  • 4G mobile traffic from subscribers connected to 4G base station (eNB)
  • Residential traffic
  • Enterprise VPN traffic

These traffic from the DCSG are destined towards the Mobile core or Internet or another DCSG.

Each type of traffic requires a different QOS treatment, for example, 5G traffic has lower latency guarantee compared to 4G traffic.

To differentiate traffic, the network infrastructure is configured as a Hierarchal MPLS Layer 3 VPN Cloud and each DCSG is configured as the Layer3 VPN service endpoint. The DCSGs and the Core router connected to the Mobile core are configured as PE (Provider Edge) routers. The PE router identifies each traffic type based on the incoming interface and encapsulates the VPN label and LSP label.

To guarantee SLA for the traffic, the MPLS cloud is configured to support Traffic Engineering (TE) using protocols and technologies like RSVP-TE (Resource Reservation protocol), CSPF (Constrained Shortest Path First), FRR (Fast Reroute), PCEP (Path Computation Element protocol).

5G outlines 3 major service classes – Enhanced Mobile Broadband (eMBB), Ultra-reliable low latency communication (URLLC), and Massive machine type communication (MMTC). The service classes are categorized by network performance attributes such as capacity (coverage and speed), latency, mobility, and scalability.

One of the key features of 5G is Network slicing technology that enables operators to virtually slice the physical network (RAN, transport and core) by optimizing resources and network topology to create logical networks or partitions that correspond to each service class. Network slicing leverages technologies such as TE, PCEP, network functions virtualization (NFV), and software-defined networking (SDN) to create layers of different virtual networks with guaranteed SLA on physical infrastructure. QOS guarantee, TE, Network Slicing is not part of the current blog and would be covered in the future.

Figure 3 – Traffic differentiation in MPLS Cloud

In Figure 3, the configuration is as follows:

DCSG-1

Interface connected to 5G Base station is configured as VRF Blue Interface connected to Enterprise Brach office is configured as VRF Brown

DCSG-2

One of its interface is connected to Enterprise HQ and is configured as VRF Brown

CR-1

Interface is configured as VRF Blue for 5G traffic Similar other interfaces are configured with VRF for 4G traffic, Home traffic etc.

Enterprise traffic traverses between DCSG-1 and DCSG-2 via the MPLS cloud. You-tube traffic for UE-1 traverses between DCSG-1 and CR-1 through the MPLS Cloud.

NiralOS Introduction

NiralOS is a disaggregated, open-source NOS that can be integrated with any White box hardware to provide DCSG, Access Router or DPI solution. NiralOS is integrable to White box hardware based on Broadcom, Intel, NXP.

Niral Networks is actively working with the following open source community to develop NiralOS:

DANOS

FRR

Disaggregated Network Operating System – https://www.danosproject.org/
The seed code was contributed by AT&T to speed the adoption and use of white boxes in a service provider’s infrastructure.
DANOS 2005 was released recently – https://danosproject.atlassian.net/wiki/spaces/
DAN/pages/320634926/DANOS+2005+Release+Notes Niral team is actively working with the community. Its contribution includes:
– ISIS support for IPv4 and IPv6
– MPLS VPN Support for IPv4 and IPv6
– FRR 7.3 Integration
– 6PE/6VPE support

Free Range Routing – https://frrouting.org/
FRR has its roots in the Quagga Project. FRR is an IP routing protocol suite for Linux and Unix platforms which includes protocol daemons for BGP, IS-IS, LDP, OSPF, PIM, and RIP.
FRR latest release – https://github.com/FRRouting/frr/releases/tag/frr-7.3.1
Niral team is working with the community to bring in Telco specific features. Some of Niral’s work includes:
– 6PE/6VPE support
– ISIS VRF Support

MPLS-VPN Network Simulation

Figure 4 – L3 VPN Topology in lab

To simulate the Enterprise traffic flow between DCSG-1 and DCSG-2 in figure – 3, and internet traffic flow between DCSG-1 and CR-1, we have created a lab environment that consists of 8 routers:

  • Routers CE 1 and CE 2 are the customer edge routers that reside in the Branch office and HQ of the Enterprise
  • PE 1 and PE 2 simulates DCSG-1 and DCSG-2 respectively and resides near the cell site
  • Routers CE A simulates the 5G Base Station and CE M simulates the router in Mobile Core
  • PE 3 simulates the CR-1 Core Router that connects to the mobile core
  • Routers PE 1, PE 2, PE 3 and P represents Telco’s MPLS Cloud

Telco MPLS Cloud

ISIS or OSPF is configured as IGP between PE 1, PE 2, PE 3 and P
– Creates IP reachability between all these routers

LDP is configured in PE 1, PE 2, PE 3 and P
– To create LSP (Labeled Switch Path) between the PEs

MP-BGP is configured in PE 1, PE 2, PE 3
– To exchange VRF routes and VPN label between the PEs

VRF Brown Routes

IGP Route Learning by PE 1 and PE 2 using OSPF:
– OSPF Routes of Branch-1 are learnt by PE 1 via CE 1
– OSPF Routes of HQ are learnt by PE 2 via CE 1

MP-BGP Route Exchange between PE 1 and PE 2:
– CE 1 Routes learnt by PE 1 are updated to PE 2 along-with a VPN label
– CE 2 Routes learnt by PE 2 are updated to PE 1 along-with a VPN label

IGP Route Exchange between HQ and Branch-1 using OSPF:
– CE 1 Routes learnt by PE 2 are distributed to CE 2
– CE 2 Routes learnt by PE 1 are distributed to CE 1

VRF Blue Routes

IGP Route Learning by PE 1 and PE 3 using ISIS:
– OSPF Routes of 5G Base Station are learnt by PE 1 via CE A
– OSPF Routes of Mobile Core are learnt by PE 3 via CE M

MP-BGP Route Exchange between PE 1 and PE 3:
– CE A Routes learnt by PE 1 are updated to PE 3 along-with a VPN label
– CE M Routes learnt by PE 3 are updated to PE 1 along-with a VPN label

IGP Route Exchange between 5G Base Station and Mobile Core using ISIS:
– CE A Routes learnt by PE 3 are distributed to CE M
– CE M Routes learnt by PE 1 are distributed to CE A

Figure 5 – Packet flow in MPLS Cloud

Figure 5 shows the packet traversal across the MPLS cloud. The traffic between CE to PE are IP packets, and the traffic between PEs is MPLS encapsulated. On executing a ping in CE 1 for 2.2.2.2 (CE 2 IP address):

  • Ping request if forwarded from CE 1 to PE 1
  • PE 1 encapsulates MPLS header and the VPN and LDP labels are pushed
  • P pops the outer LDP label
  • PE 2 decapsulates the MPLS header and pops the VPN label
  • CE 2 sends the ping response to CE 1 via PE 2

NiralOS Configuration

Figure 6 – Lab Topology for ping

In current topology:

  • PE 2 is running NiralOS on a white box
  • PE 1, P, CE 1 and CE 2 are Cisco routers
  • CE 1 to PE 1, CE 2 to PE 2 configured with static routes
  • Ping is initiated from CE 1 to CE 2

NiralOS Configuration in PE 2 with OSPF as IGP

OSPF Configuration

set protocols ospf area 0.0.0.0 network 4.4.4.4/32
set protocols ospf area 0.0.0.0 network 172.16.169.0/24

BGP Configuration

set protocols bgp 65000 address-family ipv4-vpn
set protocols bgp 65000 neighbor 2.2.2.2 address-family ipv4-vpn
set protocols bgp 65000 neighbor 2.2.2.2 remote-as 65000
set protocols bgp 65000 neighbor 2.2.2.2 update-source 4.4.4.4

BGP Configuration for VRF Brown

set routing routing-instance vrf-brown protocols bgp 65000 address-family ipv4-unicast redistribute connected
set routing routing-instance vrf-brown protocols bgp 65000 address-family ipv4-unicast redistribute static
set routing routing-instance vrf-brown protocols bgp 65000 address-family ipv4-unicast route-distinguisher 65000:1
set routing routing-instance vrf-brown protocols bgp 65000 address-family ipv4-unicast route-target 1:1 type both

LDP Configuration

set protocols mpls-ldp lsr-id 4.4.4.4
set protocols mpls-ldp address-family ipv4 label-policy allocate host-routes
set protocols mpls-ldp address-family ipv4 discovery interfaces interface dp0p192p1
set protocols mpls-ldp address-family ipv4 transport-address 4.4.4.4

NiralOS Configuration in PE 2 with ISIS as IGP

ISIS Configuration

set protocols isis 1 is-type level-2 
set protocols isis 1 net 10.0000.0000.0000.0000.0000.0000.0000.0000.0006.00
set protocols isis 1 redistribute ipv4 connected level-2
set protocols isis 1 redistribute ipv4 kernel level-2

BGP Configuration

set protocols bgp 65000 address-family ipv4-vpn
set protocols bgp 65000 neighbor 2.2.2.2 address-family ipv4-vpn
set protocols bgp 65000 neighbor 2.2.2.2 remote-as 65000
set protocols bgp 65000 neighbor 2.2.2.2 update-source 4.4.4.4

BGP Configuration for VRF Brown

set routing routing-instance vrf-brown protocols bgp 65000 address-family ipv4-unicast redistribute connected
set routing routing-instance vrf-brown protocols bgp 65000 address-family ipv4-unicast redistribute static
set routing routing-instance vrf-brown protocols bgp 65000 address-family ipv4-unicast route-distinguisher 65000:1
set routing routing-instance vrf-brown protocols bgp 65000 address-family ipv4-unicast route-target 1:1 type both

LDP Configuration

set protocols mpls-ldp lsr-id 4.4.4.4
set protocols mpls-ldp address-family ipv4 label-policy allocate host-routes
set protocols mpls-ldp address-family ipv4 discovery interfaces interface dp0p192p1
set protocols mpls-ldp address-family ipv4 transport-address 4.4.4.4

Ping packet captured in PE 2

The packet capture in PE 2 shows ping request and response packet with MPLS header. The capture confirms that ping is working between CE 1 and CE 2.

In this example, NiralOS is DCSG with MPLS-VPN Provider Edge capability and interoperates with multiple Cisco routers for IGP, BGP, LDP Signaling and IP, MPLS packet forwarding.