Multi-VRF DMVPN with NHS Cluster and IPSec IKEv1

Lab Introduction

This lab is related to my previous post DMVPN Phase3 IKEv1 and NHS Cluster. The previous post shows ‘the crypto keyring can only be tagged with fvrf’ and ‘fvrf on match statement of isakmp profile’. So what shall we do if we have a single FVRF (front door VRF) but multiple IVRFs (inside VRF)? – it would be easier if we have FVRF1 and IVRF1, FVRF2 and IVRF2.

This lab demonstrates one solution. DMVPN tunnel will use loopback address instead of physical WAN interface as source interface: DMVPN in IVRF1 with loopback IP 1.1.1.x as source address and DMVPN in IVRF2 with loopback IP 4.4.4.x as source address.

Although there is one FVRF shared by both IVRF1 and IVRF2, we make IVRF1 source 1.1.1.x/24 have a unique key Cisco1 and IVRF2 source 4.4.4.x/24 have a unique key Cisco2.

Similar to the previous lab, HUB1 and HUB2 forms NHS cluster with HUB1 as primary and HUB2 as backup.

Different from the previous lab that used a dummy switch to simulate WAN, this lab uses a router with hostname WAN to simulate WAN. Sites are connected to WAN using eBGP. WAN BGP AS is 7788; HUB1 and HUB2 are in the same AS 65111; SPOKE is in AS 65113.

Please note HUB1 doesn’t receive HUB2’s BGP route advertisement (see Verification section) due to they are in the same AS. BGP uses AS Path attribute to identify path and prevents routes advertised from the same AS from getting in, which is BGP loop prevention mechanism. This post is not going to discuss methods to override BGP loop prevention.

Topology

Topology is as below:

  • HUB1 and HUB2 forms NHS cluster with HUB1 as primary and HUB2 as backup
  • HUB1 and HUB2 are in BGP AS 65111
  • WAN is in BGP AS 7788
  • SPOKE is in BGP AS 65113
  • Physical WAN interface GE2 in FVRF
  • Tunnel 11 in IVRF1 with loopback1 as source, IPSec key Cisco1
  • Tunnel 41 in IVRF2 with loopback4 as source, IPsec key Cisco2

IKEv1_NHSCluster_Topo.jpg

Configuration

WAN

WAN#show run
Building configuration…Current configuration : 1621 bytes
!
! Last configuration change at 05:26:50 UTC Mon Jun 13 2016 by admin
!
version 15.5
service timestamps debug datetime msec
service timestamps log datetime msec
no platform punt-keepalive disable-kernel-core
platform console auto
!
hostname WAN
!
boot-start-marker
boot-end-marker
!
enable password cisco
!
no aaa new-model
!
subscriber templating
!
multilink bundle-name authenticated
!
license udi pid CSR1000V sn 9KW4WTP9Q5G
!
spanning-tree extend system-id
!
username admin password 0 cisco
!
redundancy
!
interface GigabitEthernet1
ip address 192.168.0.75 255.255.255.0
negotiation auto
!
interface GigabitEthernet2
ip address 200.10.10.1 255.255.255.252
negotiation auto
!
interface GigabitEthernet3
ip address 200.10.20.1 255.255.255.252
negotiation auto
!
interface GigabitEthernet4
ip address 200.10.30.1 255.255.255.252
negotiation auto
!
interface GigabitEthernet5
no ip address
shutdown
negotiation auto
!
interface GigabitEthernet6
no ip address
shutdown
negotiation auto
!
router bgp 7788
bgp log-neighbor-changes
neighbor 200.10.10.2 remote-as 65111
neighbor 200.10.20.2 remote-as 65111
neighbor 200.10.30.2 remote-as 65113
!
address-family ipv4
neighbor 200.10.10.2 activate
neighbor 200.10.20.2 activate
neighbor 200.10.30.2 activate
exit-address-family
!
ip forward-protocol nd
!
no ip http server
no ip http secure-server
!
control-plane
!
line con 0
stopbits 1
line vty 0 4
password cisco
login local
!
end

 

HUB1

HUB1#show run
Building configuration…Current configuration : 3736 bytes
!
! Last configuration change at 08:09:52 UTC Mon Jun 13 2016 by admin
!
version 15.5
service timestamps debug datetime msec
service timestamps log datetime msec
no platform punt-keepalive disable-kernel-core
platform console auto
!
hostname HUB1
!
boot-start-marker
boot-end-marker
!
vrf definition FVRF
rd 65111:11
!
address-family ipv4
exit-address-family
!
vrf definition IVRF1
!
address-family ipv4
exit-address-family
!
vrf definition IVRF2
!
address-family ipv4
exit-address-family
!
enable password cisco
!
no aaa new-model
!
subscriber templating
!
multilink bundle-name authenticated
!
license udi pid CSR1000V sn 9PULDF5K69H
!
spanning-tree extend system-id
!
username admin password 0 cisco
!
redundancy
!
crypto keyring FVRF vrf FVRF
pre-shared-key address 1.1.1.0 255.255.255.0 key Cisco1
pre-shared-key address 4.4.4.0 255.255.255.0 key Cisco4
!
crypto isakmp policy 1
encr aes
hash sha256
authentication pre-share
group 2
!
crypto isakmp policy 41
encr aes
hash sha256
authentication pre-share
group 2
crypto isakmp profile IVRF1-prof
keyring FVRF
match identity address 1.1.1.0 255.255.255.0 FVRF
local-address Loopback1
crypto isakmp profile IVRF2-prof
keyring FVRF
match identity address 4.4.4.0 255.255.255.0 FVRF
local-address Loopback4
!
crypto ipsec transform-set IVRF1 esp-aes esp-sha256-hmac
mode tunnel
crypto ipsec transform-set IVRF2 esp-aes esp-sha256-hmac
mode tunnel
!
crypto ipsec profile IVRF1-ipsec
set transform-set IVRF1
set isakmp-profile IVRF1-prof
!
crypto ipsec profile IVRF2-ipsec
set transform-set IVRF2
set isakmp-profile IVRF2-prof
!
interface Loopback1
vrf forwarding FVRF
ip address 1.1.1.10 255.255.255.255
!
interface Loopback4
vrf forwarding FVRF
ip address 4.4.4.10 255.255.255.255
!
interface Tunnel11
vrf forwarding IVRF1
ip address 172.16.11.1 255.255.255.0
no ip redirects
ip mtu 1400
ip nhrp authentication IVRF1
ip nhrp map multicast dynamic
ip nhrp network-id 11
ip nhrp holdtime 300
ip nhrp redirect
ip ospf network point-to-multipoint
ip ospf 1 area 0
tunnel source Loopback1
tunnel mode gre multipoint
tunnel key 11
tunnel vrf FVRF
tunnel protection ipsec profile IVRF1-ipsec
!
interface Tunnel41
vrf forwarding IVRF2
ip address 172.16.41.1 255.255.255.0
no ip redirects
ip mtu 1400
ip nhrp authentication IVRF2
ip nhrp map multicast dynamic
ip nhrp network-id 41
ip nhrp holdtime 300
ip nhrp redirect
ip ospf network point-to-multipoint
ip ospf 4 area 0
tunnel source Loopback4
tunnel mode gre multipoint
tunnel key 41
tunnel vrf FVRF
tunnel protection ipsec profile IVRF2-ipsec
!
interface GigabitEthernet1
ip address 192.168.0.71 255.255.255.0
negotiation auto
!
interface GigabitEthernet2
vrf forwarding FVRF
ip address 200.10.10.2 255.255.255.252
negotiation auto
!
interface GigabitEthernet3
no ip address
shutdown
negotiation auto
!
interface GigabitEthernet4
no ip address
shutdown
negotiation auto
!
interface GigabitEthernet5
no ip address
shutdown
negotiation auto
!
interface GigabitEthernet6
no ip address
shutdown
negotiation auto
!
router ospf 1 vrf IVRF1
!
router ospf 4 vrf IVRF2
!
router bgp 65111
bgp log-neighbor-changes
!
address-family ipv4 vrf FVRF
network 1.1.1.10 mask 255.255.255.255
network 4.4.4.10 mask 255.255.255.255
neighbor 200.10.10.1 remote-as 7788
neighbor 200.10.10.1 activate
exit-address-family
!
ip forward-protocol nd
!
no ip http server
no ip http secure-server
!
control-plane
!
line con 0
stopbits 1
line vty 0 4
login local
transport input telnet
!
end

HUB2

HUB2#show run
Building configuration…Current configuration : 3728 bytes
!
! Last configuration change at 08:09:00 UTC Mon Jun 13 2016 by admin
!
version 15.5
service timestamps debug datetime msec
service timestamps log datetime msec
no platform punt-keepalive disable-kernel-core
platform console auto
!
hostname HUB2
!
boot-start-marker
boot-end-marker
!
vrf definition FVRF
rd 65111:12
!
address-family ipv4
exit-address-family
!
vrf definition IVRF1
!
address-family ipv4
exit-address-family
!
vrf definition IVRF2
!
address-family ipv4
exit-address-family
!
enable password cisco
!
no aaa new-model
!
subscriber templating
!
multilink bundle-name authenticated
!
license udi pid CSR1000V sn 97Y8E4Z0O3Q
!
spanning-tree extend system-id
!
username admin password 0 cisco
!
redundancy
!
crypto keyring FVRF vrf FVRF
pre-shared-key address 1.1.1.0 255.255.255.0 key Cisco1
pre-shared-key address 4.4.4.0 255.255.255.0 key Cisco4
!
crypto isakmp policy 1
encr aes
hash sha256
authentication pre-share
group 2
!
crypto isakmp policy 41
encr aes
hash sha256
authentication pre-share
group 2
crypto isakmp profile IVRF1-prof
keyring FVRF
match identity address 1.1.1.0 255.255.255.0 FVRF
local-address Loopback1
crypto isakmp profile IVRF2-prof
keyring FVRF
match identity address 4.4.4.0 255.255.255.0 FVRF
local-address Loopback4
!
crypto ipsec transform-set IVRF1 esp-aes esp-sha256-hmac
mode tunnel
crypto ipsec transform-set IVRF2 esp-aes esp-sha256-hmac
mode tunnel
!
crypto ipsec profile IVRF1-ipsec
set transform-set IVRF1
set isakmp-profile IVRF1-prof
!
crypto ipsec profile IVRF2-ipsec
set transform-set IVRF2
set isakmp-profile IVRF2-prof
!
interface Loopback1
vrf forwarding FVRF
ip address 1.1.1.20 255.255.255.255
!
interface Loopback4
vrf forwarding FVRF
ip address 4.4.4.20 255.255.255.255
!
interface Tunnel11
vrf forwarding IVRF1
ip address 172.16.11.2 255.255.255.0
no ip redirects
ip mtu 1400
ip nhrp authentication IVRF1
ip nhrp map multicast dynamic
ip nhrp network-id 11
ip nhrp holdtime 300
ip nhrp redirect
ip ospf network point-to-multipoint
ip ospf 1 area 0
tunnel source Loopback1
tunnel mode gre multipoint
tunnel key 11
tunnel vrf FVRF
tunnel protection ipsec profile IVRF1-ipsec
!
interface Tunnel41
vrf forwarding IVRF2
ip address 172.16.41.2 255.255.255.0
no ip redirects
ip mtu 1400
ip nhrp authentication IVRF2
ip nhrp map multicast dynamic
ip nhrp network-id 41
ip nhrp holdtime 300
ip nhrp redirect
ip ospf network point-to-multipoint
ip ospf 4 area 0
tunnel source Loopback4
tunnel mode gre multipoint
tunnel key 41
tunnel vrf FVRF
tunnel protection ipsec profile IVRF2-ipsec
!
interface GigabitEthernet1
ip address 192.168.0.72 255.255.255.0
negotiation auto
!
interface GigabitEthernet2
vrf forwarding FVRF
ip address 200.10.20.2 255.255.255.252
negotiation auto
!
interface GigabitEthernet3
no ip address
shutdown
negotiation auto
!
interface GigabitEthernet4
no ip address
shutdown
negotiation auto
!
interface GigabitEthernet5
no ip address
shutdown
negotiation auto
!
interface GigabitEthernet6
no ip address
shutdown
negotiation auto
!
router ospf 1 vrf IVRF1
!
router ospf 4 vrf IVRF2
!
router bgp 65111
bgp log-neighbor-changes
!
address-family ipv4 vrf FVRF
network 1.1.1.20 mask 255.255.255.255
network 4.4.4.20 mask 255.255.255.255
neighbor 200.10.20.1 remote-as 7788
neighbor 200.10.20.1 activate
exit-address-family
!
ip forward-protocol nd
!
no ip http server
no ip http secure-server
!
control-plane
!
line con 0
stopbits 1
line vty 0 4
password cisco
login local
!
end

 

SPOKE

SPOKE#show run
Building configuration…Current configuration : 4247 bytes
!
! Last configuration change at 08:09:04 UTC Mon Jun 13 2016 by admin
!
version 15.5
service timestamps debug datetime msec
service timestamps log datetime msec
no platform punt-keepalive disable-kernel-core
platform console auto
!
hostname SPOKE
!
boot-start-marker
boot-end-marker
!
vrf definition FVRF
rd 65113:11
!
address-family ipv4
exit-address-family
!
vrf definition IVRF1
!
address-family ipv4
exit-address-family
!
vrf definition IVRF2
!
address-family ipv4
exit-address-family
!
enable password cisco
!
no aaa new-model
!
subscriber templating
!
multilink bundle-name authenticated
!
license udi pid CSR1000V sn 9N9D03A6MVB
!
spanning-tree extend system-id
!
username admin password 0 cisco
!
redundancy
!
crypto keyring FVRF vrf FVRF
pre-shared-key address 1.1.1.0 255.255.255.0 key Cisco1
pre-shared-key address 4.4.4.0 255.255.255.0 key Cisco4
!
crypto isakmp policy 1
encr aes
hash sha256
authentication pre-share
group 2
!
crypto isakmp policy 41
encr aes
hash sha256
authentication pre-share
group 2
crypto isakmp profile IVRF1-prof
keyring FVRF
match identity address 1.1.1.0 255.255.255.0 FVRF
local-address Loopback1
crypto isakmp profile IVRF2-prof
keyring FVRF
match identity address 4.4.4.0 255.255.255.0 FVRF
local-address Loopback4
!
!
crypto ipsec transform-set IVRF1 esp-aes esp-sha256-hmac
mode tunnel
crypto ipsec transform-set IVRF2 esp-aes esp-sha256-hmac
mode tunnel
!
crypto ipsec profile IVRF1-ipsec
set transform-set IVRF1
set isakmp-profile IVRF1-prof
!
crypto ipsec profile IVRF2-ipsec
set transform-set IVRF2
set isakmp-profile IVRF2-prof
!
interface Loopback1
vrf forwarding FVRF
ip address 1.1.1.30 255.255.255.255
!
interface Loopback4
vrf forwarding FVRF
ip address 4.4.4.30 255.255.255.255
!
interface Tunnel11
vrf forwarding IVRF1
ip address 172.16.11.3 255.255.255.0
no ip redirects
ip mtu 1400
ip nhrp authentication IVRF1
ip nhrp map multicast 1.1.1.10
ip nhrp map multicast 1.1.1.20
ip nhrp map 172.16.11.1 1.1.1.10
ip nhrp map 172.16.11.2 1.1.1.20
ip nhrp network-id 11
ip nhrp holdtime 300
ip nhrp nhs 172.16.11.1 priority 1 cluster 1
ip nhrp nhs 172.16.11.2 priority 2 cluster 1
ip nhrp nhs cluster 1 max-connections 2
ip nhrp nhs fallback 25
ip nhrp shortcut
ip ospf network point-to-multipoint
ip ospf 1 area 0
tunnel source Loopback1
tunnel mode gre multipoint
tunnel key 11
tunnel vrf FVRF
tunnel protection ipsec profile IVRF1-ipsec
!
interface Tunnel41
vrf forwarding IVRF2
ip address 172.16.41.3 255.255.255.0
no ip redirects
ip mtu 1400
ip nhrp authentication IVRF2
ip nhrp map multicast 4.4.4.10
ip nhrp map multicast 4.4.4.20
ip nhrp map 172.16.41.1 4.4.4.10
ip nhrp map 172.16.41.2 4.4.4.20
ip nhrp network-id 41
ip nhrp holdtime 300
ip nhrp nhs 172.16.41.1 priority 1 cluster 4
ip nhrp nhs 172.16.41.2 priority 2 cluster 4
ip nhrp nhs cluster 4 max-connections 2
ip nhrp nhs fallback 25
ip nhrp shortcut
ip ospf network point-to-multipoint
ip ospf 4 area 0
tunnel source Loopback4
tunnel mode gre multipoint
tunnel key 41
tunnel vrf FVRF
tunnel protection ipsec profile IVRF2-ipsec
!
interface GigabitEthernet1
ip address 192.168.0.73 255.255.255.0
negotiation auto
!
interface GigabitEthernet2
vrf forwarding FVRF
ip address 200.10.30.2 255.255.255.252
negotiation auto
!
interface GigabitEthernet3
no ip address
shutdown
negotiation auto
!
interface GigabitEthernet4
no ip address
shutdown
negotiation auto
!
interface GigabitEthernet5
no ip address
shutdown
negotiation auto
!
interface GigabitEthernet6
no ip address
shutdown
negotiation auto
!
router ospf 1 vrf IVRF1
!
router ospf 4 vrf IVRF2
!
router bgp 65113
bgp log-neighbor-changes
!
address-family ipv4 vrf FVRF
network 1.1.1.30 mask 255.255.255.255
network 4.4.4.30 mask 255.255.255.255
neighbor 200.10.30.1 remote-as 7788
neighbor 200.10.30.1 activate
exit-address-family
!
ip forward-protocol nd
!
no ip http server
no ip http secure-server
!
control-plane
!
line con 0
stopbits 1
line vty 0 4
password cisco
login local
!
end

 

Verification

SPOKE# show ip nhrp nhs
Legend: E=Expecting replies, R=Responding, W=Waiting
Tunnel11:
172.16.11.1 RE priority = 1 cluster = 1
172.16.11.2 RE priority = 2 cluster = 1
Tunnel41:
172.16.41.1 RE priority = 1 cluster = 4
172.16.41.2 RE priority = 2 cluster = 4

 

SPOKE#show ip ospf nei
Neighbor ID Pri State Dead Time Address Interface
172.16.41.1 0 FULL/ – 00:01:43 172.16.41.1 Tunnel41
172.16.41.2 0 FULL/ – 00:01:50 172.16.41.2 Tunnel41
172.16.11.2 0 FULL/ – 00:01:48 172.16.11.2 Tunnel11
172.16.11.1 0 FULL/ – 00:01:56 172.16.11.1 Tunnel11

 

SPOKE#show crypto session
Crypto session current status
Interface: Tunnel11
Profile: IVRF1-prof
Session status: UP-ACTIVE
Peer: 1.1.1.20 port 500
Session ID: 0
IKEv1 SA: local 1.1.1.30/500 remote 1.1.1.20/500 Active
Session ID: 0
IKEv1 SA: local 1.1.1.30/500 remote 1.1.1.20/500 Active
IPSEC FLOW: permit 47 host 1.1.1.30 host 1.1.1.20
Active SAs: 4, origin: crypto mapInterface: Tunnel11
Profile: IVRF1-prof
Session status: UP-ACTIVE
Peer: 1.1.1.10 port 500
Session ID: 0
IKEv1 SA: local 1.1.1.30/500 remote 1.1.1.10/500 Active
Session ID: 0
IKEv1 SA: local 1.1.1.30/500 remote 1.1.1.10/500 Active
IPSEC FLOW: permit 47 host 1.1.1.30 host 1.1.1.10
Active SAs: 6, origin: crypto map
Interface: Tunnel41
Profile: IVRF2-prof
Session status: UP-ACTIVE
Peer: 4.4.4.20 port 500
Session ID: 0
IKEv1 SA: local 4.4.4.30/500 remote 4.4.4.20/500 Active
IPSEC FLOW: permit 47 host 4.4.4.30 host 4.4.4.20
Active SAs: 2, origin: crypto map

Interface: Tunnel41
Profile: IVRF2-prof
Session status: UP-ACTIVE
Peer: 4.4.4.10 port 500
Session ID: 0
IKEv1 SA: local 4.4.4.30/500 remote 4.4.4.10/500 Active
IPSEC FLOW: permit 47 host 4.4.4.30 host 4.4.4.10
Active SAs: 2, origin: crypto map

 

SPOKE#ping vrf IVRF1 172.16.11.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 172.16.11.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 8/18/35 msSPOKE#ping vrf IVRF1 172.16.11.2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 172.16.11.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 7/18/35 ms

SPOKE#ping vrf IVRF2 172.16.41.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 172.16.41.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 7/18/35 ms

SPOKE#ping vrf IVRF2 172.16.41.2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 172.16.41.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 7/21/50 ms

 

HUB1#show bgp vpnv4 unicast all
BGP table version is 5, local router ID is 192.168.0.71
Status codes: s suppressed, d damped, h history, * valid, > best, i – internal,
r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
x best-external, a additional-path, c RIB-compressed,
Origin codes: i – IGP, e – EGP, ? – incomplete
RPKI validation codes: V valid, I invalid, N Not foundNetwork Next Hop Metric LocPrf Weight Path
Route Distinguisher: 65111:11 (default for vrf FVRF)
*> 1.1.1.10/32 0.0.0.0 0 32768 i
*> 1.1.1.30/32 200.10.10.1 0 7788 65113 i
*> 4.4.4.10/32 0.0.0.0 0 32768 i
*> 4.4.4.30/32 200.10.10.1 0 7788 65113 i

 

HUB1#show ip route vrf FVRF
1.0.0.0/32 is subnetted, 2 subnets
C 1.1.1.10 is directly connected, Loopback1
(1.1.1.20 not here)
B 1.1.1.30 [20/0] via 200.10.10.1, 02:00:48
4.0.0.0/32 is subnetted, 2 subnets
C 4.4.4.10 is directly connected, Loopback4
(4.4.4.20 not here)
B 4.4.4.30 [20/0] via 200.10.10.1, 02:00:18
200.10.10.0/24 is variably subnetted, 2 subnets, 2 masks
C 200.10.10.0/30 is directly connected, GigabitEthernet2
L 200.10.10.2/32 is directly connected, GigabitEthernet2

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s