Open ljyfree opened 4 years ago
tnaganawa
commented
4 years ago Hi, is it possible to try a subnet 10.10.30.0/24 in default-project-ipam?
ljyfree
commented
4 years ago Hi tnaganawa
According to my understanding for your last reply, I made following operationsStep 1:Create test-ipam-04
Step 2:Specify this ipam
Step 3: And extend to vmx
Step 4: Check new added configuration on vmx, but still
set groups contrail firewall family inet filter _contrail_redirect-to-public-vrfs-inet4 term term-_contrail_k8s-default-pod-network-l3-5 from destination-address 0.0.0.0/0 set groups contrail firewall family inet filter _contrail_redirect-to-public-vrfs-inet4 term term-_contrail_k8s-default-pod-network-l3-5 then routing-instance _contrail_k8s-default-pod-network-l3-5 set groups contrail firewall family inet filter _contrail_redirect-to-public-vrfs-inet4 term default-term then accept
Still "0.0.0.0/0" instead of "10.10.40.0/24". :(
Could you please correct me if any step was not same as you tried to describe?
tnaganawa
commented
4 years ago Hi,
Sorry for my late reply ..
Could you try 'User Defined' in 'Subnets' > 'Allocation Mode'?
If it worked, destination-address will be filled with that subnet .. https://github.com/tnaganawa/contrail-k8s-tutorial/blob/master/vmx-config/vmx-config-k8s-ecmp-loadbalance#L158
Best Regards, /// Tatsuya
ljyfree
commented
4 years ago Hi tnaganawa
According to your suggestion, the following configuration had been deployed to vmxset groups __contrail__ forwarding-options family inet filter input _contrail_redirect-to-public-vrfs-inet4
set groups __contrail__ policy-options policy-statement _contrail_ibgp_export_policy term inet-vpn from family inet-vpn
set groups __contrail__ policy-options policy-statement _contrail_ibgp_export_policy term inet-vpn then next-hop self
set groups __contrail__ policy-options policy-statement _contrail_net03-l3-13-export term t1 then community add _contrail_target_64512_8000010
set groups __contrail__ policy-options policy-statement _contrail_net03-l3-13-export term t1 then accept
set groups __contrail__ policy-options policy-statement _contrail_net03-l3-13-import term t1 from community _contrail_target_64512_8000010
set groups __contrail__ policy-options policy-statement _contrail_net03-l3-13-import term t1 then accept
set groups __contrail__ policy-options policy-statement _contrail_net03-l3-13-import then reject
set groups __contrail__ policy-options community _contrail_target_64512_8000010 members target:64512:8000010
set groups __contrail__ firewall family inet filter _contrail_redirect-to-public-vrfs-inet4 term term-_contrail_net03-l3-13 from destination-address 10.10.30.0/24
set groups __contrail__ firewall family inet filter _contrail_redirect-to-public-vrfs-inet4 term term-_contrail_net03-l3-13 then routing-instance _contrail_net03-l3-13
set groups __contrail__ firewall family inet filter _contrail_redirect-to-public-vrfs-inet4 term default-term then accept
set groups __contrail__ routing-instances _contrail_net03-l3-13 instance-type vrf
set groups __contrail__ routing-instances _contrail_net03-l3-13 vrf-import _contrail_net03-l3-13-import
set groups __contrail__ routing-instances _contrail_net03-l3-13 vrf-export _contrail_net03-l3-13-export
set groups __contrail__ routing-instances _contrail_net03-l3-13 vrf-table-label
set groups __contrail__ routing-instances _contrail_net03-l3-13 routing-options static route 0.0.0.0/0 next-table inet.0
set groups __contrail__ routing-instances _contrail_net03-l3-13 routing-options static route 10.10.30.0/24 discard
set groups __contrail__ routing-instances _contrail_net03-l3-13 routing-options auto-export family inet unicast
set groups __contrail__ routing-options static route 10.10.30.0/24 discard
set groups __contrail__ routing-options router-id 2.2.2.2
set groups __contrail__ routing-options route-distinguisher-id 2.2.2.2
set groups __contrail__ routing-options autonomous-system 64512
set groups __contrail__ routing-options dynamic-tunnels _contrail_asn-64512 source-address 2.2.2.2
set groups __contrail__ routing-options dynamic-tunnels _contrail_asn-64512 gre
set groups __contrail__ routing-options dynamic-tunnels _contrail_asn-64512 destination-networks 192.168.122.0/24
set groups __contrail__ routing-options dynamic-tunnels _contrail_asn-64512 destination-networks 192.168.122.90/32
set groups __contrail__ routing-options dynamic-tunnels _contrail_asn-64512 destination-networks 2.2.2.2/32
set groups __contrail__ protocols bgp group _contrail_asn-64512 type internal
set groups __contrail__ protocols bgp group _contrail_asn-64512 local-address 2.2.2.2
set groups __contrail__ protocols bgp group _contrail_asn-64512 hold-time 90
set groups __contrail__ protocols bgp group _contrail_asn-64512 family inet-vpn unicast
set groups __contrail__ protocols bgp group _contrail_asn-64512 family evpn signaling
set groups __contrail__ protocols bgp group _contrail_asn-64512 family route-target
set groups __contrail__ protocols bgp group _contrail_asn-64512 export _contrail_ibgp_export_policy
set groups __contrail__ protocols bgp group _contrail_asn-64512 neighbor 192.168.122.90 peer-as 64512
set apply-groups __contrail__
set system login user netops uid 203
set system login user netops class super-user
set system login user netops authentication encrypted-password "$6$o88FsGts$jtlGrsUF53qQkZmO/YfZ2cqyQYQBznMvVV/OeeS0MtKKX1CFHjHewBaJKegnZUX8LTQ2BOkwPDsZcT37iSMtJ/"
set system root-authentication encrypted-password "$6$C3JML9Yg$f8V/Oeadkxu.YhJ2ZIwfUQq48aqNc4/Rd0ydQA3XNsOtkZyGd5j39h9XER/Y3IoS3AeK8bXcxliYg1hcduQEB0"
set system services ftp
set system services ssh root-login allow
set system services ssh protocol-version v2
set system services ssh port 22
set system services netconf ssh port 22
set system services netconf traceoptions file nc
set system services netconf traceoptions flag all
set system services rest http addresses 192.168.122.102
set system services rest control allowed-sources 192.168.122.1
set system services rest control allowed-sources 192.168.122.177
set system services rest control allowed-sources 192.168.122.80
set system services rest enable-explorer
set system host-name vMX-102
set system syslog user * any emergency
set system syslog file messages any notice
set system syslog file messages authorization info
set system syslog file interactive-commands interactive-commands any
set interfaces lo0 unit 0 family inet address 2.2.2.2/32
set interfaces vtnet2 unit 0 family inet address 192.168.122.102/24
set interfaces vtnet3 unit 0 family inet address 60.1.1.2/24So it works!
However I still need some suggestion for my test for SDN gateway.
The topo as follows based on EVE-NG
My purpose is to ping from deployer79(60.1.1.1) to the pod on Node91
[root@master90 Dockerfile]# kubectl get pods -n test-ns1 -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE
busybox01-ns1-net03 1/1 Running 176 9d 10.10.30.3 node91 <none>
[root@master90 Dockerfile]# I keep on ping on deployer79
[root@deploy ~]# ip route
default via 192.168.122.1 dev eth0 proto static metric 100
10.10.30.0/24 via 60.1.1.2 dev eth1
60.1.1.0/24 dev eth1 proto kernel scope link src 60.1.1.1
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1
192.168.122.0/24 dev eth0 proto kernel scope link src 192.168.122.79 metric 100
[root@deploy ~]#
[root@deploy ~]# ping 10.10.30.3
PING 10.10.30.3 (10.10.30.3) 56(84) bytes of data.
And capture on Node91 where the pod located and find
[root@node91 ~]# tcpdump -i eth0 -ennn host 2.2.2.2
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
01:58:12.022743 50:00:00:05:00:02 > 00:50:00:00:07:00, ethertype IPv4 (0x0800), length 126: 2.2.2.2 > 192.168.122.91: GREv0, proto MPLS unicast (0x8847), length 92: MPLS (label 33, exp 0, [S], ttl 63) 60.1.1.1 > 10.10.30.3: ICMP echo re4
01:58:12.024608 00:50:00:00:07:00 > 02:21:3c:c1:6d:a5, ethertype IPv4 (0x0800), length 126: 192.168.122.91 > 2.2.2.2: GREv0, proto MPLS unicast (0x8847), length 92: MPLS (label 16, exp 0, [S], ttl 63) 10.10.30.3 > 60.1.1.1: ICMP echo re4
01:58:13.024253 50:00:00:05:00:02 > 00:50:00:00:07:00, ethertype IPv4 (0x0800), length 126: 2.2.2.2 > 192.168.122.91: GREv0, proto MPLS unicast (0x8847), length 92: MPLS (label 33, exp 0, [S], ttl 63) 60.1.1.1 > 10.10.30.3: ICMP echo re4
01:58:13.026292 00:50:00:00:07:00 > 02:21:3c:c1:6d:a5, ethertype IPv4 (0x0800), length 126: 192.168.122.91 > 2.2.2.2: GREv0, proto MPLS unicast (0x8847), length 92: MPLS (label 16, exp 0, [S], ttl 63) 10.10.30.3 > 60.1.1.1: ICMP echo re4
01:58:14.026604 50:00:00:05:00:02 > 00:50:00:00:07:00, ethertype IPv4 (0x0800), length 126: 2.2.2.2 > 192.168.122.91: GREv0, proto MPLS unicast (0x8847), length 92: MPLS (label 33, exp 0, [S], ttl 63) 60.1.1.1 > 10.10.30.3: ICMP echo re4
01:58:14.028826 00:50:00:00:07:00 > 02:21:3c:c1:6d:a5, ethertype IPv4 (0x0800), length 126: 192.168.122.91 > 2.2.2.2: GREv0, proto MPLS unicast (0x8847), length 92: MPLS (label 16, exp 0, [S], ttl 63) 10.10.30.3 > 60.1.1.1: ICMP echo re4
^CSo it means the pod had received ICMP Request and transmit ICMP Reply. Unfortunately，vmx did not decapsulate the Reply in MPLSoUDP and forward to deployer. I don't know why the ICMP request could span from default vrf to _contrail_net03-l3-13 but ICMP Reply could not span from _contrail_net03-l3-13 to default vrf
Any suggestion?
netops@vMX-102> show dynamic-tunnels database
*- Signal Tunnels #- PFE-down
Table: inet.3
Destination-network: 2.2.2.2/32
Destination-network: 192.168.122.0/24
Tunnel to: 192.168.122.91/32 State: Up
Reference count: 1
Next-hop type: gre
Source address: 2.2.2.2
Next hop: gre.32770
State: Up
Destination-network: 192.168.122.90/32
Tunnel to: 192.168.122.90/32 State: Up
Reference count: 1
Next-hop type: gre
Source address: 2.2.2.2
Next hop: gre.32769
State: Up
netops@vMX-102>
netops@vMX-102> show route forwarding-table
Routing table: default.inet
Internet:
Enabled protocols: Bridging,
Destination Type RtRef Next hop Type Index NhRef Netif
default perm 0 rjct 36 1
0.0.0.0/32 perm 0 dscd 34 2
2.2.2.2/32 intf 0 2.2.2.2 locl 516 1
2.2.2.2.192.168.122.90.47/72
dest 0 locl 526 1
2.2.2.2.192.168.122.91.47/72
dest 0 locl 572 1
10.10.30.0/24 user 0 dscd 34 2
60.1.1.0/24 intf 0 rslv 524 1 vtnet3.0
60.1.1.0/32 dest 0 60.1.1.0 recv 522 1 vtnet3.0
60.1.1.1/32 dest 0 0:50:0:0:b:1 ucst 575 1 vtnet3.0
60.1.1.2/32 intf 0 60.1.1.2 locl 523 2
60.1.1.2/32 dest 0 60.1.1.2 locl 523 2
60.1.1.255/32 dest 0 60.1.1.255 bcst 521 1 vtnet3.0
192.168.122.0/24 intf 0 rslv 520 1 vtnet2.0
192.168.122.0/32 dest 0 192.168.122.0 recv 518 1 vtnet2.0
192.168.122.90/32 dest 2 0:50:0:0:6:0 ucst 525 3 vtnet2.0
192.168.122.91/32 dest 1 0:50:0:0:7:0 ucst 576 2 vtnet2.0
192.168.122.102/32 intf 0 192.168.122.102 locl 519 2
192.168.122.102/32 dest 0 192.168.122.102 locl 519 2
192.168.122.255/32 dest 0 192.168.122.255 bcst 517 1 vtnet2.0
224.0.0.0/4 perm 0 mdsc 35 1
224.0.0.1/32 perm 0 224.0.0.1 mcst 31 1
255.255.255.255/32 perm 0 bcst 32 1
Routing table: __master.anon__.inet
Internet:
Enabled protocols: Bridging, Dual VLAN,
Destination Type RtRef Next hop Type Index NhRef Netif
default perm 0 rjct 531 1
0.0.0.0/32 perm 0 dscd 529 1
224.0.0.0/4 perm 0 mdsc 530 1
224.0.0.1/32 perm 0 224.0.0.1 mcst 533 1
255.255.255.255/32 perm 0 bcst 534 1
Routing table: _contrail_net03-l3-13.inet
Internet:
Enabled protocols: Bridging, All VLANs,
Destination Type RtRef Next hop Type Index NhRef Netif
default user 0 rtbl 1 3
default perm 0 rjct 553 1
0.0.0.0/32 perm 0 dscd 527 2
10.10.30.0/24 user 0 dscd 527 2
10.10.30.3/32 user 0 indr 131070 2
Push 33 574 2 gre.32770
224.0.0.0/4 perm 0 mdsc 528 1
224.0.0.1/32 perm 0 224.0.0.1 mcst 555 1
255.255.255.255/32 perm 0 bcst 556 1
Routing table: default.iso
ISO:
Enabled protocols: Bridging,
Destination Type RtRef Next hop Type Index NhRef Netif
default perm 0 rjct 60 1
Routing table: __master.anon__.iso
ISO:
Enabled protocols: Bridging, Dual VLAN,
Destination Type RtRef Next hop Type Index NhRef Netif
default perm 0 rjct 540 1
Routing table: _contrail_net03-l3-13.iso
ISO:
Enabled protocols: Bridging, All VLANs,
Destination Type RtRef Next hop Type Index NhRef Netif
default perm 0 rjct 562 1
Routing table: default.inet6
Internet6:
Enabled protocols: Bridging,
Destination Type RtRef Next hop Type Index NhRef Netif
default perm 0 rjct 44 1
::/128 perm 0 dscd 42 2
::1/128 perm 0 locl 45 1
fe80::/10 perm 0 dscd 42 2
ff00::/8 perm 0 mdsc 43 1
ff02::1/128 perm 0 ff02::1 mcst 39 2
ff02::2/128 user 0 ff02::2 mcst 39 2
Routing table: __master.anon__.inet6
Internet6:
Enabled protocols: Bridging, Dual VLAN,
Destination Type RtRef Next hop Type Index NhRef Netif
default perm 0 rjct 547 1
::/128 perm 0 dscd 545 2
::1/128 perm 0 locl 548 1
fe80::/10 perm 0 dscd 545 2
ff00::/8 perm 0 mdsc 546 1
ff02::1/128 perm 0 ff02::1 mcst 549 1
Routing table: _contrail_net03-l3-13.inet6
Internet6:
Enabled protocols: Bridging, All VLANs,
Destination Type RtRef Next hop Type Index NhRef Netif
default perm 0 rjct 566 1
::/128 perm 0 dscd 564 2
::1/128 perm 0 locl 567 1
fe80::/10 perm 0 dscd 564 2
ff00::/8 perm 0 mdsc 565 1
ff02::1/128 perm 0 ff02::1 mcst 568 2
ff02::2/128 user 0 ff02::2 mcst 568 2
Routing table: default.mpls
MPLS:
Destination Type RtRef Next hop Type Index NhRef Netif
default perm 0 dscd 50 1
Routing table: __mpls-oam__.mpls
MPLS:
Enabled protocols: Bridging, Single VLAN, Dual VLAN,
Destination Type RtRef Next hop Type Index NhRef Netif
default perm 0 dscd 542 1
Routing table: default.dhcp-snooping
DHCP Snooping:
Destination Type RtRef Next hop Type Index NhRef Netif
default perm 0 dscd 98 1
netops@vMX-102> Hmm, I might be wrong, but could you try to add set chassis fpc 0 pic 0 tunnel-service ? https://github.com/tnaganawa/contrail-k8s-tutorial/blob/master/vmx-config/vmx-config-k8s-ecmp-loadbalance#L216
Or, if vMX license also might be an issue .. Could I see the result of
show system license ?
ljyfree
commented
4 years ago Hi tnaganawa
There is no “set chassis fpc 0 pic 0 tunnel-service” available.
netops@vMX-102# set chassis fpc 0 pic 0 ?
Possible completions:
> adaptive-services Adaptive services configuration
aggregate-ports Aggregate multiple ports on a PIC as a single port
+ apply-groups Groups from which to inherit configuration data
+ apply-groups-except Don't inherit configuration data from these groups
atm-cell-relay-accumulation Enable ATM cell-relay accumulation mode
> atm-l2circuit-mode Enable ATM Layer 2 circuit transport mode
> ce1 CE1 NxDS0 PIC configuration
> ct3 CT3 NxDS0 PIC configuration
egress-policer-overhead Number of policer overhead bytes in egress (bytes)
framing Framing mode
> idle-cell-format ATM idle cell configuration
ingress-policer-overhead Number of policer overhead bytes in ingress (bytes)
mlfr-uni-nni-bundles Number of multilink Frame Relay UNI NNI (FRF.16) bundles to allocate on PIC
mlfr-uni-nni-bundles-inline Number of inline multilink frame relay UNI NNI bundles
no-concatenate Do not concatenate channels
no-multi-rate Disable multi-rate mode
> port Port number
> q-pic-large-buffer Run in large delay buffer mode
> red-buffer-occupancy Computation type for RED buffer occupancy
sparse-dlcis Run in sparse data-link connection identifier mode
> traffic-manager Configure traffic manager attributes
vtmapping Virtual tunnel mapping mode
[edit]
netops@vMX-102# set chassis fpc 0 pic 0 tun
^
syntax error.
netops@vMX-102# show system license
[edit]
netops@vMX-102#
netops@vMX-102> show version
Hostname: vMX-102
Model: olive
Junos: 18.4R3.3
JUNOS OS Kernel 64-bit [20191211.fa5e90e_builder_stable_11]
JUNOS OS libs [20191211.fa5e90e_builder_stable_11]
JUNOS OS runtime [20191211.fa5e90e_builder_stable_11]
JUNOS OS time zone information [20191211.fa5e90e_builder_stable_11]
JUNOS network stack and utilities [20191221.040256_builder_junos_184_r3]
JUNOS libs [20191221.040256_builder_junos_184_r3]
JUNOS OS libs compat32 [20191211.fa5e90e_builder_stable_11]
JUNOS OS 32-bit compatibility [20191211.fa5e90e_builder_stable_11]
JUNOS libs compat32 [20191221.040256_builder_junos_184_r3]
JUNOS runtime [20191221.040256_builder_junos_184_r3]
JUNOS Packet Forwarding Engine Simulation Package [20191221.040256_builder_junos_184_r3]
JUNOS sflow mx [20191221.040256_builder_junos_184_r3]
JUNOS py extensions [20191221.040256_builder_junos_184_r3]
JUNOS py base [20191221.040256_builder_junos_184_r3]
JUNOS OS vmguest [20191211.fa5e90e_builder_stable_11]
JUNOS OS crypto [20191211.fa5e90e_builder_stable_11]
JUNOS na telemetry [18.4R3.3]
JUNOS mx libs compat32 [20191221.040256_builder_junos_184_r3]
JUNOS mx runtime [20191221.040256_builder_junos_184_r3]
JUNOS common platform support [20191221.040256_builder_junos_184_r3]
JUNOS Openconfig [18.4R3.3]
JUNOS mtx network modules [20191220.185702_builder_junos_184_r3]
JUNOS modules [20191221.040256_builder_junos_184_r3]
JUNOS mx modules [20191221.040256_builder_junos_184_r3]
JUNOS mx libs [20191221.040256_builder_junos_184_r3]
JUNOS SQL Sync Daemon [20191221.040256_builder_junos_184_r3]
JUNOS mtx Data Plane Crypto Support [20191221.040256_builder_junos_184_r3]
JUNOS daemons [20191221.040256_builder_junos_184_r3]
JUNOS mx daemons [20191221.040256_builder_junos_184_r3]
JUNOS -MX appidd application-identification daemon [20191221.040256_builder_junos_184_r3]
JUNOS Simulation Linux Package [20191221.040256_builder_junos_184_r3]
JUNOS Simulation Package [20191221.040256_builder_junos_184_r3]
JUNOS Services URL Filter package [20191221.040256_builder_junos_184_r3]
JUNOS Services TLB Service PIC package [20191221.040256_builder_junos_184_r3]
JUNOS Services Telemetry [20191221.040256_builder_junos_184_r3]
JUNOS Services TCP-LOG [20191221.040256_builder_junos_184_r3]
JUNOS Services SSL [20191221.040256_builder_junos_184_r3]
JUNOS Services SOFTWIRE [20191221.040256_builder_junos_184_r3]
JUNOS Services Stateful Firewall [20191221.040256_builder_junos_184_r3]
JUNOS Services RTCOM [20191221.040256_builder_junos_184_r3]
JUNOS Services RPM [20191221.040256_builder_junos_184_r3]
JUNOS Services PCEF package [20191221.040256_builder_junos_184_r3]
JUNOS Services NAT [20191221.040256_builder_junos_184_r3]
JUNOS Services Mobile Subscriber Service Container package [20191221.040256_builder_junos_184_r3]
JUNOS Services MobileNext Software package [20191221.040256_builder_junos_184_r3]
JUNOS Services Logging Report Framework package [20191221.040256_builder_junos_184_r3]
JUNOS Services LL-PDF Container package [20191221.040256_builder_junos_184_r3]
JUNOS Services Jflow Container package [20191221.040256_builder_junos_184_r3]
JUNOS Services Deep Packet Inspection package [20191221.040256_builder_junos_184_r3]
JUNOS Services IPSec [20191221.040256_builder_junos_184_r3]
JUNOS Services IDS [20191221.040256_builder_junos_184_r3]
JUNOS IDP Services [20191221.040256_builder_junos_184_r3]
JUNOS Services HTTP Content Management package [20191221.040256_builder_junos_184_r3]
JUNOS Services Flowd MS-MPC Software package [20191221.040256_builder_junos_184_r3]
JUNOS Services Crypto [20191221.040256_builder_junos_184_r3]
JUNOS Services Captive Portal and Content Delivery Container package [20191221.040256_builder_junos_184_r3]
JUNOS Services COS [20191221.040256_builder_junos_184_r3]
JUNOS AppId Services [20191221.040256_builder_junos_184_r3]
JUNOS Services Application Level Gateways [20191221.040256_builder_junos_184_r3]
JUNOS Services AACL Container package [20191221.040256_builder_junos_184_r3]
JUNOS Extension Toolkit [20191221.040256_builder_junos_184_r3]
JUNOS J-Insight [20191221.040256_builder_junos_184_r3]
JUNOS Online Documentation [20191221.040256_builder_junos_184_r3]
JUNOS jail runtime [20191211.fa5e90e_builder_stable_11]
netops@vMX-102>
Hi Tnaganawa
(1)create ipam:ipam-03 10.10.30.0/24 (2)ccreate virtual-network:net-3 with ipam-03 (3)extend net-03 to MX (4)check added configuration on MX
set groups contrail firewall family inet filter _contrail_redirect-to-public-vrfs-inet4 term term-_contrail_net03-l3-13 from destination-address 0.0.0.0/0 set groups contrail firewall family inet filter _contrail_redirect-to-public-vrfs-inet4 term term-_contrail_net03-l3-13 then routing-instance _contrail_net03-l3-13 set groups contrail firewall family inet filter _contrail_redirect-to-public-vrfs-inet4 term default-term then accept
This filter would block traffic between mx and TF-controller so the BGP neighbor turned down. This same issue exist on R1912 and R2003. According to my understanding, “0.0.0.0/0” should be “10.10.30.0/24”。
Any suggestion?