search

devopsjourney23 / free5gcdemo

This repo provide my demonstration of free5gc project.
7 stars 3 forks source link

UPF & SMF are in stuck in CrashLoopBackOff (suspecting the Network configuration) #2

Open sam-sre opened 5 months ago

sam-sre commented 5 months ago

Hi,

I have what I think is a networking problem, but I'm not sure where the problem is exactly..

kg po
NAME                                                 READY   STATUS             RESTARTS        AGE
free5gc-free5gc-amf-amf-68854479ff-dn6xg             1/1     Running            0               13m
free5gc-free5gc-ausf-ausf-76b64786db-2cjg5           1/1     Running            0               13m
free5gc-free5gc-dbpython-dbpython-796555989d-lxcn5   1/1     Running            0               13m
free5gc-free5gc-nrf-nrf-57bf977957-g2rwq             1/1     Running            0               13m
free5gc-free5gc-nssf-nssf-7c5d7fd7c7-fnlhg           1/1     Running            0               13m
free5gc-free5gc-pcf-pcf-5d4fff6875-6g2p6             1/1     Running            0               13m
free5gc-free5gc-smf-smf-5f8dd77d68-kknhg             0/1     CrashLoopBackOff   7 (116s ago)    13m
free5gc-free5gc-udm-udm-ccb6c7d77-mz9nr              1/1     Running            0               13m
free5gc-free5gc-udr-udr-58d7dbc844-mxcg2             1/1     Running            0               13m
free5gc-free5gc-upf-upf-67949d95fd-wq7cl             0/1     CrashLoopBackOff   7 (2m37s ago)   13m
free5gc-free5gc-webui-webui-849964856f-prmnf         1/1     Running            0               13m
mongodb-0                                            1/1     Running            0               13m

Logs from the UPF Pod:

2024-02-02T21:45:34.634298864Z [INFO][UPF][CFG] ==================================================
2024-02-02T21:45:34.634317896Z [INFO][UPF][Main] Log level is set to [info]
2024-02-02T21:45:34.634326421Z [INFO][UPF][Main] Report Caller is set to [false]
2024-02-02T21:45:34.634381451Z [INFO][UPF][Main] starting Gtpu Forwarder [gtp5g]
2024-02-02T21:45:34.634389109Z [INFO][UPF][Main] GTP Address: "10.100.50.233:2152"
2024-02-02T21:45:34.634904041Z [ERRO][UPF][Main] UPF Cli Run Error: open Gtp5g: open link: listen: listen udp4 10.100.50.233:2152: bind: cannot assign requested address

Here are some extra info from my setup:

Master node interfaces ( Vagrant Box)

vagrant@kmaster:~$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 52:54:00:2f:d0:5c brd ff:ff:ff:ff:ff:ff
    inet 192.168.121.78/24 brd 192.168.121.255 scope global dynamic eth0
       valid_lft 2669sec preferred_lft 2669sec
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 52:54:00:d9:2d:40 brd ff:ff:ff:ff:ff:ff
    inet 172.16.16.100/24 brd 172.16.16.255 scope global eth1
       valid_lft forever preferred_lft forever
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 52:54:00:c0:5a:f3 brd ff:ff:ff:ff:ff:ff
    inet 10.10.0.10/24 brd 10.10.0.255 scope global eth2
       valid_lft forever preferred_lft forever

Master node routing info

vagrant@kmaster:~$ ip r
default via 192.168.121.1 dev eth0 proto dhcp src 192.168.121.78 metric 100 
10.0.0.0/24 via 10.0.1.176 dev cilium_host proto kernel src 10.0.1.176 mtu 1450 
10.0.1.0/24 via 10.0.1.176 dev cilium_host proto kernel src 10.0.1.176 
10.0.1.176 dev cilium_host proto kernel scope link 
10.10.0.0/24 dev eth2 proto kernel scope link src 10.10.0.10 
172.16.16.0/24 dev eth1 proto kernel scope link src 172.16.16.100 
192.168.121.0/24 dev eth0 proto kernel scope link src 192.168.121.78 
192.168.121.1 dev eth0 proto dhcp scope link src 192.168.121.78 metric 100

Worker node interfaces ( Vagrant Box)

vagrant@kworker1:~$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 52:54:00:48:e6:09 brd ff:ff:ff:ff:ff:ff
    inet 192.168.121.10/24 brd 192.168.121.255 scope global dynamic eth0
       valid_lft 3367sec preferred_lft 3367sec
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 52:54:00:06:53:88 brd ff:ff:ff:ff:ff:ff
    inet 172.16.16.101/24 brd 172.16.16.255 scope global eth1
       valid_lft forever preferred_lft forever
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 52:54:00:0f:5f:02 brd ff:ff:ff:ff:ff:ff
    inet 10.10.0.11/24 brd 10.10.0.255 scope global eth2
       valid_lft forever preferred_lft forever

Worker node routing info

vagrant@kworker1:~$ ip r
default via 192.168.121.1 dev eth0 proto dhcp src 192.168.121.10 metric 100 
10.0.0.0/24 via 10.0.0.154 dev cilium_host proto kernel src 10.0.0.154 
10.0.0.154 dev cilium_host proto kernel scope link 
10.0.1.0/24 via 10.0.0.154 dev cilium_host proto kernel src 10.0.0.154 mtu 1450 
10.10.0.0/24 dev eth2 proto kernel scope link src 10.10.0.11 
172.16.16.0/24 dev eth1 proto kernel scope link src 172.16.16.101 
192.168.121.0/24 dev eth0 proto kernel scope link src 192.168.121.10 
192.168.121.1 dev eth0 proto dhcp scope link src 192.168.121.10 metric 100

Editing the UPF n6if IP address in values.yml here charts/free5gc/charts/free5gc-upf/values.yaml

  # network paramters
  n3if:  # GTP-U
    ipAddress: 10.100.50.233
  n4if:  # PFCP
    ipAddress: 10.100.50.241
  n6if:  # DN
    ipAddress: 172.16.16.20

Editing networking info in values.yml here charts/free5gc/values.yaml 

  #Global network parametes
  n2network:
    enabled: true
    name: n2network
    type: ipvlan
    masterIf: eth1
    subnetIP: 10.100.50.248
    cidr: 29
    gatewayIP: 10.100.50.254
    excludeIP: 10.100.50.254
  n3network:
    enabled: true
    name: n3network
    type: ipvlan
    masterIf: eth1
    subnetIP: 10.100.50.232
    cidr: 29
    gatewayIP: 10.100.50.238
    excludeIP: 10.100.50.238
  n4network:
    enabled: true
    name: n4network
    type: ipvlan
    masterIf: eth1
    subnetIP: 10.100.50.240
    cidr: 29
    gatewayIP: 10.100.50.246
    excludeIP: 10.100.50.246
  n6network:
    enabled: true
    name: n6network
    type: ipvlan
    masterIf: eth1
    subnetIP: 172.16.16.0
    cidr: 24
    gatewayIP: 172.16.16.1
    excludeIP: 172.16.16.254
  n9network:
    enabled: true
    name: n9network
    type: ipvlan
    masterIf: eth1
    subnetIP: 10.100.50.224
    cidr: 29
    gatewayIP: 10.100.50.230
    excludeIP: 10.100.50.230

The Virtual Bridge on the Host machine where both vagrant boxes are running. )I'm using this IP 172.16.16.1/24 as a GW IP for the n6network

10: virbr2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 52:54:00:2f:95:80 brd ff:ff:ff:ff:ff:ff
    inet 172.16.16.1/24 brd 172.16.16.255 scope global virbr2
       valid_lft forever preferred_lft forever

Thanks for help..

BR Sami

devopsjourney23 commented 4 weeks ago

Were you able to successfully compile gtp5g module on your linux kernal ?