search

srsran / srsRAN_Project

Open source O-RAN 5G CU/DU solution from Software Radio Systems (SRS) https://docs.srsran.com/projects/project
https://www.srsran.com
GNU Affero General Public License v3.0
466 stars 159 forks source link

ZMQ ue-gnb connection validation #593

Closed hududed closed 4 months ago

hududed commented 4 months ago

I successfully connected srsue to srsgnb via zmq

Attaching UE...
Random Access Transmission: prach_occasion=0, preamble_index=0, ra-rnti=0x39, tti=814
Random Access Complete.     c-rnti=0x4601, ta=0
RRC Connected
PDU Session Establishment successful. IP: 10.45.1.2
RRC NR reconfiguration successful.

For validation, I ping but see nothing transmitted

❯ sudo ip netns exec ue1 ping 10.45.1.1 -c4
PING 10.45.1.1 (10.45.1.1) 56(84) bytes of data.
^C
--- 10.45.1.1 ping statistics ---
4 packets transmitted, 0 received, 100% packet loss, time 3050ms

However, there was gnb traffic:

          |--------------------DL---------------------|-------------------UL------------------------------
 pci rnti | cqi  ri  mcs  brate   ok  nok  (%)  dl_bs | pusch  mcs  brate   ok  nok  (%)    bsr    ta  phr
   1 4601 |  15   1   26    608    1    0   0%      0 |  65.5   28   7.2k    3    0   0%      0   0us  n/a
   1 4601 |  15   1   26    608    1    0   0%      0 |  65.5   28   5.8k    2    0   0%      0   0us  n/a
   1 4601 |  15   1   26    608    1    0   0%      0 |  65.5   28   7.2k    3    0   0%      0   0us  n/a
   1 4601 |  15   1   26    608    1    0   0%      0 |  65.5   28   7.2k    3    0   0%      0   0us  n/a
   1 4601 |  15   1    0      0    0    0   0%      0 |   n/a    0      0    0    0   0%      0   0us  n/a
   1 4601 |  15   1    0      0    0    0   0%      0 |   n/a    0      0    0    0   0%      0   0us  n/a
   1 4601 |  15   1    0      0    0    0   0%      0 |   n/a    0      0    0    0   0%      0   0us  n/a
   1 4601 |  15   1    0      0    0    0   0%      0 |   n/a    0      0    0    0   0%      0   0us  n/a
   1 4601 |  15   1    0      0    0    0   0%      0 |   n/a    0      0    0    0   0%      0   0us  n/a
   1 4601 |  15   1    0      0    0    0   0%      0 |   n/a    0      0    0    0   0%      0   0us  n/a
   1 4601 |  15   1    0      0    0    0   0%      0 |   n/a    0      0    0    0   0%      0   0us  n/a

and ue traffic:

---------Signal-----------|-----------------DL-----------------|-----------UL-----------
rat  pci  rsrp   pl   cfo | mcs  snr  iter  brate  bler  ta_us | mcs   buff  brate  bler
 nr    1    44    0 -393n |   0  118   0.0    0.0    0%    0.0 |   0    0.0    0.0    0%
 nr    1    44    0 -1.5u |   0   67   0.0    0.0    0%    0.0 |   0    0.0    0.0    0%
 nr    1    44    0  7.5u |   0  n/a   0.0    0.0    0%    0.0 |   0    0.0    0.0    0%
 nr    1    44    0  3.2u |  26  n/a   1.0    645    0%    0.0 |  27    0.0   7.7k    0%
 nr    1    44    0  4.4u |  26   70   1.0    639    0%    0.0 |  27    0.0   6.1k    0%
 nr    1    44    0  7.2u |  26  118   1.0    639    0%    0.0 |  27    0.0   7.6k    0%
 nr    1    44    0  6.7u |  26  n/a   1.0    639    0%    0.0 |  27    0.0   7.6k    0%
 nr    1    44    0  2.1u |   0   67   0.0    0.0    0%    0.0 |   0    0.0    0.0    0%
 nr    1    44    0  1.3u |   0  n/a   0.0    0.0    0%    0.0 |   0    0.0    0.0    0%
 nr    1    44    0  4.4u |   0   70   0.0    0.0    0%    0.0 |   0    0.0    0.0    0%
 nr    1    44    0  1.2u |   0  118   0.0    0.0    0%    0.0 |   0    0.0    0.0    0%

Is this expected?

pgawlowicz commented 4 months ago

ping should work. Could you check UE routing table?

sudo ip netns exec ue1 route -n

Do you have another instance of open5gs (installed manually) running on the PC?

hududed commented 4 months ago 
❯ route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.0.0.1        0.0.0.0         UG    20600  0        0 wlp69s0
10.0.0.0        0.0.0.0         255.255.255.0   U     600    0        0 wlp69s0
10.7.0.0        0.0.0.0         255.255.248.0   U     0      0        0 nordtun
10.45.0.0       10.53.1.2       255.255.255.0   UG    0      0        0 nordtun
10.53.1.0       0.0.0.0         255.255.255.0   U     0      0        0 nordtun
169.254.0.0     0.0.0.0         255.255.0.0     U     1000   0        0 wlp69s0
❯ sudo ip netns exec ue1 ip route add default via 10.45.1.1 dev tun_srsue
❯ sudo ip netns exec ue1 route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.45.1.1       0.0.0.0         UG    0      0        0 tun_srsue
10.45.1.0       0.0.0.0         255.255.255.0   U     0      0        0 tun_srsue

I am pretty sure I installed and am running just one instance of it. Is there a way to double check?

pgawlowicz commented 4 months ago

what is this route?

10.7.0.0        0.0.0.0         255.255.248.0   U     0      0        0 nordtun
hududed commented 4 months ago

Seems to be specific to my VPN. Unfortunately, I couldn't find a way to get this running without nordtun interface. Not sure how to properly set up ogstun interface

pgawlowicz commented 4 months ago

then your traffic goes to the VPN interface. Could you disable the VPN and test? Did you use the dockerized Open5gs versoin? see here: https://docs.srsran.com/projects/project/en/latest/tutorials/source/srsUE/source/index.html#open5gs-core

hududed commented 4 months ago

Ok so I was able to run it without the VPN but ping still doesnt work (but with UE gnb traffic)

❯ sudo ip netns exec ue1 route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.45.1.1       0.0.0.0         UG    0      0        0 tun_srsue
10.45.1.0       0.0.0.0         255.255.255.0   U     0      0        0 tun_srsue
❯ sudo ip netns exec ue1 ping 10.45.1.1 -c4
PING 10.45.1.1 (10.45.1.1) 56(84) bytes of data.

--- 10.45.1.1 ping statistics ---
4 packets transmitted, 0 received, 100% packet loss, time 3074ms

❯ route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.0.0.1        0.0.0.0         UG    600    0        0 wlp69s0
10.0.0.0        0.0.0.0         255.255.255.0   U     600    0        0 wlp69s0
10.45.0.0       10.53.1.2       255.255.255.0   UG    0      0        0 ogstun
10.45.1.1       10.0.0.1        255.255.255.255 UGH   0      0        0 wlp69s0
10.53.1.0       0.0.0.0         255.255.255.0   U     0      0        0 ogstun
169.254.0.0     0.0.0.0         255.255.0.0     U     1000   0        0 wlp69s0

I did try the open5gs docker first and we did go at length not being able to reproduce it (see here) - this is why I tried to run it manually instead. I did triple-checked and updated all the default open5gs configs to srsRAN configs open5gs-5gc.yaml

pgawlowicz commented 4 months ago

Why do you have those two routes?

10.0.0.0        0.0.0.0         255.255.255.0   U     600    0        0 wlp69s0
...
10.45.1.1       10.0.0.1        255.255.255.255 UGH   0      0        0 wlp69s0

Can you remove it?

hududed commented 4 months ago

So I restarted so I'll show you what routes are there step by step:

  1. fresh restart 
    Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.0.0.1        0.0.0.0         UG    600    0        0 wlp69s0
10.0.0.0        0.0.0.0         255.255.255.0   U     600    0        0 wlp69s0
169.254.0.0     0.0.0.0         255.255.0.0     U     1000   0        0 wlp69s0
  2. Run all open5gs components: 
    Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.0.0.1        0.0.0.0         UG    600    0        0 wlp69s0
10.0.0.0        0.0.0.0         255.255.255.0   U     600    0        0 wlp69s0
10.45.0.0       0.0.0.0         255.255.0.0     U     0      0        0 ogstun
10.45.0.0       0.0.0.0         255.255.0.0     U     0      0        0 ogstun
10.53.1.0       0.0.0.0         255.255.255.0   U     0      0        0 wlp69s0
169.254.0.0     0.0.0.0         255.255.0.0     U     1000   0        0 wlp69s0

    Not sure why ogstun routes are there, but I assume its from smf and amf:

    ❯ grep "10.45." /etc/open5gs/*
grep: /etc/open5gs/backup: Is a directory
grep: /etc/open5gs/hnet: Is a directory
/etc/open5gs/smf.yaml:    - subnet: 10.45.0.1/24
/etc/open5gs/smf.yaml:      gateway: 10.45.0.1
grep: /etc/open5gs/tls: Is a directory
/etc/open5gs/upf.yaml:    - subnet: 10.45.0.1/24
/etc/open5gs/upf.yaml:      gateway: 10.45.0.1
  3. After gnb and ue ran: 
    
❯ sudo ip ro add 10.45.0.0/24 via 10.53.1.2
❯ route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.0.0.1        0.0.0.0         UG    600    0        0 wlp69s0
10.0.0.0        0.0.0.0         255.255.255.0   U     600    0        0 wlp69s0
10.45.0.0       10.53.1.2       255.255.255.0   UG    0      0        0 wlp69s0
10.45.0.0       0.0.0.0         255.255.0.0     U     0      0        0 ogstun
10.45.0.0       0.0.0.0         255.255.0.0     U     0      0        0 ogstun
10.53.1.0       0.0.0.0         255.255.255.0   U     0      0        0 wlp69s0
169.254.0.0     0.0.0.0         255.255.0.0     U     1000   0        0 wlp69s0
❯ sudo ip netns exec ue1 ip route add default via 10.45.1.1 dev tun_srsue
❯ sudo ip netns exec ue1 route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.45.1.1       0.0.0.0         UG    0      0        0 tun_srsue
10.45.1.0       0.0.0.0         255.255.255.0   U     0      0        0 tun_srsue
❯ sudo ip netns exec ue1 ping 10.45.1.1 -c4
PING 10.45.1.1 (10.45.1.1) 56(84) bytes of data.

--- 10.45.1.1 ping statistics --- 4 packets transmitted, 0 received, 100% packet loss, time 3069ms


4. Delete some routes:

❯ sudo ip route del 10.0.0.0/24 dev wlp69s0 ❯ sudo ip route del 10.45.0.0/16 dev ogstun ❯ route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 10.0.0.1 0.0.0.0 UG 600 0 0 wlp69s0 10.45.0.0 10.53.1.2 255.255.255.0 UG 0 0 0 wlp69s0 10.45.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ogstun 10.53.1.0 0.0.0.0 255.255.255.0 U 0 0 0 wlp69s0 169.254.0.0 0.0.0.0 255.255.0.0 U 1000 0 0 wlp69s0 ❯ sudo ip netns exec ue1 ping 10.45.1.1 -c4 PING 10.45.1.1 (10.45.1.1) 56(84) bytes of data.

--- 10.45.1.1 ping statistics --- 4 packets transmitted, 0 received, 100% packet loss, time 3076ms

pgawlowicz commented 4 months ago

could you try:

sudo ip ro add 10.45.0.0/24 via 10.53.1.2 dev ogstun

instead of 

sudo ip ro add 10.45.0.0/24 via 10.53.1.2
hududed commented 4 months ago

Sorry, wasn't too sure if it mattered with the /16 or /24 set in open5gs and srsran, so I made them all consistent /16:

❯ sudo ip ro add 10.45.0.0/16 via 10.53.1.2 dev ogstun
Error: Nexthop has invalid gateway.
❯ sudo ip ro add 10.45.0.0/24 via 10.53.1.2 dev ogstun
Error: Nexthop has invalid gateway.
pgawlowicz commented 4 months ago

indeed it should be:

sudo ip ro add 10.45.0.0/16 via 10.53.1.2 dev ogstun
pgawlowicz commented 4 months ago

ah wait, do you run the dockerized version of the Open5gs?

hududed commented 4 months ago

No its a local instance - the docker didnt work for me

pgawlowicz commented 4 months ago

ok, then you do not need the sudo ip ro add 10.45.0.0/16 via 10.53.1.2 dev ogstun this route at all.

hududed commented 4 months ago 
❯ sudo ip netns exec ue1 ip route add default via 10.45.1.1 dev tun_srsue
❯ route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.0.0.1        0.0.0.0         UG    600    0        0 wlp69s0
10.0.0.0        0.0.0.0         255.255.255.0   U     600    0        0 wlp69s0
10.45.0.0       0.0.0.0         255.255.0.0     U     0      0        0 ogstun
10.45.0.0       0.0.0.0         255.255.0.0     U     0      0        0 ogstun
10.53.0.0       0.0.0.0         255.255.0.0     U     0      0        0 wlp69s0
169.254.0.0     0.0.0.0         255.255.0.0     U     1000   0        0 wlp69s0
❯ sudo ip netns exec ue1 route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.45.1.1       0.0.0.0         UG    0      0        0 tun_srsue
10.45.1.0       0.0.0.0         255.255.255.0   U     0      0        0 tun_srsue
❯ sudo ip netns exec ue1 ping 10.45.1.1 -c4
PING 10.45.1.1 (10.45.1.1) 56(84) bytes of data.

--- 10.45.1.1 ping statistics ---
4 packets transmitted, 0 received, 100% packet loss, time 3059ms

Got excited there but still the same - could you check if I need to delete some of these routes?

pgawlowicz commented 4 months ago

why there are two same routes?

10.45.0.0       0.0.0.0         255.255.0.0     U     0      0        0 ogstun
10.45.0.0       0.0.0.0         255.255.0.0     U     0      0        0 ogstun
hududed commented 4 months ago

Hm, that was peculiar - so I reran the open5gs component 1 by 1 (in the order given in the open5gs tutorial), and the two ogstun are created when I ran upf:

❯ sudo systemctl start open5gs-sgwud
❯ route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.0.0.1        0.0.0.0         UG    600    0        0 wlp69s0
10.0.0.0        0.0.0.0         255.255.255.0   U     600    0        0 wlp69s0
10.53.0.0       0.0.0.0         255.255.0.0     U     0      0        0 wlp69s0
169.254.0.0     0.0.0.0         255.255.0.0     U     1000   0        0 wlp69s0
❯ sudo systemctl start open5gs-upfd
❯ route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.0.0.1        0.0.0.0         UG    600    0        0 wlp69s0
10.0.0.0        0.0.0.0         255.255.255.0   U     600    0        0 wlp69s0
10.45.0.0       0.0.0.0         255.255.0.0     U     0      0        0 ogstun
10.45.0.0       0.0.0.0         255.255.0.0     U     0      0        0 ogstun
10.53.0.0       0.0.0.0         255.255.0.0     U     0      0        0 wlp69s0
169.254.0.0     0.0.0.0         255.255.0.0     U     1000   0        0 wlp69s0

I am not sure why its creating these. This is the config for upf

❯ sudo vi /etc/open5gs/upf.yaml
logger:
  file:
    path: /var/log/open5gs/all.log

global:
  max:
    ue: 1024 # The number of UE can be increased depending on memory size.

upf:
  pfcp:
    server:
      - address: 127.0.0.7
    client:
      smf: #  UPF PFCP Client try to associate SMF PFCP Server
        - address: 127.0.0.4
  gtpu:
    server:
      - address: 10.53.1.2 #127.0.0.7
        #advertise: 127.0.0.8
  session:
    - subnet: 10.45.0.1/16 #10.45.0.0/16
      gateway: 10.45.0.1
    - subnet: 2001:db8:cafe::/48
      gateway: 2001:db8:cafe::1

  metrics:
    server:
      - address: 127.0.0.7
        port: 9090

Should I delete those?

pgawlowicz commented 4 months ago

why did you set this:

  gtpu:
    server:
      - address: 10.53.1.2 #127.0.0.7
hududed commented 4 months ago

Well I followed everything in srsran config, but I suppose that was meant for docker? Not sure which to include and which not to.

#srsran_Project/docker/open5gs-5gc.yaml

mme:
  s1ap:
    server:
      - address: ${OPEN5GS_IP}
  gtpc:
    server:
      - address: ${OPEN5GS_IP}
      ...
amf:
  ngap:
    server:
      - address: ${OPEN5GS_IP}

upf:
  ...
  gtpu:
    server:
      - address: ${OPEN5GS_IP}
❯ grep "10.53.1." /etc/open5gs/*
/etc/open5gs/amf.yaml:      - address: 10.53.1.2
/etc/open5gs/mme.yaml:      - address: 10.53.1.2
/etc/open5gs/mme.yaml:      - address: 10.53.1.2
/etc/open5gs/upf.yaml:      - address: 10.53.1.2 #127.0.0.7
pgawlowicz commented 4 months ago

yes, this is meant for the docker-based setup. Please just follow this tutorial: https://open5gs.org/open5gs/docs/guide/01-quickstart/

In addition, our Amarisoft UE tutorial uses Open5gs installed manually: https://docs.srsran.com/projects/project/en/latest/tutorials/source/amariUE/source/index.html#id6

hududed commented 4 months ago

I think I solved it. So in contrast to the tutorial, I didnt have to change any IP addresses at all (in AMF, UPF etc) (I guess since its a single host machine). I just updated the rest (TAC PLMN etc) following srsran. 

❯ sudo ip netns exec ue1 ip route add default via 10.45.1.1 dev tun_srsue
❯ sudo ip addr add 10.45.1.1/24 dev ogstun # this was important
❯ sudo ip netns exec ue1 ping 10.45.1.1 -c4
PING 10.45.1.1 (10.45.1.1) 56(84) bytes of data.
64 bytes from 10.45.1.1: icmp_seq=1 ttl=64 time=32.7 ms
64 bytes from 10.45.1.1: icmp_seq=2 ttl=64 time=25.3 ms
64 bytes from 10.45.1.1: icmp_seq=3 ttl=64 time=35.0 ms
64 bytes from 10.45.1.1: icmp_seq=4 ttl=64 time=24.6 ms

--- 10.45.1.1 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3004ms

Thanks for the help @pgawlowicz !