search

aligungr / UERANSIM

Open source 5G UE and RAN (gNodeB) implementation.
GNU General Public License v3.0
760 stars 315 forks source link

Ping timeout for google.com #226

Open Milton0630 opened 3 years ago

Milton0630 commented 3 years ago

Hi,

I installed Open5GS v2.1.1 and UERANSIM v2.2.1 on the same VM. I successfully established PDU connection, but getting [ERRO] [UEAPP] [ue-901700000000003] Ping timeout for google.com (172.217.24.14) after 3 sec no response. Here is my log:

UERANSIM v2.2.1 Selected profile: "open5gs" 2021-01-06 13:21:28.387 [INFO] [SYS] UERANSIM agent has been started. 2021-01-06 13:23:01.520 [INFO] [CONN] [gnb-1] Trying to establish SCTP connection... (127.0.0.5:38412) 2021-01-06 13:23:01.540 [INFO] [CONN] [gnb-1] SCTP connection established 2021-01-06 13:23:02.062 [SUCC] [PROC] [gnb-1] NGSetup procedure is successful 2021-01-06 13:23:30.299 [INFO] [STATE] [ue-901700000000003] UE switches to state: MM_DEREGISTERED/MM_DEREGISTEREDPLMN_SEARCH 2021-01-06 13:23:30.513 [INFO] [CONN] [ue-901700000000003] UE connected to gNB. 2021-01-06 13:23:30.521 [INFO] [STATE] [ue-901700000000003] UE switches to state: MM_DEREGISTERED/MM_DEREGISTERED__NORMAL_SERVICE 2021-01-06 13:23:30.611 [INFO] [STATE] [ue-901700000000003] UE switches to state: MM_REGISTERED_INITIATED/MM_REGISTERED_INITIATEDNA 2021-01-06 13:23:32.312 [INFO] [STATE] [ue-901700000000003] UE switches to state: RM_REGISTERED 2021-01-06 13:23:32.314 [INFO] [STATE] [ue-901700000000003] UE switches to state: MM_REGISTERED/MM_REGISTERED__NORMAL_SERVICE 2021-01-06 13:23:32.314 [SUCC] [PROC] [ue-901700000000003] Registration is successful 2021-01-06 13:23:47.597 [INFO] [TIMER] [ue-901700000000003] NAS Timer expired: NasTimer{timerCode=3510, interval=15} 2021-01-06 13:24:22.026 [INFO] [UEAPP] [ue-901700000000003] IPv4 connection setup with local IP: 10.45.0.2 2021-01-06 13:24:22.041 [INFO] [FLOW] [ue-901700000000003] PDU session established: PDU session identity value 1 2021-01-06 13:24:22.109 [SUCC] [PROC] [gnb-1] PDU Session Establishment is successful 2021-01-06 13:24:22.111 [SUCC] [PROC] [ue-901700000000003] PDU Session Establishment is successful 2021-01-06 13:25:51.130 [ERRO] [UEAPP] [ue-901700000000003] Ping timeout for google.com (172.217.24.14) after 3 sec no response

Here is upf log: Open5GS daemon v2.1.1-16-gc9cfc4c

01/06 13:18:42.852: [app] INFO: Configuration: '/home/open5gs/open5gs/install/etc/open5gs/upf.yaml' (../lib/app/ogs-init.c:129) 01/06 13:18:42.853: [app] INFO: File Logging: '/home/open5gs/open5gs/install/var/log/open5gs/upf.log' (../lib/app/ogs-init.c:132) 01/06 13:18:42.859: [app] INFO: UPF initialize...done (../src/upf/app.c:31) 01/06 13:18:42.859: [pfcp] INFO: pfcp_server() [127.0.0.7]:8805 (../lib/pfcp/path.c:32) 01/06 13:18:42.859: [gtp] INFO: gtp_server() [127.0.0.7]:2152 (../lib/gtp/path.c:32) 01/06 13:18:46.645: [pfcp] INFO: ogs_pfcp_connect() [127.0.0.4]:8805 (../lib/pfcp/path.c:60) 01/06 13:18:46.647: [upf] INFO: PFCP associated (../src/upf/pfcp-sm.c:173) 01/06 13:24:21.900: [upf] INFO: [Added] Number of UPF-Sessions is now 1 (../src/upf/context.c:446) 01/06 13:24:21.900: [upf] INFO: UE F-SEID[CP:0x1,UP:0x1] APN[internet] PDN-Type[1] IPv4[10.45.0.2] IPv6[] (../src/upf/context.c:608) 01/06 13:24:22.029: [gtp] INFO: gtp_connect() [127.0.0.1]:2152 (../lib/gtp/path.c:59)

These are the iptable rules. enp0s3 is the internet interface. sudo iptables -S -t nat

-P PREROUTING ACCEPT -P INPUT ACCEPT -P OUTPUT ACCEPT -P POSTROUTING ACCEPT -A POSTROUTING -o enp0s3 -j MASQUERADE

sudo iptables -S

-P INPUT ACCEPT -P FORWARD ACCEPT -P OUTPUT ACCEPT -A INPUT -s 10.45.0.0/16 -j ACCEPT -A FORWARD -i ogstun -o enp0s3 -j ACCEPT -A FORWARD -i enp0s3 -o ogstun -j ACCEPT

sudo sysctl net.ipv4.ip_forward net.ipv4.ip_forward = 1

Thanks.

aligungr commented 3 years ago

@Milton0630 Please see this issue: https://github.com/aligungr/UERANSIM/issues/191

Milton0630 commented 3 years ago

@aligungr Thanks for your reply.

But I followed issue, I still have same problem.

There is my pcap file. open5gs_UERANSIM.zip

Sorry to trouble you. Thanks

aligungr commented 3 years ago

According to PCAP file we can say that there is a NAT related problem.

Actually ping scenario is well tested between UERANSIM and Open5GS many times by many people. Therefore if you encounter this kind of problems then almost always there is a configuration/setup related problem.

My suggestions are (in order):

1) Use UERANSIM and Open5GS in two different fresh virtual machines. 2) Double check configurations for both UERANSIM and Open5GS 3) Run sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward" to enable IP forwarding (on Open5G's machine) 4) Run this command: https://github.com/aligungr/UERANSIM/issues/191#issuecomment-733550823 (on Open5G's machine)

Let me know if this solves your problem, otherwise we can investigate the problem again. But most probably these 4 suggestion will fix your issue (especially 1st item).

Milton0630 commented 3 years ago

@aligungr Thanks for your help.

My suggestions are (in order):

  1. Use UERANSIM and Open5GS in two different fresh virtual machines.
  2. Double check configurations for both UERANSIM and Open5GS
  3. Run sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward" to enable IP forwarding (on Open5G's machine)
  4. Run this command: #191 (comment) (on Open5G's machine)

After following your suggestion1, I used UERANSIM and Open5GS in two different VMs and Ping is normal. Thank you very much.

By the way, if I use UERANSIM and Open5GS in same VM, how do I set NAT rules?

Thanks again.

mohuchi commented 3 years ago

@Milton0630 Can you please try the following command ?

sysctl -w net.ipv4.conf.all.route_localnet=1

Milton0630 commented 3 years ago

@mohuchi Thanks for your reply.

I ran the commond you said in virtual machine where open5gs and UERANSIM are deployed, but the ping timeout still occurred.

mohuchi commented 3 years ago

@Milton0630 I faced the similar issue with free5GC(v3.0.4) + UERANSIM v2.2.1. I reverted UERANSIM v2.2.1 to v2.0.2 and the issue was resolved.

Milton0630 commented 3 years ago

@mohuchi I also reverted UERANSIM v2.2.1 to v2.0.2 and ping was normal.

aligungr commented 3 years ago

@mohuchi

There was a bug on v2.0.1. But from v2.0.2 to the latest version, it should work with free5gc. Here's an example screenshot from my setup: (UERANSIM v2.2.1, free5gc v3.0.4)

image

And here's the gnb.yaml file:

# MCC and MNC values of the gNB
plmn:
  mcc: 208
  mnc: 93

# ID of the gNB inside of its PLMN
gnbId: 1

# Tracking Area Code (TAC)
tac: 1

# NR Cell Identity (NCI)
nci: '0000000100'

# gNB's local IP address for N2 interface.
ngapIp: 192.168.1.37

# gNB's local IP address for N3 interface.
gtpIp: 192.168.1.37

# List of AMF address information
amfConfigs:
  - ngapIp: 192.168.1.48
    ngapPort: 38412

# List of NSSAIs supported by this gNB
nssais:
  - sst: '0x01'
    sd: '0x010203'

# Indicates whether or not SCTP stream number errors should be ignored.
ignoreStreamIds: true
mohuchi commented 3 years ago

@aligungr

Thank you for the comment. I ran both free5GC and UERANSIM on the same VM. The environment is similar to @Milton0630 's one. The issue which ping failed occrued on my environment.

raoufkh commented 3 years ago

Hello!

I'm running UERANSIM v2.2.1 and free5gc v3.0.4 in different containers (one container for each free5gc NF and one container for UERANSIM). When running with one upf for free5gc PDU session is successfully created and ping works fine. But I have the following in UERANSIM logs: 2021-01-07 21:03:35.379 [INFO] [TIMER] [ue-208930000000004] NAS Timer expired: NasTimer{timerCode=3510, interval=15} Should I neglect this error? And in the AMF log I have this error when creating a UE but it creates it successfuly: 2021-01-07T20:03:16Z [ERRO][AMF][NGAP] [Build Error] DecodeString imeisv error: encoding/hex: odd length hex string (/go/src/free5gc/src/amf/ngap/message/build.go:1144 free5gc/src/amf/ngap/message.BuildInitialContextSetupRequest) Is this because the IMSI format in UERANSIM is not compatible with free5gc?

Another question, has anyone tested UERANSIM with free5gc in ULCL mode (One branching and 2 anchors UPFs). When I test this, the UE traffic goes only through the first anchor UPF (upf1)

Regards,

s5uishida commented 3 years ago

Hi @raoufkh

Another question, has anyone tested UERANSIM with free5gc in ULCL mode (One branching and 2 anchors UPFs). When I test this, the UE traffic goes only through the first anchor UPF (upf1)

When I tried ULCL at the end of the year, it seemed that the IMSI string length was probably not handled properly in free5GC. Also, ULCL traffic has gone through an unintended route.

I will check my environment again later.

aligungr commented 3 years ago

@raoufk

For your other question;

"NAS timer expired" message is related to internals of the UE, and it is not an error. You can neglect that message.

It basicly says that UE needs to generate a fresh SUPI/SUCI in case of an identity request, since the previous one is evicted. (More on TS 24.501)

mohuchi commented 3 years ago

@aligungr 

I ran both free5GC and UERANSIM on the same VM. The environment is similar to @Milton0630 's one.
The issue which ping failed occrued on my environment.

I found out the workaroud. In order to avoid the issue with v2.2.1, I have to use the VRF feature in Linux kernel. See) https://www.kernel.org/doc/Documentation/networking/vrf.txt

Here is the steps to use VRF feature.

[Prerequisite] 1) Installing and loadading VRF featutre. I assume that you use free5gc.

   apt install linux-modules-extra-5.0.0-23-generic
   modprobe vrf

2) Starting with './nr-agent --no-route-config'. 3) Making the VRF table for ueransim

  ip link add dev vrf_uesim type vrf table 1000
  ip link set dev vrf_uesim up

[Steps] 1) Establishing a PDU connection via './nr-cli' commands. 2) Binding uesimtun0 interfae to the VRF table:vrf_uesim and adding default route.

 ip link set dev uesimtun0 master vrf_uesim up
 ip route add default dev uesimtun0 table 1000

3) Checking the routing tables related to the VRF table.

root@free5gc-ohuchi:~/UERANSIM# ip route list vrf vrf_uesim <<< VRF table default dev uesimtun0 scope link

root@free5gc-ohuchi:~/UERANSIM# ip route <<< 'main' routitng talbe default via 100.207.2.4 dev ens192 proto dhcp metric 101 100.207.2.0/24 dev ens192 proto kernel scope link src 100.207.2.244 metric 101 100.64.4.0/24 dev upfgtp proto static 169.254.0.0/16 dev ens192 scope link metric 1000

root@free5gc-ohuchi:~/UERANSIM# ip rule <<< NO policy-based routing configured 0: from all lookup local 1000: from all lookup [l3mdev-table] <<< VRF table 32766: from all lookup main 32767: from all lookup default

4) Using uesimtun0 interface through VRF table. Ping succeeded.

root@free5gc-ohuchi:~/UERANSIM# ping -I uesimtun0 8.8.8.8 PING 8.8.8.8 (8.8.8.8) from 100.64.4.1 uesimtun0: 56(84) bytes of data. 64 bytes from 8.8.8.8: icmp_seq=1 ttl=112 time=3.57 ms 64 bytes from 8.8.8.8: icmp_seq=2 ttl=112 time=3.30 ms 64 bytes from 8.8.8.8: icmp_seq=3 ttl=112 time=3.18 ms 64 bytes from 8.8.8.8: icmp_seq=4 ttl=112 time=2.95 ms 64 bytes from 8.8.8.8: icmp_seq=5 ttl=112 time=3.35 ms ^C --- 8.8.8.8 ping statistics --- 5 packets transmitted, 5 received, 0% packet loss, time 4004ms rtt min/avg/max/mdev = 2.950/3.273/3.576/0.218 ms

aligungr commented 3 years ago

@mohuchi

Thanks a lot for the detailed description.

We developed the --no-route-config feature for exactly this purpose. Some users may need a different routing configurations. And default one may be unusable for some cases.

However routing configuration is an optional operation, you can always use --no-route-config if you manually bind your connection to uesimtunX interface. Therefore I wonder something in this case. I think --no-route-config option should be sufficient if you use ping -I uesimtun0 8.8.8.8, and it should work without VRF settings. (Because ping -I command directly binds to the specified interface, without using routing tables.)

Does --no-route-config work for you in your environment without using VRF? I'd be glad if you inform me about this.

raoufkh commented 3 years ago

Hello @aligungr

Thank you for this explanation. This reassures me.

raoufkh commented 3 years ago

Hello @s5uishida

It works fine now for me. I'm using UERANSIM v2.2.1 and free5gc v3.0.4

I have another question as you've already tried to test the ULCL feature. In 3GPP specs, they say that the interface name between the Branching UPF and Anchors is named N9 interface. Should it be different from the N3 interface? And is there any way to specify it in the free5gc configuration?

mohuchi commented 3 years ago

@aligungr

Does --no-route-config work for you in your environment without using VRF? I'd be glad if you inform me about this.

No. That did not work. Here is the test log without VRF.

root@free5gc-ohuchi:~/UERANSIM# ./nr-agent --no-route-config
UERANSIM v2.2.1
Selected profile: "free5gc"
2021-01-13 21:49:34.653 [INFO] [SYS] UERANSIM agent has been started.
2021-01-13 21:50:05.196 [INFO] [CONN] [gnb-1] Trying to establish SCTP connection... (127.0.0.1:38412)
2021-01-13 21:50:05.227 [INFO] [CONN] [gnb-1] SCTP connection established
2021-01-13 21:50:06.662 [SUCC] [PROC] [gnb-1] NGSetup procedure is successful
2021-01-13 21:50:10.862 [INFO] [STATE] [ue-208930000000003] UE switches to state: MM_DEREGISTERED/MM_DEREGISTERED__PLMN_SEARCH
2021-01-13 21:50:11.065 [INFO] [CONN] [ue-208930000000003] UE connected to gNB.
2021-01-13 21:50:11.066 [INFO] [STATE] [ue-208930000000003] UE switches to state: MM_DEREGISTERED/MM_DEREGISTERED__NORMAL_SERVICE
2021-01-13 21:50:11.165 [INFO] [STATE] [ue-208930000000003] UE switches to state: MM_REGISTERED_INITIATED/MM_REGISTERED_INITIATED__NA
2021-01-13 21:50:13.595 [INFO] [STATE] [ue-208930000000003] UE switches to state: RM_REGISTERED
2021-01-13 21:50:13.597 [INFO] [STATE] [ue-208930000000003] UE switches to state: MM_REGISTERED/MM_REGISTERED__NORMAL_SERVICE
2021-01-13 21:50:13.599 [SUCC] [PROC] [ue-208930000000003] Registration is successful
2021-01-13 21:50:25.146 [WARN] [FLOW] [ue-208930000000003] SM cause received in PduSessionEstablishmentAccept: PDU session type IPv4 only allowed
2021-01-13 21:50:25.149 [INFO] [UEAPP] [ue-208930000000003] IPv4 connection setup with local IP: 100.64.4.1
2021-01-13 21:50:25.161 [INFO] [FLOW] [ue-208930000000003] PDU session established: PDU session identity value 1
2021-01-13 21:50:25.171 [SUCC] [PROC] [ue-208930000000003] PDU Session Establishment is successful
2021-01-13 21:50:25.250 [SUCC] [PROC] [gnb-1] PDU Session Establishment is successful
2021-01-13 21:50:27.247 [INFO] [TIMER] [ue-208930000000003] NAS Timer expired: NasTimer{timerCode=3510, interval=15}
2021-01-13 21:50:50.798 [ERRO] [UEAPP] [ue-208930000000003] Ping timeout for 8.8.8.8 after 3 sec no response
MSA-lab commented 3 years ago

Hello!

I can't run UE and Core on different server because nr-agent can't connect throw SCTP with AMF 

root@free5gc:~/UERANSIM-2.2.3$ sudo ./nr-agent
UERANSIM v2.2.3
Selected profile: "free5gc"
2021-01-28 08:40:50.064 [INFO] [SYS] UERANSIM agent has been started.
2021-01-28 08:40:59.507 [INFO] [CONN] [gnb-1] Trying to establish SCTP connection... (172.18.104.101:38412)
2021-01-28 08:46:41.548 [ERRO] [CONN] [gnb-1] SCTP connection could not established: Connection timed out

but if I try to connect via withsctp telnet on the same server it works fine and even amf can get some message

root@free5gc:~/UERANSIM-2.2.3$ withsctp telnet 172.18.104.101 38412
Trying 172.18.104.101...
Connected to 172.18.104.101.
Escape character is '^]'.

AMF log

2021-01-28T13:11:10+03:00 [DEBU][AMF][NGAP] Get default sent param[value: &{Stream:0 SSN:0 Flags:0 _:0 PPID:0 Context:0 TTL:0 TSN:0 CumTSN:0 AssocID:0}]
2021-01-28T13:11:10+03:00 [DEBU][AMF][NGAP] Set default sent param[value: &{Stream:0 SSN:0 Flags:0 _:0 PPID:1006632960 Context:0 TTL:0 TSN:0 CumTSN:0 AssocID:0}]
2021-01-28T13:11:10+03:00 [DEBU][AMF][NGAP] Subscribe SCTP event[DATA_IO, SHUTDOWN_EVENT, ASSOCIATION_CHANGE]
2021-01-28T13:11:10+03:00 [DEBU][AMF][NGAP] Set read buffer to 8192 bytes
2021-01-28T13:11:10+03:00 [DEBU][AMF][NGAP] Set read timeout: {Sec:2 Usec:0}
2021-01-28T13:11:10+03:00 [INFO][AMF][NGAP] [AMF] SCTP Accept from: 172.18.104.20:36100

Anybody have some idea about this issue?

aligungr commented 3 years ago

@MSA-lab Make sure that you set ngapIp in gnb config file. Can you share your configuration file.

MSA-lab commented 3 years ago 
# MCC and MNC values of the gNB
plmn:
  mcc: 208
  mnc: 93

# ID of the gNB inside of its PLMN
gnbId: 1

# Tracking Area Code (TAC)
tac: 1

# NR Cell Identity (NCI)
nci: '0000000100'

# gNB's local IP address for N2 interface.
ngapIp: 127.0.0.1

# gNB's local IP address for N3 interface.
gtpIp: 127.0.0.1

# List of AMF address information
amfConfigs:
  - ngapIp: 172.18.104.101
    ngapPort: 38412

# List of NSSAIs supported by this gNB
nssais:
  - sst: '0x01'
    sd: '0x010203'

# Indicates whether or not SCTP stream number errors should be ignored.
ignoreStreamIds: true
aligungr commented 3 years ago

@MSA-lab

You need to set ngapIp and gtpIp to your local IP, instead of loopback address (127.0.0.1)

For example in my computer I use 192.168.1.37 as local IP.

AMF will try to send the response to the your IP address specified the ngapIp field. That'ts the exact cause of the case.

I think that configuration is a little bit confusing, sorry.

Please let me know if that solves your problem

raoufkh commented 3 years ago

@MSA-lab

Can you show the lish of interfaces on the host in wich you are installing UERANSIM with their ip addresses.

gNB's local IP address for N2 interface.

ngapIp: 127.0.0.1 ==> has to match the AMF ip address (NGAP)

gNB's local IP address for N3 interface.

gtpIp: 127.0.0.1 ==> has to match the UPF ip address (GTP )

for example in your case, ngapIp should be something like 172.18.104.x (i.e. the address of the interface which is in the same subnet as AMF NGAP interface)

You can also set it to 0.0.0.0 to use all host's interfaces.

Hope that will help you!

MSA-lab commented 3 years ago

thx for answers guys, replacing the default address with the real ip of my server, solve the problem

raoufkh commented 3 years ago

Great