search

Gradiant / 5g-charts

Helm charts for 5G Technologies
Apache License 2.0
117 stars 47 forks source link

I can't make UE from UERANSIM work with open5gs, both openverso-charts #37

Closed DiogoCruz40 closed 2 years ago

DiogoCruz40 commented 2 years ago

Captura de ecrã de 2022-02-04 22-01-22

So i ping for exemple google.com and it's 100% package lost I used your configs and only changed the version to 3.2.4, 3.2.2 didn t work either. I can't find the reason why this is not working with open5gs tag latest

Can you help plz?

DiogoCruz40 commented 2 years ago

I Still didn´t find a solution for this, but i think amf is refusing the connection...

02/08 16:58:27.041: [amf] INFO: gNB-N2 accepted[10.42.4.68]:48966 in ng-path module (../src/amf/ngap-sctp.c:105)

8/2/2022 16:58:27 | 02/08 16:58:27.041: [amf] INFO: gNB-N2 accepted[10.42.4.68] in master_sm module (../src/amf/amf-sm.c:619) 8/2/2022 16:58:27 | 02/08 16:58:27.041: [amf] INFO: [GNB] max_num_of_ostreams : 30 (../src/amf/context.c:854) 8/2/2022 16:58:27 | 02/08 16:58:27.041: [amf] INFO: [Added] Number of gNBs is now 1 (../src/amf/context.c:869) 8/2/2022 16:58:29 | 02/08 16:58:29.592: [amf] INFO: InitialUEMessage (../src/amf/ngap-handler.c:361) 8/2/2022 16:58:29 | 02/08 16:58:29.592: [amf] INFO: [Added] Number of gNB-UEs is now 1 (../src/amf/context.c:2064) 8/2/2022 16:58:29 | 02/08 16:58:29.592: [amf] INFO: RAN_UE_NGAP_ID[1] AMF_UE_NGAP_ID[5] TAC[1] CellID[0x100] (../src/amf/ngap-handler.c:497) 8/2/2022 16:58:29 | 02/08 16:58:29.598: [amf] INFO: [suci-0-999-70-0000-0-0-0000000001] Unknown UE by SUCI (../src/amf/context.c:1390) 8/2/2022 16:58:29 | 02/08 16:58:29.598: [amf] INFO: [Added] Number of AMF-UEs is now 1 (../src/amf/context.c:1189) 8/2/2022 16:58:29 | 02/08 16:58:29.598: [gmm] INFO: Registration request (../src/amf/gmm-sm.c:134) 8/2/2022 16:58:29 | 02/08 16:58:29.598: [gmm] INFO: [suci-0-999-70-0000-0-0-0000000001] SUCI (../src/amf/gmm-handler.c:157) 8/2/2022 16:58:29 | 02/08 16:58:29.749: [amf] INFO: InitialUEMessage (../src/amf/ngap-handler.c:361) 8/2/2022 16:58:29 | 02/08 16:58:29.749: [amf] INFO: [Added] Number of gNB-UEs is now 2 (../src/amf/context.c:2064) 8/2/2022 16:58:29 | 02/08 16:58:29.749: [amf] INFO: RAN_UE_NGAP_ID[2] AMF_UE_NGAP_ID[6] TAC[1] CellID[0x100] (../src/amf/ngap-handler.c:497) 8/2/2022 16:58:29 | 02/08 16:58:29.749: [amf] INFO: [suci-0-999-70-0000-0-0-0000000004] Unknown UE by SUCI (../src/amf/context.c:1390) 8/2/2022 16:58:29 | 02/08 16:58:29.749: [amf] INFO: [Added] Number of AMF-UEs is now 2 (../src/amf/context.c:1189) 8/2/2022 16:58:29 | 02/08 16:58:29.749: [gmm] INFO: Registration request (../src/amf/gmm-sm.c:134) 8/2/2022 16:58:29 | 02/08 16:58:29.749: [gmm] INFO: [suci-0-999-70-0000-0-0-0000000004] SUCI (../src/amf/gmm-handler.c:157) 8/2/2022 16:58:29 | 02/08 16:58:29.750: [amf] INFO: InitialUEMessage (../src/amf/ngap-handler.c:361) 8/2/2022 16:58:29 | 02/08 16:58:29.750: [amf] INFO: [Added] Number of gNB-UEs is now 3 (../src/amf/context.c:2064) 8/2/2022 16:58:29 | 02/08 16:58:29.750: [amf] INFO: RAN_UE_NGAP_ID[3] AMF_UE_NGAP_ID[7] TAC[1] CellID[0x100] (../src/amf/ngap-handler.c:497) 8/2/2022 16:58:29 | 02/08 16:58:29.750: [amf] INFO: [suci-0-999-70-0000-0-0-0000000002] Unknown UE by SUCI (../src/amf/context.c:1390) 8/2/2022 16:58:29 | 02/08 16:58:29.750: [amf] INFO: [Added] Number of AMF-UEs is now 3 (../src/amf/context.c:1189) 8/2/2022 16:58:29 | 02/08 16:58:29.750: [gmm] INFO: Registration request (../src/amf/gmm-sm.c:134) 8/2/2022 16:58:29 | 02/08 16:58:29.750: [gmm] INFO: [suci-0-999-70-0000-0-0-0000000002] SUCI (../src/amf/gmm-handler.c:157) 8/2/2022 16:58:29 | 02/08 16:58:29.750: [amf] INFO: InitialUEMessage (../src/amf/ngap-handler.c:361) 8/2/2022 16:58:29 | 02/08 16:58:29.750: [amf] INFO: [Added] Number of gNB-UEs is now 4 (../src/amf/context.c:2064) 8/2/2022 16:58:29 | 02/08 16:58:29.750: [amf] INFO: RAN_UE_NGAP_ID[4] AMF_UE_NGAP_ID[8] TAC[1] CellID[0x100] (../src/amf/ngap-handler.c:497) 8/2/2022 16:58:29 | 02/08 16:58:29.750: [amf] INFO: [suci-0-999-70-0000-0-0-0000000003] Unknown UE by SUCI (../src/amf/context.c:1390) 8/2/2022 16:58:29 | 02/08 16:58:29.750: [amf] INFO: [Added] Number of AMF-UEs is now 4 (../src/amf/context.c:1189) 8/2/2022 16:58:29 | 02/08 16:58:29.750: [gmm] INFO: Registration request (../src/amf/gmm-sm.c:134) 8/2/2022 16:58:29 | 02/08 16:58:29.750: [gmm] INFO: [suci-0-999-70-0000-0-0-0000000003] SUCI (../src/amf/gmm-handler.c:157) 8/2/2022 16:58:29 | 02/08 16:58:29.967: [gmm] WARNING: [suci-0-999-70-0000-0-0-0000000001] Cannot find SUCI [404] (../src/amf/gmm-sm.c:649) 8/2/2022 16:58:29 | 02/08 16:58:29.967: [amf] WARNING: [suci-0-999-70-0000-0-0-0000000001] Registration reject [11] (../src/amf/nas-path.c:154) 8/2/2022 16:58:29 | 02/08 16:58:29.987: [gmm] WARNING: [suci-0-999-70-0000-0-0-0000000002] Cannot find SUCI [404] (../src/amf/gmm-sm.c:649) 8/2/2022 16:58:29 | 02/08 16:58:29.987: [amf] WARNING: [suci-0-999-70-0000-0-0-0000000002] Registration reject [11] (../src/amf/nas-path.c:154) 8/2/2022 16:58:29 | 02/08 16:58:29.987: [gmm] WARNING: [suci-0-999-70-0000-0-0-0000000004] Cannot find SUCI [404] (../src/amf/gmm-sm.c:649) 8/2/2022 16:58:29 | 02/08 16:58:29.987: [amf] WARNING: [suci-0-999-70-0000-0-0-0000000004] Registration reject [11] (../src/amf/nas-path.c:154) 8/2/2022 16:58:29 | 02/08 16:58:29.987: [gmm] WARNING: [suci-0-999-70-0000-0-0-0000000003] Cannot find SUCI [404] (../src/amf/gmm-sm.c:649) 8/2/2022 16:58:29 | 02/08 16:58:29.987: [amf] WARNING: [suci-0-999-70-0000-0-0-0000000003] Registration reject [11] (../src/amf/nas-path.c:154) 8/2/2022 16:58:29 | 02/08 16:58:29.987: [amf] INFO: UE Context Release [Action:3] (../src/amf/ngap-handler.c:1398) 8/2/2022 16:58:29 | 02/08 16:58:29.987: [amf] INFO: RAN_UE_NGAP_ID[1] AMF_UE_NGAP_ID[5] (../src/amf/ngap-handler.c:1399) 8/2/2022 16:58:29 | 02/08 16:58:29.987: [amf] INFO: SUCI[suci-0-999-70-0000-0-0-0000000001] (../src/amf/ngap-handler.c:1402) 8/2/2022 16:58:29 | 02/08 16:58:29.987: [amf] INFO: [Removed] Number of gNB-UEs is now 3 (../src/amf/context.c:2070) 8/2/2022 16:58:29 | 02/08 16:58:29.987: [amf] INFO: [Removed] Number of AMF-UEs is now 3 (../src/amf/context.c:1273) 8/2/2022 16:58:29 | 02/08 16:58:29.988: [amf] INFO: UE Context Release [Action:3] (../src/amf/ngap-handler.c:1398) 8/2/2022 16:58:29 | 02/08 16:58:29.988: [amf] INFO: RAN_UE_NGAP_ID[3] AMF_UE_NGAP_ID[7] (../src/amf/ngap-handler.c:1399) 8/2/2022 16:58:29 | 02/08 16:58:29.988: [amf] INFO: SUCI[suci-0-999-70-0000-0-0-0000000002] (../src/amf/ngap-handler.c:1402) 8/2/2022 16:58:29 | 02/08 16:58:29.988: [amf] INFO: [Removed] Number of gNB-UEs is now 2 (../src/amf/context.c:2070) 8/2/2022 16:58:29 | 02/08 16:58:29.988: [amf] INFO: [Removed] Number of AMF-UEs is now 2 (../src/amf/context.c:1273) 8/2/2022 16:58:29 | 02/08 16:58:29.988: [amf] INFO: UE Context Release [Action:3] (../src/amf/ngap-handler.c:1398) 8/2/2022 16:58:29 | 02/08 16:58:29.988: [amf] INFO: RAN_UE_NGAP_ID[2] AMF_UE_NGAP_ID[6] (../src/amf/ngap-handler.c:1399) 8/2/2022 16:58:29 | 02/08 16:58:29.988: [amf] INFO: SUCI[suci-0-999-70-0000-0-0-0000000004] (../src/amf/ngap-handler.c:1402) 8/2/2022 16:58:29 | 02/08 16:58:29.988: [amf] INFO: [Removed] Number of gNB-UEs is now 1 (../src/amf/context.c:2070) 8/2/2022 16:58:29 | 02/08 16:58:29.988: [amf] INFO: [Removed] Number of AMF-UEs is now 1 (../src/amf/context.c:1273) 8/2/2022 16:58:30 | 02/08 16:58:30.190: [amf] INFO: UE Context Release [Action:3] (../src/amf/ngap-handler.c:1398) 8/2/2022 16:58:30 | 02/08 16:58:30.190: [amf] INFO: RAN_UE_NGAP_ID[4] AMF_UE_NGAP_ID[8] (../src/amf/ngap-handler.c:1399) 8/2/2022 16:58:30 | 02/08 16:58:30.190: [amf] INFO: SUCI[suci-0-999-70-0000-0-0-0000000003] (../src/amf/ngap-handler.c:1402) 8/2/2022 16:58:30 | 02/08 16:58:30.190: [amf] INFO: [Removed] Number of gNB-UEs is now 0 (../src/amf/context.c:2070) 8/2/2022 16:58:30 | 02/08 16:58:30.190: [amf] INFO: [Removed] Number of AMF-UEs is now 0 (../src/amf/context.c:1273)

cgiraldo commented 2 years ago

Did you correctly register the UEs in the core?

DiogoCruz40 commented 2 years ago

I'm sry there's no manual on how to do that, can you specify what i have to register?

EDIT: SO i Subscribed in web-ui 999700000000001,...,999700000000004 and now the uesimtun0,...,uesimtun4 are showing but when i try to ping something with this is 100% packet loss

PS: In amf,upf and logs of ue it's everything ok,it seems.

Maybe something with NAT and iptables but i dont know...?

i have tested everything and i can't figure out where the interfaces are failling to proceed to public internet

cgiraldo commented 2 years ago

Nat should be enabled by default in upf, giving you internet connectivity to ues. Take a look to the entrypoint.sh of the openverso/open5gs image (https://github.com/Gradiant/openverso-images/blob/main/open5gs/entrypoint.sh).

You can check there how upf containers configures NAT.

I hope that helps to debug your problem.

DiogoCruz40 commented 2 years ago

Is something wrong with ogstun

Captura de ecrã de 2022-02-10 18-52-19

And about my subscribers they are ok? Captura de ecrã de 2022-02-10 18-59-11

I think it's trying to use ipv6 to acess internet or uerasim Captura de ecrã de 2022-02-10 19-03-24

Network ipv6 not reachable Captura de ecrã de 2022-02-10 19-12-24

DiogoCruz40 commented 2 years ago

I'm in portugal, do my mcc and mnc matter here? Is there any configurations that i have to do/change?

cgiraldo commented 2 years ago

No. You are using UERANSIM so you are not actually using radio resources but simulation. The requirements is that MCC, MNC and TAC to be the same across UERANSIM and open5gs. Default UERANSIM and open5gs in values.yaml are the same for this parameters:

MCC: 901
MNC: 70
TAC: 0001

So you can keep the default values and It should work.

Have you try internet access through the ue container?

DiogoCruz40 commented 2 years ago

Yes i have tried, so for the eth0 it ping's outside ip's like 8.8.8.8, but for interfaces like uesimtun0 it's not working, 100% package loss.

cgiraldo commented 2 years ago

have the ue receive an IP in the uesimtun0 interface (10.45.0.X)? have you ping to the IP of the other side of the tunel (10.45.0.1)? have you a correct ip route entry in ue setting the gateway to the IP of the other side of the uesimtun0?

DiogoCruz40 commented 2 years ago

have the ue receive an IP in the uesimtun0 interface (10.45.0.X)?

Yes, both 4 ue appear but the state is unknown.

have you ping to the IP of the other side of the tunel (10.45.0.1)?

I have tried pinging with interface of one ue, it didn't work. Then i tried ping with eth0 it didn't work either.

have you a correct ip route entry in ue setting the gateway to the IP of the other side of the uesimtun0?

I didn't understand the question, how can i do that?

I didn't find /etc/network/interface or /etc/sysconfig/network-scripts/ or /etc/netplan/, it doesn't exist inside container.

EDIT: image_2022_02_16T17_08_44_178Z

Ok what i find so far, the problem is that the packages doesn't come back only goes to internet, and it doesn't enter in the tunnel backwords.

It seems to have a route to the UEs IPs (.45.), and the tcpdumps seem to indicate that it is sending the packets, but for some reason they don't enter the tunnel (no packets appear between the .42.)

avrodriguezgrad commented 2 years ago

Hi @DiogoCruz40, How did you do the deployment of both charts? Can you tell me which commands did you execute and which files did you change? On the other hand, which CNI do you have in your kubernetes cluster?

DiogoCruz40 commented 2 years ago

Yes @avrodriguezgrad so first i deployed a rke2 cluster and then rancher server. In rancher server i created a namespace called open5gs where i deployed open5gs and ueransim like this:

Deploy of open5gs - https://github.com/Gradiant/openverso-charts

  1. Install helm openebs for dynamic persistence volumes:

    $ helm repo add openebs https://openebs.github.io/charts

    $ helm repo update

    $ helm install openebs --namespace openebs openebs/openebs --create-namespace

  2. Install helm openverso/open5gs:

    $ kubectl create ns open5gs

    $ helm repo add openverso https://gradiant.github.io/openverso-charts/

    $ helm install open5gs openverso/open5gs -n open5gs

  3. Criate persistence volume claim of open5gs-mongodb-pvc with class openebs-hostpath

  4. Edit web ui service to nodeport

Deploy of ueransim

  1. Install the ueransim:

    $ helm repo add openverso https://gradiant.github.io/openverso-charts/

    $ helm install ueransim openverso/ueransim -n open5gs

  2. Put the same MCC and MNC on ueransim from open5gs amf and mme

The canal CNI is the default of rke2 cluster. I've deployed the multus cni on default namespace of cluster to.

Other strange thing happen, after a few hours when i come back to ueransim the network interfaces disappear like they went timeout.

avrodriguezgrad commented 2 years ago

@DiogoCruz40 I'm going to figure out what is happening and I'll tell you something tomorrow

DiogoCruz40 commented 2 years ago

@avrodriguezgrad thank you so much

avrodriguezgrad commented 2 years ago

Hi @DiogoCruz40 We discovered why the chart is failing, so next days we are going to upload a new working version. Sorry for that, I'll tell you when this new version is ready for deployment.

DiogoCruz40 commented 2 years ago

Hello @avrodriguezgrad, thank you good sir. If you could explain why is failling to, i would appreciate it .

avrodriguezgrad commented 2 years ago

Hi @DiogoCruz40 We've updated UERANSIM chart so right now following the steps you told us, both charts (open5gs and ueransim) are working. I attach a image to show you that UEs have connectivity. UEs_working

The previous UERANSIM chart was failing due to the deployment in an unique pod. Our theory is UE's default route was overwritting gNB's default route, so that was the cause of not having connectivity in UE's.

¿Can you test it and tell us if it is working for you? Remember to update the openverso repo with: helm repo update

DiogoCruz40 commented 2 years ago

Hi @avrodriguezgrad it's working, thank you soo much that makes all the sense. Another question, you deployed the UERANSIM through statefulset, shouldn't it be through deployment as this has no state?

avrodriguezgrad commented 2 years ago

Hi @DiogoCruz40

Yes, it could be an option to deploy gnb as deployment, but we follow the same pattern as in open5gs. Design decision, not matter at all.

I'm going to close the PR. Feel free to open it again or open another if you have any problem. Greetings!