search

srsran / srsRAN_Project

Open source O-RAN 5G CU/DU solution from Software Radio Systems (SRS) https://docs.srsran.com/projects/project
https://www.srsran.com
GNU Affero General Public License v3.0
514 stars 173 forks source link

OSC RIC: E2setup Response Failure #853

Closed rcbarke closed 2 weeks ago

rcbarke commented 3 weeks ago

Issue Description

Please see this issue, closed ambiguously last August: https://github.com/srsran/srsRAN_Project/issues/788

I am experiencing the exact same problem described here, preventing xApp deployment or development. The solution presented at the end of the issue appears incomplete, as it causes build and deployment failures.

All other components of my implementation Open5GS 5GC, SRSGNB, SRSUE, and GNURadio are functional. I am able to deploy multiple UEs on the RAN network within a single VM, however, I am currently unable to connect the RAN gNB to the RIC.

Setup Details

Following this sequence of steps: https://docs.srsran.com/projects/project/en/latest/tutorials/source/near-rt-ric/source/index.html

During this set of steps, the RIC deploys correctly: ric_submgr | RMR is ready now ...

The gNB deploys correctly:

ric@OSC-Core:~/srsRAN_Project/build/apps/gnb$ sudo ./gnb -c gnb_zmq.yaml e2 --addr="10.0.2.10" --bind_addr="10.0.2.1"

--== srsRAN gNB (commit ee1d86cd8) ==--

The PRACH detector will not meet the performance requirements with the configuration {Format 0, ZCZ 0, SCS 1.25kHz, Rx ports 1}.
Lower PHY in executor blocking mode.
Connecting to NearRT-RIC on 10.0.2.10:36421
Available radio types: zmq.
Cell pci=1, bw=10 MHz, 1T1R, dl_arfcn=368500 (n3), dl_freq=1842.5 MHz, dl_ssb_arfcn=368410, ul_freq=1747.5 MHz

N2: Connection to AMF on 10.53.1.2:38412 completed
==== gNB started ===
Type <h> to view help

Expected Behavior

At this point, the ric_rtmgr_sim should log: ric_rtmgr_sim | 2024/04/02 11:07:39 POST /ric/v1/handles/associate-ran-to-e2t body: [{"E2TAddress":"10.0.2.10:38000","ranNamelist":["gnb_001_001_00019b"]}] elapsed: 10.77µs

Actual Behaviour

The RIC never acknowledges the gNB's subscription request. However, the gNB logs reflects registering on the shared E2 link.

Steps to reproduce the problem

The rtmgr git repo has not been updated since 2019 by AT&T/Nokia.

I note that the docker deployment of this RIC is setting up a bridged network adapter for the E2 link between the RIC and gNB on 10.0.2.1. I believe this may be conflicting with the virtual machine's subnet. The original issue seems to suggest replacing these addresses within the git repo into an unused subnet. Wouldn't this break communication with the gNB if its configuration isn't also updated?

Is there a known issue with the OSC RIC's E2 interface? I note several research efforts (ex: OAIC, NS-O-RAN, etc.) are modifying the E2Term/E2 Interface. I have not made any modifications to the RIC.

rcbarke commented 3 weeks ago

Note this log with the e2 docker logs: {"ts":1728489330638,"crit":"ERROR","id":"E2Terminator","mdc":{"PID":"1","POD_NAME":"e2term_pod","CONTAINER_NAME":"ric_e2term","SERVICE_NAME":"ric_e2term_service","HOST_NAME":"ric_e2term_host","SYSTEM_NAME":"oran_sc_ric"},"msg":"Error sending E2_TERM_INIT cause : RMR_ERR_NOENDPT - send//call could not find an endpoint based on msg type "}

Full log files are attached. logs.zip

rcbarke commented 3 weeks ago

I am also wondering if the RCA of both of these issues are related. Are the RIC messages being intercepted by the VirtualBox subnet? https://github.com/srsran/srsRAN_Project/issues/849

rcbarke commented 3 weeks ago

Here are runtime logs:

Open5GS

open5gs_5gc  | 10/09 20:56:53.866: [amf] INFO: [Added] Number of gNB-UEs is now 1 (../src/amf/context.c:2550)
open5gs_5gc  | 10/09 20:56:53.866: [amf] INFO:     RAN_UE_NGAP_ID[0] AMF_UE_NGAP_ID[1] TAC[7] CellID[0x66c000] (../src/amf/ngap-handler.c:562)
open5gs_5gc  | 10/09 20:56:53.866: [amf] INFO: [suci-0-001-01-0000-0-0-0123456780] Unknown UE by SUCI (../src/amf/context.c:1835)
open5gs_5gc  | 10/09 20:56:53.866: [amf] INFO: [Added] Number of AMF-UEs is now 1 (../src/amf/context.c:1616)
open5gs_5gc  | 10/09 20:56:53.866: [gmm] INFO: Registration request (../src/amf/gmm-sm.c:1165)
open5gs_5gc  | 10/09 20:56:53.866: [gmm] INFO: [suci-0-001-01-0000-0-0-0123456780]    SUCI (../src/amf/gmm-handler.c:166)
open5gs_5gc  | 10/09 20:56:54.388: [gmm] INFO: [imsi-001010123456780] Registration complete (../src/amf/gmm-sm.c:2146)
open5gs_5gc  | 10/09 20:56:54.388: [amf] INFO: [imsi-001010123456780] Configuration update command (../src/amf/nas-path.c:612)
open5gs_5gc  | 10/09 20:56:54.388: [gmm] INFO:     UTC [2024-10-09T20:56:54] Timezone[0]/DST[0] (../src/amf/gmm-build.c:559)
open5gs_5gc  | 10/09 20:56:54.388: [gmm] INFO:     LOCAL [2024-10-09T20:56:54] Timezone[0]/DST[0] (../src/amf/gmm-build.c:564)
open5gs_5gc  | 10/09 20:56:54.388: [amf] INFO: [Added] Number of AMF-Sessions is now 1 (../src/amf/context.c:2571)
open5gs_5gc  | 10/09 20:56:54.388: [gmm] INFO: UE SUPI[imsi-001010123456780] DNN[srsapn] S_NSSAI[SST:1 SD:0xffffff] smContextRef [NULL] (../src/amf/gmm-handler.c:1241)
open5gs_5gc  | 10/09 20:56:54.388: [gmm] INFO: No SMF Instance (../src/amf/gmm-handler.c:1278)
open5gs_5gc  | 10/09 20:56:54.389: [sbi] WARNING: [SMF] (NRF-discover) NF has already been added [87518e96-8680-41ef-af99-09ea17f61dda:1] (../lib/sbi/nnrf-handler.c:1057)
open5gs_5gc  | 10/09 20:56:54.389: [sbi] WARNING: NF EndPoint(addr) updated [127.0.0.4:80] (../lib/sbi/context.c:2174)
open5gs_5gc  | 10/09 20:56:54.389: [sbi] WARNING: NF EndPoint(addr) updated [127.0.0.4:7777] (../lib/sbi/context.c:1917)
open5gs_5gc  | 10/09 20:56:54.389: [sbi] INFO: [SMF] (NF-discover) NF Profile updated [87518e96-8680-41ef-af99-09ea17f61dda:1] (../lib/sbi/nnrf-handler.c:1095)
open5gs_5gc  | 10/09 20:56:54.390: [smf] INFO: [Added] Number of SMF-UEs is now 1 (../src/smf/context.c:1019)
open5gs_5gc  | 10/09 20:56:54.390: [smf] INFO: [Added] Number of SMF-Sessions is now 1 (../src/smf/context.c:3068)
open5gs_5gc  | 10/09 20:56:54.392: [sbi] INFO: [SMF] (SCP-discover) NF registered [87518e96-8680-41ef-af99-09ea17f61dda:1] (../lib/sbi/path.c:211)
open5gs_5gc  | 10/09 20:56:54.394: [smf] INFO: UE SUPI[imsi-001010123456780] DNN[srsapn] IPv4[10.45.1.2] IPv6[] (../src/smf/npcf-handler.c:539)
open5gs_5gc  | 10/09 20:56:54.394: [upf] INFO: [Added] Number of UPF-Sessions is now 1 (../src/upf/context.c:208)
open5gs_5gc  | 10/09 20:56:54.394: [gtp] INFO: gtp_connect() [127.0.0.4]:2152 (../lib/gtp/path.c:60)
open5gs_5gc  | 10/09 20:56:54.394: [upf] INFO: UE F-SEID[UP:0x3ab CP:0x3a1] APN[srsapn] PDN-Type[1] IPv4[10.45.1.2] IPv6[] (../src/upf/context.c:485)
open5gs_5gc  | 10/09 20:56:54.394: [upf] INFO: UE F-SEID[UP:0x3ab CP:0x3a1] APN[srsapn] PDN-Type[1] IPv4[10.45.1.2] IPv6[] (../src/upf/context.c:485)
open5gs_5gc  | 10/09 20:56:54.394: [gtp] INFO: gtp_connect() [10.53.1.2]:2152 (../lib/gtp/path.c:60)
open5gs_5gc  | 10/09 20:56:54.507: [gtp] INFO: gtp_connect() [10.53.1.1]:2152 (../lib/gtp/path.c:60)
open5gs_5gc  | 10/09 20:56:54.508: [amf] INFO: [imsi-001010123456780:1:11][0:0:NULL] /nsmf-pdusession/v1/sm-contexts/{smContextRef}/modify (../src/amf/nsmf-handler.c:837)
open5gs_5gc  | 10/09 20:56:54.628: [gmm] INFO: [imsi-001010123456780] No GUTI allocated (../src/amf/gmm-sm.c:1443)
open5gs_5gc  | 10/09 20:56:54.628: [amf] INFO: InitialUEMessage (../src/amf/ngap-handler.c:401)
open5gs_5gc  | 10/09 20:56:54.628: [amf] INFO: [Added] Number of gNB-UEs is now 2 (../src/amf/context.c:2550)
open5gs_5gc  | 10/09 20:56:54.628: [amf] INFO:     RAN_UE_NGAP_ID[1] AMF_UE_NGAP_ID[2] TAC[7] CellID[0x66c000] (../src/amf/ngap-handler.c:562)
open5gs_5gc  | 10/09 20:56:54.628: [amf] INFO: [suci-0-001-01-0000-0-0-0123456790] Unknown UE by SUCI (../src/amf/context.c:1835)
open5gs_5gc  | 10/09 20:56:54.628: [amf] INFO: [Added] Number of AMF-UEs is now 2 (../src/amf/context.c:1616)
open5gs_5gc  | 10/09 20:56:54.628: [gmm] INFO: Registration request (../src/amf/gmm-sm.c:1165)
open5gs_5gc  | 10/09 20:56:54.628: [gmm] INFO: [suci-0-001-01-0000-0-0-0123456790]    SUCI (../src/amf/gmm-handler.c:166)
open5gs_5gc  | 10/09 20:56:54.628: [amf] INFO: InitialUEMessage (../src/amf/ngap-handler.c:401)
open5gs_5gc  | 10/09 20:56:54.628: [amf] INFO: [Added] Number of gNB-UEs is now 3 (../src/amf/context.c:2550)
open5gs_5gc  | 10/09 20:56:54.628: [amf] INFO:     RAN_UE_NGAP_ID[2] AMF_UE_NGAP_ID[3] TAC[7] CellID[0x66c000] (../src/amf/ngap-handler.c:562)
open5gs_5gc  | 10/09 20:56:54.628: [amf] INFO: [suci-0-001-01-0000-0-0-0123456791] Unknown UE by SUCI (../src/amf/context.c:1835)
open5gs_5gc  | 10/09 20:56:54.628: [amf] INFO: [Added] Number of AMF-UEs is now 3 (../src/amf/context.c:1616)
open5gs_5gc  | 10/09 20:56:54.628: [gmm] INFO: Registration request (../src/amf/gmm-sm.c:1165)
open5gs_5gc  | 10/09 20:56:54.628: [gmm] INFO: [suci-0-001-01-0000-0-0-0123456791]    SUCI (../src/amf/gmm-handler.c:166)
open5gs_5gc  | 10/09 20:56:55.112: [gmm] INFO: [imsi-001010123456790] Registration complete (../src/amf/gmm-sm.c:2146)
open5gs_5gc  | 10/09 20:56:55.112: [amf] INFO: [imsi-001010123456790] Configuration update command (../src/amf/nas-path.c:612)
open5gs_5gc  | 10/09 20:56:55.112: [gmm] INFO:     UTC [2024-10-09T20:56:55] Timezone[0]/DST[0] (../src/amf/gmm-build.c:559)
open5gs_5gc  | 10/09 20:56:55.112: [gmm] INFO:     LOCAL [2024-10-09T20:56:55] Timezone[0]/DST[0] (../src/amf/gmm-build.c:564)
open5gs_5gc  | 10/09 20:56:55.112: [amf] INFO: [Added] Number of AMF-Sessions is now 2 (../src/amf/context.c:2571)
open5gs_5gc  | 10/09 20:56:55.112: [gmm] INFO: UE SUPI[imsi-001010123456790] DNN[srsapn] S_NSSAI[SST:1 SD:0xffffff] smContextRef [NULL] (../src/amf/gmm-handler.c:1241)
open5gs_5gc  | 10/09 20:56:55.112: [gmm] INFO: SMF Instance [87518e96-8680-41ef-af99-09ea17f61dda] (../src/amf/gmm-handler.c:1280)
open5gs_5gc  | 10/09 20:56:55.112: [gmm] INFO: [imsi-001010123456791] Registration complete (../src/amf/gmm-sm.c:2146)
open5gs_5gc  | 10/09 20:56:55.112: [amf] INFO: [imsi-001010123456791] Configuration update command (../src/amf/nas-path.c:612)
open5gs_5gc  | 10/09 20:56:55.112: [gmm] INFO:     UTC [2024-10-09T20:56:55] Timezone[0]/DST[0] (../src/amf/gmm-build.c:559)
open5gs_5gc  | 10/09 20:56:55.112: [gmm] INFO:     LOCAL [2024-10-09T20:56:55] Timezone[0]/DST[0] (../src/amf/gmm-build.c:564)
open5gs_5gc  | 10/09 20:56:55.112: [amf] INFO: [Added] Number of AMF-Sessions is now 3 (../src/amf/context.c:2571)
open5gs_5gc  | 10/09 20:56:55.112: [gmm] INFO: UE SUPI[imsi-001010123456791] DNN[srsapn] S_NSSAI[SST:1 SD:0xffffff] smContextRef [NULL] (../src/amf/gmm-handler.c:1241)
open5gs_5gc  | 10/09 20:56:55.112: [gmm] INFO: SMF Instance [87518e96-8680-41ef-af99-09ea17f61dda] (../src/amf/gmm-handler.c:1280)
open5gs_5gc  | 10/09 20:56:55.113: [smf] INFO: [Added] Number of SMF-UEs is now 2 (../src/smf/context.c:1019)
open5gs_5gc  | 10/09 20:56:55.113: [smf] INFO: [Added] Number of SMF-Sessions is now 2 (../src/smf/context.c:3068)
open5gs_5gc  | 10/09 20:56:55.113: [smf] INFO: [Added] Number of SMF-UEs is now 3 (../src/smf/context.c:1019)
open5gs_5gc  | 10/09 20:56:55.113: [smf] INFO: [Added] Number of SMF-Sessions is now 3 (../src/smf/context.c:3068)
open5gs_5gc  | 10/09 20:56:55.117: [smf] INFO: UE SUPI[imsi-001010123456791] DNN[srsapn] IPv4[10.45.1.5] IPv6[] (../src/smf/npcf-handler.c:539)
open5gs_5gc  | 10/09 20:56:55.117: [smf] INFO: UE SUPI[imsi-001010123456790] DNN[srsapn] IPv4[10.45.1.4] IPv6[] (../src/smf/npcf-handler.c:539)
open5gs_5gc  | 10/09 20:56:55.117: [upf] INFO: [Added] Number of UPF-Sessions is now 2 (../src/upf/context.c:208)
open5gs_5gc  | 10/09 20:56:55.117: [upf] INFO: UE F-SEID[UP:0x3e0 CP:0x5f4] APN[srsapn] PDN-Type[1] IPv4[10.45.1.5] IPv6[] (../src/upf/context.c:485)
open5gs_5gc  | 10/09 20:56:55.117: [upf] INFO: UE F-SEID[UP:0x3e0 CP:0x5f4] APN[srsapn] PDN-Type[1] IPv4[10.45.1.5] IPv6[] (../src/upf/context.c:485)
open5gs_5gc  | 10/09 20:56:55.117: [upf] INFO: [Added] Number of UPF-Sessions is now 3 (../src/upf/context.c:208)
open5gs_5gc  | 10/09 20:56:55.117: [upf] INFO: UE F-SEID[UP:0x705 CP:0x738] APN[srsapn] PDN-Type[1] IPv4[10.45.1.4] IPv6[] (../src/upf/context.c:485)
open5gs_5gc  | 10/09 20:56:55.117: [upf] INFO: UE F-SEID[UP:0x705 CP:0x738] APN[srsapn] PDN-Type[1] IPv4[10.45.1.4] IPv6[] (../src/upf/context.c:485)
open5gs_5gc  | 10/09 20:56:55.231: [amf] INFO: [imsi-001010123456790:1:11][0:0:NULL] /nsmf-pdusession/v1/sm-contexts/{smContextRef}/modify (../src/amf/nsmf-handler.c:837)
open5gs_5gc  | 10/09 20:56:55.352: [gmm] INFO: [imsi-001010123456790] No GUTI allocated (../src/amf/gmm-sm.c:1443)
open5gs_5gc  | 10/09 20:56:55.352: [gmm] INFO: [imsi-001010123456791] No GUTI allocated (../src/amf/gmm-sm.c:1443)
open5gs_5gc  | 10/09 20:56:55.354: [amf] INFO: [imsi-001010123456791:1:11][0:0:NULL] /nsmf-pdusession/v1/sm-contexts/{smContextRef}/modify (../src/amf/nsmf-handler.c:837)

OSC RIC

[+] Running 7/0
 ✔ Container ric_dbaas           Created                                                                                                                                                       0.0s 
 ✔ Container ric_submgr          Created                                                                                                                                                       0.0s 
 ✔ Container ric_rtmgr_sim       Created                                                                                                                                                       0.0s 
 ✔ Container ric_e2term          Created                                                                                                                                                       0.0s 
 ✔ Container ric_appmgr          Created                                                                                                                                                       0.0s 
 ✔ Container ric_e2mgr           Created                                                                                                                                                       0.0s 
 ✔ Container python_xapp_runner  Created                                                                                                                                                       0.0s 
Attaching to python_xapp_runner, ric_appmgr, ric_dbaas, ric_e2mgr, ric_e2term, ric_rtmgr_sim, ric_submgr
ric_e2mgr           | {"ts":1728507317034,"crit":"INFO","id":"e2mgr","mdc":{"CONTAINER_NAME":"ric_e2mgr","HOST_NAME":"ric_e2mgr_host","PID":"1","POD_NAME":"e2mgr_pod","SERVICE_NAME":"ric_e2mgr_service","SYSTEM_NAME":"oran_sc_ric","e2mgr":"0.2.2","time":"2024-10-09T20:55:17Z"},"msg":"#app.main - Configuration {logging.logLevel: debug, http.port: 3800, rmr: { port: 3801, maxMsgSize: 65536}, routingManager.baseUrl: http://10.0.2.15:12020/ric/v1/handles/, notificationResponseBuffer: 100, bigRedButtonTimeoutSec: 5, maxRnibConnectionAttempts: 3, rnibRetryIntervalMs: 10, keepAliveResponseTimeoutMs: 4500, keepAliveDelayMs: 1500, e2tInstanceDeletionTimeoutMs: 15000,e2ResetTimeOutSec: 10,globalRicId: { ricId: AACCE, mcc: 001, mnc: 01}, rnibWriter: { stateChangeMessageChannel: RAN_CONNECTION_STATUS_CHANGE, ranManipulationChannel: RAN_MANIPULATION}"}
ric_e2mgr           | {"ts":1728507317034,"crit":"INFO","id":"e2mgr","mdc":{"CONTAINER_NAME":"ric_e2mgr","HOST_NAME":"ric_e2mgr_host","PID":"1","POD_NAME":"e2mgr_pod","SERVICE_NAME":"ric_e2mgr_service","SYSTEM_NAME":"oran_sc_ric","e2mgr":"0.2.2","time":"2024-10-09T20:55:17Z"},"msg":"Using config file: /opt/E2Manager/resources/configuration.yaml\n"}
ric_rtmgr_sim       | 2024/10/09 20:55:17 Server started on port 12020
ric_dbaas           | 1:C 09 Oct 2024 20:55:17.388 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
ric_dbaas           | 1:C 09 Oct 2024 20:55:17.388 # Redis version=6.2.14, bits=64, commit=00000000, modified=0, pid=1, just started
ric_dbaas           | 1:C 09 Oct 2024 20:55:17.388 # Configuration loaded
ric_dbaas           | 1:M 09 Oct 2024 20:55:17.388 * monotonic clock: POSIX clock_gettime
ric_dbaas           | 1:M 09 Oct 2024 20:55:17.388 * Running mode=standalone, port=6379.
ric_dbaas           | 1:M 09 Oct 2024 20:55:17.388 # Server initialized
ric_dbaas           | 1:M 09 Oct 2024 20:55:17.389 * Module 'exstrings' loaded from /usr/local/libexec/redismodule/libredismodule.so
ric_dbaas           | 1:M 09 Oct 2024 20:55:17.389 * Loading RDB produced by version 6.2.14
ric_dbaas           | 1:M 09 Oct 2024 20:55:17.389 * RDB age 165 seconds
ric_dbaas           | 1:M 09 Oct 2024 20:55:17.389 * RDB memory usage when created 0.78 Mb
ric_dbaas           | 1:M 09 Oct 2024 20:55:17.389 # Done loading RDB, keys loaded: 1, keys expired: 0.
ric_dbaas           | 1:M 09 Oct 2024 20:55:17.389 * DB loaded from disk: 0.000 seconds
ric_dbaas           | 1:M 09 Oct 2024 20:55:17.389 * Ready to accept connections
ric_e2term          | 1728507317553 1/RMR [INFO] ric message routing library on SI95 p=38000 mv=3 flg=00 id=a (f447e29 4.9.4 built: Dec 13 2023)
ric_appmgr          | cp: cannot stat '/opt/ric/certificates/*': No such file or directory
ric_appmgr          | 2024/10/09 20:55:17 Using config file: /opt/ric/config/appmgr.yaml
ric_appmgr          | redis: got 7 elements in COMMAND reply, wanted 6
ric_appmgr          | 2024/10/09 20:55:17 Serving app manager at http://[::]:8080
ric_submgr          | {"ts":1728507317792,"crit":"INFO","id":"submgr","mdc":{"time":"2024-10-09T20:55:17"},"msg":"Using config file: /opt/config/submgr-config.yaml"}
ric_submgr          | redis: got 7 elements in COMMAND reply, wanted 6
ric_submgr          | redis: got 7 elements in COMMAND reply, wanted 6
ric_submgr          | redis: got 7 elements in COMMAND reply, wanted 6
ric_submgr          | redis: got 7 elements in COMMAND reply, wanted 6
ric_submgr          | 1728507317797 1/RMR [INFO] ric message routing library on SI95 p=4560 mv=3 flg=00 id=a (f447e29 4.9.4 built: Dec 13 2023)
ric_submgr          | 2024/10/09 20:55:17 Serving xapp framework at http://[::]:8088
ric_e2mgr           | redis: got 7 elements in COMMAND reply, wanted 6
ric_e2mgr           | 1728507318049 1/RMR [INFO] ric message routing library on SI95 p=tcp:3801 mv=3 flg=00 id=a (f447e29 4.9.4 built: Dec 13 2023)
ric_submgr          | RMR is ready now ...

SRSGNB

ric@OSC-Core:~/srsRAN_Project/build/apps/gnb$ sudo ./gnb -c gnb_zmq.yaml e2 --addr="10.0.2.10" --bind_addr="10.0.2.1"
[sudo] password for ric: 

--== srsRAN gNB (commit ee1d86cd8) ==--

The PRACH detector will not meet the performance requirements with the configuration {Format 0, ZCZ 0, SCS 1.25kHz, Rx ports 1}.
Lower PHY in executor blocking mode.
Connecting to NearRT-RIC on 10.0.2.10:36421
Available radio types: zmq.
Cell pci=1, bw=10 MHz, 1T1R, dl_arfcn=368500 (n3), dl_freq=1842.5 MHz, dl_ssb_arfcn=368410, ul_freq=1747.5 MHz

N2: Connection to AMF on 10.53.1.2:38412 completed
==== gNB started ===
Type <h> to view help

SRSUE1

ric@OSC-Core:~/srsRAN_4G/build/srsue/src$ sudo ./srsue ./ue1_zmq.conf
[sudo] password for ric: 
Active RF plugins: libsrsran_rf_zmq.so
Inactive RF plugins: 
Reading configuration file ./ue1_zmq.conf...

Built in Release mode using commit ec29b0c1f on branch master.

Opening 1 channels in RF device=zmq with args=tx_port=tcp://127.0.0.1:2101,rx_port=tcp://127.0.0.1:2100,base_srate=11.52e6
Supported RF device list: zmq file
CHx base_srate=11.52e6
Current sample rate is 1.92 MHz with a base rate of 11.52 MHz (x6 decimation)
CH0 rx_port=tcp://127.0.0.1:2100
CH0 tx_port=tcp://127.0.0.1:2101
Current sample rate is 11.52 MHz with a base rate of 11.52 MHz (x1 decimation)
Current sample rate is 11.52 MHz with a base rate of 11.52 MHz (x1 decimation)
Waiting PHY to initialize ... done!
Attaching UE...
Random Access Transmission: prach_occasion=0, preamble_index=34, ra-rnti=0x39, tti=174
Random Access Complete.     c-rnti=0x4601, ta=0
RRC Connected
Failed to setup/configure GW interface
RRC NR reconfiguration successful.

SRSUE2

ric@OSC-Core:~/srsRAN_4G/build/srsue/src$ sudo ./srsue ./ue2_zmq.conf
[sudo] password for ric: 
Active RF plugins: libsrsran_rf_zmq.so
Inactive RF plugins: 
Reading configuration file ./ue2_zmq.conf...

Built in Release mode using commit ec29b0c1f on branch master.

Opening 1 channels in RF device=zmq with args=tx_port=tcp://127.0.0.1:2201,rx_port=tcp://127.0.0.1:2200,base_srate=11.52e6
Supported RF device list: zmq file
CHx base_srate=11.52e6
Current sample rate is 1.92 MHz with a base rate of 11.52 MHz (x6 decimation)
CH0 rx_port=tcp://127.0.0.1:2200
CH0 tx_port=tcp://127.0.0.1:2201
Current sample rate is 11.52 MHz with a base rate of 11.52 MHz (x1 decimation)
Current sample rate is 11.52 MHz with a base rate of 11.52 MHz (x1 decimation)
Waiting PHY to initialize ... done!
Attaching UE...
Random Access Transmission: prach_occasion=0, preamble_index=55, ra-rnti=0x39, tti=174
Random Access Transmission: prach_occasion=0, preamble_index=36, ra-rnti=0x39, tti=334
Random Access Complete.     c-rnti=0x4602, ta=0
RRC Connected
Failed to setup/configure GW interface
RRC NR reconfiguration successful.

SRSUE3

ric@OSC-Core:~/srsRAN_4G/build/srsue/src$ sudo ./srsue ./ue3_zmq.conf
[sudo] password for ric: 
Active RF plugins: libsrsran_rf_zmq.so
Inactive RF plugins: 
Reading configuration file ./ue3_zmq.conf...

Built in Release mode using commit ec29b0c1f on branch master.

Opening 1 channels in RF device=zmq with args=tx_port=tcp://127.0.0.1:2301,rx_port=tcp://127.0.0.1:2300,base_srate=11.52e6
Supported RF device list: zmq file
CHx base_srate=11.52e6
Current sample rate is 1.92 MHz with a base rate of 11.52 MHz (x6 decimation)
CH0 rx_port=tcp://127.0.0.1:2300
CH0 tx_port=tcp://127.0.0.1:2301
Current sample rate is 11.52 MHz with a base rate of 11.52 MHz (x1 decimation)
Current sample rate is 11.52 MHz with a base rate of 11.52 MHz (x1 decimation)
Waiting PHY to initialize ... done!
Attaching UE...
Random Access Transmission: prach_occasion=0, preamble_index=16, ra-rnti=0x39, tti=174
Random Access Transmission: prach_occasion=0, preamble_index=42, ra-rnti=0x39, tti=334
Random Access Complete.     c-rnti=0x4603, ta=0
RRC Connected
Failed to setup/configure GW interface
RRC NR reconfiguration successful.

GNURadio:

ric@OSC-Core:~/srsRAN_4G/build/srsue/src$ sudo gnuradio-companion ./multi_ue_scenario.grc
>>> Warning: vocoder_codec2_decode_ps - option_attributes are for enums only, ignoring
>>> Warning: vocoder_codec2_encode_sp - option_attributes are for enums only, ignoring
<<< Welcome to GNU Radio Companion 3.10.1.1 >>>

Block paths:
    /usr/share/gnuradio/grc/blocks

Loading: "/home/ric/srsRAN_4G/build/srsue/src/multi_ue_scenario.grc"
>>> Done

Generating: '/home/ric/srsRAN_4G/build/srsue/src/multi_ue_scenario.py'

Executing: /usr/bin/python3 -u /home/ric/srsRAN_4G/build/srsue/src/multi_ue_scenario.py

QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root'
pgawlowicz commented 3 weeks ago

In your gnb config you have

e2:
  enable_du_e2: true                # Enable DU E2 agent (one for each DU instance)
  e2sm_kpm_enabled: true            # Enable KPM service module
  addr: 127.0.0.1                   # RIC IP address
  #bind_addr: 127.0.0.100           # A local IP that the E2 agent binds to for traffic from the RIC. ONLY required if running the RIC on a separate machine. 
  port: 36421                       # RIC port

Do you run dockerized ORAN SC RIC, or a manually installed one?

rcbarke commented 3 weeks ago

In your gnb config you have

e2:
  enable_du_e2: true                # Enable DU E2 agent (one for each DU instance)
  e2sm_kpm_enabled: true            # Enable KPM service module
  addr: 127.0.0.1                   # RIC IP address
  #bind_addr: 127.0.0.100           # A local IP that the E2 agent binds to for traffic from the RIC. ONLY required if running the RIC on a separate machine. 
  port: 36421                       # RIC port

Do you run dockerized ORAN SC RIC, or a manually installed one?

The dockerized RIC, built and deployed with docker compose up

pgawlowicz commented 3 weeks ago

In the tutorial, we overwrite those parameters when starting with the following command:

sudo ./gnb -c gnb_zmq.yaml e2 --addr="10.0.2.10" --bind_addr="10.0.2.1"

However, when running dockerized gnb the arguments are not added. You can add them to this command in docker-compose.yml file: https://github.com/srsran/srsRAN_Project/blob/main/docker/docker-compose.yml#L88

Or simply change the IPs in the gnb config to

e2:
  enable_du_e2: true                # Enable DU E2 agent (one for each DU instance)
  e2sm_kpm_enabled: true            # Enable KPM service module
  addr: 10.0.2.10                  # RIC IP address
  bind_addr: 10.0.2.1           # A local IP that the E2 agent binds to for traffic from the RIC. 
  port: 36421                       # RIC port
pgawlowicz commented 3 weeks ago

ah, you did already

rcbarke commented 3 weeks ago

In the tutorial, we overwrite those parameters when starting with the following command:

sudo ./gnb -c gnb_zmq.yaml e2 --addr="10.0.2.10" --bind_addr="10.0.2.1"

However, when running dockerized gnb the arguments are not added. You can add them to this command in docker-compose.yml file: https://github.com/srsran/srsRAN_Project/blob/main/docker/docker-compose.yml#L88

Or simply change the IPs in the gnb config to

e2:
  enable_du_e2: true                # Enable DU E2 agent (one for each DU instance)
  e2sm_kpm_enabled: true            # Enable KPM service module
  addr: 10.0.2.10                  # RIC IP address
  bind_addr: 10.0.2.1           # A local IP that the E2 agent binds to for traffic from the RIC. 
  port: 36421                       # RIC port

ah, you did already

Our RIC is built via docker as per the tutorial, however srsRAN Project, including srsgnb, is built via the instructions within the same tutorial.

git clone https://github.com/srsran/srsRAN_Project.git
cd srsRAN_Project
mkdir build
cd build
cmake ../ -DENABLE_EXPORT=ON -DENABLE_ZEROMQ=ON
make -j`nproc`

When we installed, we noted ZeroMQ was built correctly, as evidenced by the connectivity between srsgnb, srsue, and gnuradio.

I'll try modifying the config file this afternoon, though the logs seem to point to the parameters being replaced as you just noted. If you check the /tmp/gnb.log file here, you will also see that the gnb seems to connect to the RIC, but the RIC's RMR never responds to it.

Thank you for all of the information you provided in https://github.com/srsran/srsRAN_Project/issues/849; I will also work through updating the RIC subnet, leverage your response to ensure everything is modified correctly, and reflect the modified RIC subnet in the srsgnb config file. Is there anywhere else I would need to modify the RIC IP within SRSRAN outside of the above E2 link?

Also, did the tutorial assume a direct installation on the local host OS where the 10.0.2.0/24 subnet would be completely available?

My current postulation is that the overlapping RIC and VM subnets are creating communication problems on both sides (RMR responses for the RIC and cross machine connectivity for the VMs), though I'd welcome any other ideas you have. Thank you for your help, Piotr!

rcbarke commented 3 weeks ago

ah, you did already

One last question: When xApps (ex: KPIMon) are deployed, would they need to be pointed at the modified RIC subnet? I haven't been successful there yet, though the E2 link would need to be fully established for us to begin testing.

pgawlowicz commented 3 weeks ago

It should be already handled here : https://github.com/srsran/oran-sc-ric/blob/main/xApps/python/lib/xAppBase.py#L28-L32

rcbarke commented 2 weeks ago

Root cause of this issue was the virtualbox subnet overlapping with the RIC's subnet. Changing the RIC to an unused class C IP address range as suggested here created connectivity across the gNB/RIC E2 link as well as restored connectivity across both virtualbox VMs. https://github.com/srsran/srsRAN_Project/issues/788#issuecomment-2311996007