search

srsran / srsRAN_Project

Open source O-RAN 5G CU/DU solution from Software Radio Systems (SRS) https://docs.srsran.com/projects/project
https://www.srsran.com
GNU Affero General Public License v3.0
522 stars 178 forks source link

srsUE unable to connect to srsgNB #859

Closed LuxeZN closed 2 weeks ago

LuxeZN commented 1 month ago

Issue Description

Following the tutorial here https://docs.srsran.com/projects/project/en/latest/tutorials/source/srsUE/source/index.html we are able to get srsUE and the srsgNB working with B210s. We are trying to adapt the setup to work on N310s, we have seen some issues here and on the 4G repo to say that the N310 is not reliable as the UE. We were wondering if anyone could provide any insight or solutions.

Setup Details

Both computers have the same internals and setups:

CPU: Intel Core i9-13900TE Ubuntu: 20.04 UHD: 4.6.0.0 Radio: N310 srsRAN_Project commit: 51e44a642 srsRAN_4G commit: ec29b0c1f

Expected Behavior

srsUE to be able to find the srsgNB and connect/sync

Actual Behaviour

srsUE completes cell selection, starts the SIB1 acquisition, but fails the CRC check for PBCH-MIB. Looping error message "PBCH-MIB: CRC failed"

Steps to reproduce the problem

Configuration files: gNB - gnb_conf.txt UE - ue_conf.txt

Additional Information

Benchmark scripts and srsRAN performance scripts have been ran, we have the radios working using OpenAirInterface.

Log files: gNB - gnb.log UE - ue.log

yanggao017 commented 1 month ago

we have met the same problem

LuxeZN commented 1 month ago

we have met the same problem

have you had any progress?

pgawlowicz commented 1 month ago

please try to add the following to the srsue config file:

[expert]
lte_sample_rates = true

check here: https://docs.srsran.com/projects/project/en/latest/tutorials/source/srsUE/source/index.html#usrp-x300-x310

LuxeZN commented 1 month ago

please try to add the following to the srsue config file:

[expert]
lte_sample_rates = true

check here: https://docs.srsran.com/projects/project/en/latest/tutorials/source/srsUE/source/index.html#usrp-x300-x310

Retested with that parameter in the UE file, also tested with no timing advance, 45 timing advance, and 300 timing advance. Here are the logs: No timing advance: gNB - gnb.log UE - ue.log

45 timing advance: gNB - gnb45.log UE - ue45.log

300 timing advance: gNB - gnb300.log UE - ue300.log

The 45 timing advance seems to provide the most interesting result in which we are getting "PBCH-MIB: NR SFN (XXX) does not match current SFN (XXX)". Not sure if this is just a timing advance problem and we just need to iron out what value to use or something else? As a reference using OpenAirInterface gNB and UE we used 45 as our timing advance parameter which is why that value was tested.

LuxeZN commented 1 month ago

Also, in the terminal output at least we are getting confirmation that the RA procedure is trying to start after adding the parameter into the UE config. " Random Access Transmission: prach_occasion=0, preamble_index=0, ra-rnti=0xf, tti=5451 Random Access Transmission: prach_occasion=0, preamble_index=0, ra-rnti=0xf, tti=5481 Random Access Transmission: prach_occasion=0, preamble_index=0, ra-rnti=0xf, tti=5511 Random Access Transmission: prach_occasion=0, preamble_index=0, ra-rnti=0xf, tti=5541 Random Access Transmission: prach_occasion=0, preamble_index=0, ra-rnti=0xf, tti=5571 Random Access Transmission: prach_occasion=0, preamble_index=0, ra-rnti=0xf, tti=5601 Random Access Transmission: prach_occasion=0, preamble_index=0, ra-rnti=0xf, tti=5611 "

pgawlowicz commented 1 month ago

could you try the patch from this comment https://github.com/srsran/srsRAN_Project/issues/826#issuecomment-2370588502

LuxeZN commented 1 month ago

could you try the patch from this comment #826 (comment)

Just tried, here are logs. Not sure why there are so many underflows in UE. Same terminal output as above. Logs: gNB - gnb_after_patch.log UE - ue_after_patch.log

pgawlowicz commented 1 month ago

do you run over RF cables or over the air?

could you reduce tx/rx gains of both gnb and ue?

LuxeZN commented 1 month ago

do you run over RF cables or over the air?

could you reduce tx/rx gains of both gnb and ue?

RF cables, 30 dB attenuation on each TX port, reduced RX and TX gain to 35 on gNB and UE. Logs: gNB - gnb_gain_down.log UE - ue_gain_down.log

pgawlowicz commented 1 month ago

the UE sends PRACH but the gnb does not receive it.

there are also issues with RF on gnb:

2024-10-16T17:38:27.499954 [RF      ] [W] Real-time failure in RF: underflow
2024-10-16T17:38:27.502869 [RF      ] [W] Real-time failure in RF: late

Do you run both gnb and srsue on the same pc?

Did you execute srsran_performance script?

Are the n310 directly connected to hostmachines?

Could you try this config for the gnb?

ru_sdr:
  device_driver: uhd
  device_args: type=n3xx, clock_source=external, time_source=external
  clock: external
  sync: external
  srate: 30.72
  tx_gain: 35
  rx_gain: 60
LuxeZN commented 1 month ago

I realized that, still not sure why there are so many RF issues.

  1. gNB and srsue are ran on separate machines
  2. The performance script has been ran on both machines
  3. The N310s are connected directly to host machines

Ran with that configuration you just sent, here are the logs: gNB - gnb_new_conf.log UE - ue_new_gnb_conf.log

pgawlowicz commented 1 month ago

Which UHD version do you use? Could you update ubuntu to at least 22.04? I think we are no longer supporting 20.04

LuxeZN commented 1 month ago

UHD is 4.6.0.0 on both machines, need to use 20.04 unfortunately. What do you think could be the difference in 22 vs 20 for this problem?

LuxeZN commented 1 month ago

Also, is there any patch needed to apply to srsRAN_Project repo instead of 4G from https://github.com/srsran/srsRAN_Project/issues/826#issuecomment-2370588502 ?

pgawlowicz commented 1 month ago

It is worth checking. A user was able to run gnb correctly with ubuntu 24.04 and n310 here: https://github.com/srsran/srsRAN_Project/discussions/863

LuxeZN commented 1 month ago

It is worth checking. A user was able to run gnb correctly with ubuntu 24.04 and n310 here: #863

We have to use 20.04 unfortunately. We will be doing some more testing I will put results here, let me know if you have any more insights on what could be going on.

LuxeZN commented 1 month ago

Testing cross configuration with N310/B210 as gNB and UE:

N310 as gNB and B210 as UE: Configs: gNB - gnb_n310_config.txt UE - ue_b210_config.txt Logs: gNB - gnb_n310_ue_b210.log UE - ue_b210_gnb_n310.log

Does not work, won't start RA procedure. Also tried using the working B210 config (the both config) below for the UE, same result

N310 as UE and B210 as gNB: Configs: gNB - gnb_b210_config.txt UE - ue_n310_config.txt Logs: gNB - gnb_b210.log UE - ue_n310.log

Does not work, won't start RA procedure. Also tried using the working B210 config (the both config) below for the gNB, same result.

B210 as UE and B210 as gNB: Configs: gNB - gnb_b210_both_config.txt UE - ue_b210_both_config.txt Logs: gNB - gnb_b210_both.log UE - ue_b210_both.log

This one works as expected, config files are almost identical to ones tried with N310s. Not sure what would change between them or have any affect but maybe you have some more insight? I really don't believe this to be an operating system support issue, please let me know your thoughts. I can do more testing to investigate specifics too.

LuxeZN commented 1 month ago

Just tried configuration with B210 as UE and N310 as gNB using configurations from above with both hooked up to the octoclock with the same result. Configs: gNB - gnb_n310_config (above) UE - ue_b210_both_config (above) Logs: gNB - gnb_n310_octo.log UE - ue_b210_octo.log

RA procedure never starts.

Erick5G commented 1 month ago

Hi @pgawlowicz @LuxeZN Will you know how to solve this issue?

unam5g@unam5g-Precision-3630-Tower:~$ uhd_usrp_probe [INFO] [UHD] linux; GNU C++ version 11.4.0; Boost_107400; UHD_4.7.0.0-0ubuntu1~jammy1 [INFO] [MPMD] Initializing 1 device(s) in parallel with args: mgmt_addr=192.168.20.2,type=n3xx,product=n310,serial=316CD0E,claimed=False,addr=192.168.20.2 [INFO] [MPM.main] Launching USRP/MPM, version: 3.13.0.2-gunknown [INFO] [MPM.main] Spawning RPC process... [INFO] [MPM.PeriphManager] Device serial number: 316CD0E [INFO] [MPM.PeriphManager] Initialized 2 daughterboard(s). [ERROR] [MPM.PeriphManager] Major compat number mismatch for component `FPGA': Expected: 5.2 Actual: 8.2 [ERROR] [MPM.PeriphManager] Failed to initialize motherboard: Major compat number mismatch for component `FPGA': Expected: 5.2 Actual: 8.2 [ERROR] [MPM.PeriphManager] Cannot run init(), device was never fully initialized! [INFO] [MPM.RPCServer] RPC server ready! [INFO] [MPM.RPCServer] Spawning watchdog task... Error: rpc::timeout: Timeout of 2000ms while calling RPC function 'set_device_id' unam5g@unam5g-Precision-3630-Tower:~$ uhd_image_loader --args="type=n3xx,addr=192.168.20.2" [INFO] [UHD] linux; GNU C++ version 11.4.0; Boost_107400; UHD_4.7.0.0-0ubuntu1~jammy1 [INFO] [MPMD] Initializing 1 device(s) in parallel with args: mgmt_addr=192.168.20.2,type=n3xx,product=n310,serial=316CD0E,claimed=False,skip_init=1 [WARNING] [MPM.RPCServer] A timeout event occured! [WARNING] [MPM.PeriphManager] Cannot run deinit(), device was never fully initialized! [INFO] [MPMD] Claimed device without full initialization. Error: EnvironmentError: IOError: Could not find the image 'usrp_n310_fpga.bit' in the image directory /usr/share/uhd/images/ For more information regarding image paths, please refer to the UHD manual.

I've already tried to follow the steps from ettus manual & uhd

LuxeZN commented 1 month ago

Hi @pgawlowicz @LuxeZN Will you know how to solve this issue?

unam5g@unam5g-Precision-3630-Tower:~$ uhd_usrp_probe [INFO] [UHD] linux; GNU C++ version 11.4.0; Boost_107400; UHD_4.7.0.0-0ubuntu1~jammy1 [INFO] [MPMD] Initializing 1 device(s) in parallel with args: mgmt_addr=192.168.20.2,type=n3xx,product=n310,serial=316CD0E,claimed=False,addr=192.168.20.2 [INFO] [MPM.main] Launching USRP/MPM, version: 3.13.0.2-gunknown [INFO] [MPM.main] Spawning RPC process... [INFO] [MPM.PeriphManager] Device serial number: 316CD0E [INFO] [MPM.PeriphManager] Initialized 2 daughterboard(s). [ERROR] [MPM.PeriphManager] Major compat number mismatch for component `FPGA': Expected: 5.2 Actual: 8.2 [ERROR] [MPM.PeriphManager] Failed to initialize motherboard: Major compat number mismatch for component `FPGA': Expected: 5.2 Actual: 8.2 [ERROR] [MPM.PeriphManager] Cannot run init(), device was never fully initialized! [INFO] [MPM.RPCServer] RPC server ready! [INFO] [MPM.RPCServer] Spawning watchdog task... Error: rpc::timeout: Timeout of 2000ms while calling RPC function 'set_device_id' unam5g@unam5g-Precision-3630-Tower:~$ uhd_image_loader --args="type=n3xx,addr=192.168.20.2" [INFO] [UHD] linux; GNU C++ version 11.4.0; Boost_107400; UHD_4.7.0.0-0ubuntu1~jammy1 [INFO] [MPMD] Initializing 1 device(s) in parallel with args: mgmt_addr=192.168.20.2,type=n3xx,product=n310,serial=316CD0E,claimed=False,skip_init=1 [WARNING] [MPM.RPCServer] A timeout event occured! [WARNING] [MPM.PeriphManager] Cannot run deinit(), device was never fully initialized! [INFO] [MPMD] Claimed device without full initialization. Error: EnvironmentError: IOError: Could not find the image 'usrp_n310_fpga.bit' in the image directory /usr/share/uhd/images/ For more information regarding image paths, please refer to the UHD manual.

I've already tried to follow the steps from ettus manual & uhd

This isn't really a srsRAN question related to this issue, but are the UHD versions the same on the USRP and on the host computer? Did you power-cycle the USRP after running uhd_image_loader? UHD should be able to figure all of this out on it's own, I would post this on a UHD forum/mailing list somewhere and myself/others might be able to provide more guidance there.

pgawlowicz commented 4 weeks ago

@LuxeZN your gnb configs for n310 are different from the one for b210 in cell_cfg. Could you unify and check again?

LuxeZN commented 4 weeks ago

@LuxeZN your gnb configs for n310 are different from the one for b210 in cell_cfg. Could you unify and check again?

Here are the logs and configs per request. RA procedure doesn't start when the cell_cfg is updated. Configs: gNB - gnb_n310_config_cell_cfg_update.txt UE - ue_n310_cell_cfg_update.txt Logs: gNB - gnb_n310_cell_cfg_update.log UE - ue_n310_cell_cfg_update.log

pgawlowicz commented 2 weeks ago

there is a lot of Real-time failure in RF: late warnings in the gnb. Could you check with lower BW?

LuxeZN commented 2 weeks ago

there is a lot of Real-time failure in RF: late warnings in the gnb. Could you check with lower BW?

RA procedure doesn't start.

Configs: gNB - gnb_n310_10mhz_conf.txt UE - ue_n310_10mhz_conf.txt Logs: gNB - gnb_n310_10mhz.log UE - ue_n310_10mhz.log

pgawlowicz commented 2 weeks ago

could you use srate = 11.52e6 ?

LuxeZN commented 2 weeks ago

could you use srate = 11.52e6 ?

gNB - "Error: setting Tx sampling rate. Tx Rate 11.52 MHz is invalid. The nearest valid value is 11.170909090909092. srsRAN ERROR: Unable to create radio session."

pgawlowicz commented 2 weeks ago

what about srate=15.36e6?

LuxeZN commented 2 weeks ago

RA Procedure doesn't start. Logs: gNB - gnb_n310_10mhz_new.log UE - ue_n310_10mhz_new.log

pgawlowicz commented 2 weeks ago

are you sure the ssb_nr_arfcn=367230 in the UE config is correct?

gnb prints dl_ssb_arfcn=368190:

sudo ./gnb -c ./gnb_zmq.yaml cell_cfg --dl_arfcn=368640 --band=3 --channel_bandwidth_MHz=10 --common_scs=15

--== srsRAN gNB (commit e1bc0f5c32) ==--

Lower PHY in executor blocking mode.
Cell pci=1, bw=10 MHz, 1T1R, dl_arfcn=368640 (n3), dl_freq=1843.2 MHz, dl_ssb_arfcn=368190, ul_freq=1748.2 MHz

Could you change and try again?

Could you increase TX gains to 50-60?

LuxeZN commented 2 weeks ago

RA Procedure completes, however RRC Connected, then immediately the RRC released. Configs: gNB - gnb_n310_10mhz_conf_new.txt UE - ue_n310_10mhz_conf_new.txt Logs: gNB - gnb_n310_10mhz_new_params.log UE - ue_n310_10mhz_new_params.log

pgawlowicz commented 2 weeks ago

Add the following to your gnb config:

cu_cp:
  inactivity_timer: 7200            # Sets the UE/PDU Session/DRB inactivity timer to 7200 seconds. Supported: [1 - 7200].

It seems that you use an old version of the srsRAN-Project, but I cannot find your commit in the git log. Could you update to the newest version?

LuxeZN commented 2 weeks ago

I checked out release_24_04 as 24_10 wouldn't build on Ubuntu 20.04. Same problem, immediate RRC release. Configs: gNB - gnb_n310_10mhz_conf_new_checkout.txt UE - ue_n310_10mhz_conf_checkout.txt Logs: gNB - gnb_n310_10mhz_new_checkout.log UE - ue_n310_10mhz_new_checkout.log

pgawlowicz commented 2 weeks ago

which gcc version do you have? you need at least 11 to build the newest version.

pgawlowicz commented 2 weeks ago

try to reduce rx_gain in gnb to 30

LuxeZN commented 2 weeks ago

I am upgrading gcc version and will build latest release on Monday. Decreasing rx_gain to 30 resulted in same problem. Here are logs: gNB - gnb_n310_10mhz_newest.log UE - ue_n310_10mhz_newest.log

pgawlowicz commented 2 weeks ago

do you still have this patch in the UE? https://github.com/srsran/srsRAN_Project/issues/826#issuecomment-2370588502

LuxeZN commented 2 weeks ago

do you still have this patch in the UE? #826 (comment)

yes, i can reapply the patch again to make sure on Monday as well.

LuxeZN commented 2 weeks ago

So, I have updated the srsRAN_Project repo to the latest commit, as there was some issues parsing on the release_24_10 version. I also updated gcc to v11.4 (I am still running Ubuntu 20.04 for others reference if they missed earlier in the thread) so I was able to build that version of the repo. I also sanity checked the patch on the srsRAN_4G repo, while there was no changes to be applied, I decided to rebuild the repo again just to make sure that everything was as expected.

All of these changes resulted in the gNB and UE to be able to connect!

Here are the configs and logs, I am planning on doing some more work with the 20mhz configurations to see if anything is different. I will post updates here not sure if you want to close the issue but do as you wish @pgawlowicz. Configurations: gNB - gnb_n310_10mhz_working.txt UE - ue_n310_10mhz_conf_working.txt Logs: gNB - gnb_n310_10mhz_working.log UE - ue_n310_10mhz_working.log

LuxeZN commented 2 weeks ago

Here are the working 20MHz configurations as well for anyone that needs them. gNB - gnb_n310_20mhz_working.txt UE - ue_n310_20mhz_conf_working.txt

LuxeZN commented 2 weeks ago

@yanggao017 see results above.

pgawlowicz commented 2 weeks ago

@LuxeZN Great! Thanks for the configs!