srsran / srsRAN_Project

Open source O-RAN 5G CU/DU solution from Software Radio Systems (SRS) https://docs.srsran.com/projects/project
https://www.srsran.com
GNU Affero General Public License v3.0
479 stars 163 forks source link

RF status overflow #597

Closed ruedagarcialaura closed 3 months ago

ruedagarcialaura commented 4 months ago

Issue Description

When trying to connect the UE to the gNB it doesn't attach and gets stuck with overflow values. Sometimes the overflow is up to 150 and the best I got was overflow 1.

Setup Details

Os: Ubuntu 22.04 on 3 PCs UE: srsRAN_4G connected via usb3 to USRP b200 mini gNB: srsRAN_Project connected via usb3 to USRP b200 core: open5gs

Expected Behavior

UE attaches to gNB, a pdu session is established and it gets an IP address.

Something like:

Attaching UE... Random Access Transmission: prach_occasion=0, preamble_index=0, ra-rnti=0x39, tti=2094 Random Access Complete. c-rnti=0x4602, ta=0 RRC Connected PDU Session Establishment successful. IP: 10.45.1.2 RRC NR reconfiguration successful.

Actual Behaviour

$sudo ./srsue ue_rf.conf

AActive RF plugins: libsrsran_rf_uhd.so Inactive RF plugins: Reading configuration file ue_rf.conf...

Built in Release mode using commit ec29b0c1f on branch master.

Opening 1 channels in RF device=uhd with args=default Supported RF device list: UHD file [INFO] [UHD] linux; GNU C++ version 11.2.0; Boost_107400; UHD_4.1.0.5-3 [INFO] [LOGGING] Fastpath logging disabled at runtime. Opening USRP channels=1, args: type=b200,master_clock_rate=23.04e6 [INFO] [UHD RF] RF UHD Generic instance constructed [INFO] [B200] Detected Device: B200mini [INFO] [B200] Operating over USB 3. [INFO] [B200] Initialize CODEC control... [INFO] [B200] Initialize Radio control... [INFO] [B200] Performing register loopback test... [INFO] [B200] Register loopback test passed [INFO] [B200] Asking for clock rate 23.040000 MHz... [INFO] [B200] Actually got clock rate 23.040000 MHz. Setting manual TX/RX offset to 300 samples Waiting PHY to initialize ... done! Attaching UE... RF status: O=1, U=0, L=0

Steps to reproduce the problem

I have already set the CPU governor to performance in both the gNB computer and the UE computer. I have checked if the overflow problem has to do with the CPU but it is free/inactive most of the time $mpstat Linux 6.5.0-28-generic (tueilkn-ue) 02.05.2024 _x8664 (4 CPU)

16:35:57 CPU %usr %nice %sys %iowait %irq %soft %steal %guest %gnice %idle 16:35:57 all 0,94 0,01 0,75 0,03 0,00 0,01 0,00 0,00 0,00 98,25

I have verify the USRP is correctly configured by running sudo ./benchmark_rate --rx_rate 23.04e6 --tx_rate 23.04e6 and this is the result I get: Benchmark rate summary: Num received samples: 236706097 Num dropped samples: 0 Num overruns detected: 0 Num transmitted samples: 236213640 Num sequence errors (Tx): 0 Num sequence errors (Rx): 0 Num underruns detected: 0 Num late commands: 0 Num timeouts (Tx): 0 Num timeouts (Rx): 0

Additional Information

The gNB and core are connected and seem to work fine

The PRACH detector will not meet the performance requirements with the configuration {Format 0, ZCZ 0, SCS 1.25kHz, Rx ports 1}. srsLog error - Unable to create log file "/tmp/gnb.log": Permission denied Lower PHY in quad executor mode.

--== srsRAN gNB (commit 2f90c8b60) ==--

Connecting to AMF on 10.162.149.143:38412 Available radio types: uhd. [INFO] [UHD] linux; GNU C++ version 11.2.0; Boost_107400; UHD_4.1.0.5-3 Making USRP object with args 'type=b200' [INFO] [LOGGING] Fastpath logging disabled at runtime. [INFO] [B200] Detected Device: B210 [INFO] [B200] Operating over USB 3. [INFO] [B200] Initialize CODEC control... [INFO] [B200] Initialize Radio control... [INFO] [B200] Performing register loopback test... [INFO] [B200] Register loopback test passed [INFO] [B200] Performing register loopback test... [INFO] [B200] Register loopback test passed [INFO] [B200] Setting master clock rate selection to 'automatic'. [INFO] [B200] Asking for clock rate 16.000000 MHz... [INFO] [B200] Actually got clock rate 16.000000 MHz. [INFO] [MULTI_USRP] Setting master clock rate selection to 'manual'. [INFO] [B200] Asking for clock rate 23.040000 MHz... [INFO] [B200] Actually got clock rate 23.040000 MHz. Cell pci=1, bw=20 MHz, dl_arfcn=368500 (n3), dl_freq=1842.5 MHz, dl_ssb_arfcn=368410, ul_freq=1747.5 MHz

==== gNodeB started === Type to view trace

pgawlowicz commented 4 months ago

Could you check CPU usage on the srsUE host PC? with htop. Could you run the srsran_performance on the srsUE host PC?

ruedagarcialaura commented 4 months ago

@pgawlowicz Thank you for your answer. If I run htop on the gNodeB host computer I get 1.77 1.72 1.62 load average values, and the process that is currently consuming the most resources is the gNB with CPU% 107. By running it on the UE host computer I get that the load average is 0.30 0.26 0.21 and the most consuming process has CPU% 30.

And I had already set the CPU governor to performance in the UE host computer both by doing echo "performance" | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor and by running the srsran_performance file.

pgawlowicz commented 4 months ago

hi @ruedagarcialaura, any updates on this issue?

ruedagarcialaura commented 4 months ago

Hi, I changed the GNB channel bandwith to 10 MHz and nof_prb to 52 and I don't get overflows or underflows most of the times. But it is still stuck in Attaching UE...
Here are my config files ue_rf1.conf.txt gnb_rf_b210_fdd.txt

And also I get this in the ue log everytime:

2024-05-13T09:33:21.847132 [PHY-SA ] [I] [ 0] SYNC: The SFN sync reached the maximum number of trials (100). Transitioning to IDLE... 2024-05-13T09:33:21.847139 [PHY-SA ] [I] [ 0] Cell Select: Could not synchronize SFN 2024-05-13T09:33:21.847234 [RRC-NR ] [E] Proc "Cell Selection" - Couldn't select new serving cell 2024-05-13T09:33:21.847235 [RRC-NR ] [I] Proc "Cell Selection" - Completed with failure. 2024-05-13T09:33:21.847241 [RRC-NR ] [W] Could not finish setup request. Deallocating dedicatedInfoNAS PDU

Here is the log: ue3.log

pgawlowicz commented 4 months ago

Could you try to tune TX and RX gains? It seems that the srsUE was not even able to detect the cell

ruedagarcialaura commented 4 months ago

I've been trying different gain combinations in steps of 5 for different channel bandwidths (5,10,15 and 20 MHz) and no successful results, still stuck in attaching ue... Any recommendation? Thank you in advance

pgawlowicz commented 4 months ago

could you try with an external clock?

ruedagarcialaura commented 4 months ago

I am trying with the external clock Leo Bdonar GPSDO but it doesn't seem to work. Here is my config and log of the ue. ue_external_clock.zip

pgawlowicz commented 4 months ago

could you share also gnb config?

ruedagarcialaura commented 4 months ago

gnb config file gnb_rf_b210_fdd.txt

pgawlowicz commented 4 months ago

could you try the default configs from the tutorial with 10mhz bandwidth?

ruedagarcialaura commented 4 months ago

tutorial_10Mhz_external_clock.zip Here it is, it doesn't seem to use the external clock source. But the leo bodnar gpsdo has the two leds on and it's connected to both usrps.

pgawlowicz commented 4 months ago

hi, you have to add clock parameter to your config:

ru_sdr: 
  clock: external
ruedagarcialaura commented 4 months ago

tutorial_10MHz_2.zip I changed it but it looks the results are the same

pgawlowicz commented 4 months ago

could you change sync to default. What is the spec of the UE PC (cpu, and memory)?

ruedagarcialaura commented 4 months ago

uecpu memoryUe tutorial_10MHz_3.zip I got the same result with sync: default

pgawlowicz commented 4 months ago

hmm, strange, the setup runs over the air or RF cables? did you try to tune TX/RX gains?

ruedagarcialaura commented 4 months ago

Over the air. I tried with cable one day and the UE didn't find the cell either. I also tried the ueransim and at least I know the core works fine.

And I have been trying different gain combinations but always get similar results of overflows and doesn't go further than attaching UE... is there any tip to tune the gains? should the UE tx gain be higher than the gnb gains or maybe the other way around?

And also do the srates necessarily need to be the same for both usrps or can the UE work in a lower srate?

Thank you for all your responses

xiongruoting commented 3 months ago

I also got this problem,,, following the tutorial but still got RF status overflow problem... O=21, U=0, L=0. seems that it is nothing to do with the GPS sync issue... was wondering if changing to a powerful PC works?

pgawlowicz commented 3 months ago

@ruedagarcialaura any update on this issue?

Pramodsmvdu commented 3 months ago

Hi all, I am seeing similar issue . any update ?

On my side ,I am trying on raspberry pi5 , which 20MHZ bandwidth. ( using USB3 link) with usrp b210 .. seeing issue like.. 2024-06-07T16:20:35.866653 [PHY ] [I] [ 4591] Detected overflow, trying to resync SFN 2024-06-07T16:20:35.866661 [PHY ] [W] [ 4591] SFN not yet synchronized, sending out-of-sync 2024-06-07T16:20:35.867576 [PHY ] [W] [ 4592] SFN not yet synchronized, sending out-of-sync

2024-06-07T16:20:35.853009 [RF ] [I] Overflow 2024-06-07T16:20:35.855587 [PHY ] [I] [ 242] SYNC: Waiting for PSS while trying to decode MIB (0/100) 2024-06-07T16:20:35.856773 [PHY ] [I] [ 243] SYNC: Waiting for PSS while trying to decode MIB (1/100) 2024-06-07T16:20:35.857782 [PHY ] [I] [ 244] SYNC: Waiting for PSS while trying to decode MIB (2/100) 2024-06-07T16:20:35.858770 [PHY ] [I] [ 245] SYNC: Waiting for PSS while trying to decode MIB (3/100) 2024-06-07T16:20:35.859804 [PHY ] [I] [ 246] SYNC: Waiting for PSS while trying to decode MIB (4/100) 2024-06-07T16:20:35.860771 [PHY ] [I] [ 247] SYNC: Waiting for PSS while trying to decode MIB (5/100) 2024-06-07T16:20:35.862165 [PHY ] [I] [ 248] SYNC: Waiting for PSS while trying to decode MIB (6/100)

Opening 1 channels in RF device=uhd with args=type=b200,clock=gpsdo,master_clock_rate=23.04e6 Supported RF device list: UHD file [INFO] [UHD] linux; GNU C++ version 13.2.0; Boost_108300; UHD_4.6.0.0+ds1-5.1build4 [INFO] [LOGGING] Fastpath logging disabled at runtime. Opening USRP channels=1, args: type=b200,master_clock_rate=23.04e6 [INFO] [UHD RF] RF UHD Generic instance constructed [INFO] [B200] Detected Device: B210 [INFO] [B200] Operating over USB 3. [INFO] [B200] Detecting internal GPSDO.... [INFO] [GPS] Found an internal GPSDO: LC_XO, Firmware Rev 1.101 [INFO] [B200] Initialize CODEC control... [INFO] [B200] Initialize Radio control... [INFO] [B200] Performing register loopback test... [INFO] [B200] Register loopback test passed [INFO] [B200] Performing register loopback test... [INFO] [B200] Register loopback test passed [INFO] [B200] Asking for clock rate 23.040000 MHz... [INFO] [B200] Actually got clock rate 23.040000 MHz. Setting USRP time to 1136082616.000000s [INFO] [MULTI_USRP] 1) catch time transition at pps edge [INFO] [MULTI_USRP] 2) set times next pps (synchronously)

Waiting PHY to initialize ... done! Attaching UE... . Found Cell: Mode=FDD, PCI=6, PRB=100, Ports=4, CP=Normal, CFO=0.2 KHz Found PLMN: Id=302720, TAC=25060 Could not find Home PLMN Id=310260, trying to connect to PLMN Id=302720 Random Access Transmission: seq=23, tti=4657, ra-rnti=0x8 [INFO] [UHD RF] Tx while waiting for EOB, timed out... 1.13608e+09 >= 0. Starting new burst... Random Access Transmission: seq=11, tti=4677, ra-rnti=0x8 Random Access Transmission: seq=25, tti=4697, ra-rnti=0x8 Random Access Transmission: seq=21, tti=4737, ra-rnti=0x8 Random Access Transmission: seq=9, tti=4757, ra-rnti=0x8 Random Access Transmission: seq=10, tti=4777, ra-rnti=0x8 Random Access Transmission: seq=6, tti=4797, ra-rnti=0x8 Random Access Transmission: seq=11, tti=4817, ra-rnti=0x8 RF status: O=31, U=10, L=286 RF status: O=42, U=11, L=490 RF status: O=57, U=6, L=482 RF status: O=38, U=8, L=654 RF status: O=31, U=20, L=655 RF status: O=55, U=4, L=568 RF status: O=43, U=19, L=541 RF status: O=37, U=10, L=615 RF status: O=48, U=10, L=468 RF status: O=45, U=16, L=538 RF status: O=46, U=10, L=640 RF status: O=45, U=6, L=579 RF status: O=58, U=8, L=508 RF status: O=42, U=12, L=697 RF status: O=31, U=4, L=324 Attach failed (attempt 1/5) Quit

it works fine with 5,10 and 15 MHZ. any recommendation ?

pgawlowicz commented 3 months ago

probably RPi is too slow for 20MHz, you could try to tune the CPU affinities (worker_cpu_mask and sync_cpu_affinity), see here: https://github.com/srsran/srsRAN_4G/blob/master/srsue/src/main.cc#L265-L270, but not sure if it will help.

also you might try to set the cpu governor


sudo systemctl disable ondemand
sudo apt install linux-tools-raspi

sudo nano /etc/default/cpufrequtils
* insert:
* GOVERNOR="performance"

## reboot

sudo cpupower frequency-info
* should show that the CPU is running in performance mode, at maxiumum clock speed
* ```
ruedagarcialaura commented 3 months ago

@pgawlowicz Instead of using a usrp B210 and a B200-mini I am now using two usrp B200-mini and it worked at the first try ;). Thank you for your help.

xiongruoting commented 1 month ago

Hello, I solved this problem by changing the TX/RX gain: 85 75.