srsran / srsRAN_Project

Open source O-RAN 5G CU/DU solution from Software Radio Systems (SRS) https://docs.srsran.com/projects/project
https://www.srsran.com
GNU Affero General Public License v3.0
476 stars 163 forks source link

Missing UP PRACH messages + late wake-up #639

Open melkael opened 3 months ago

melkael commented 3 months ago

Hello!

Issue Description

I managed to run srsRAN with RAN550 on a beefy xeon server fine. I now want to run it on a smaller PC but I keep getting timing issues in the flavor of (see the attached logs):

Real-time timing worker woke up late, sleep time has been '1000us', or equivalently, '28' symbols Missed incoming User-Plane PRACH messages

The UE also does not attach in this setup but it did on the xeon server

Setup Details

Expected Behavior

The UE should attach

Actual Behaviour

The UE doesnt attach but the RU receives all the packets on time. However the DU does not seem to receive anything

2024-05-29T20:24:54.251883 [OFH ] [I] Opened successfully the NIC interface 'enp23s0f1np1' used by the Ethernet receiver 2024-05-29T20:24:54.251889 [OFH ] [I] Opened successfully the NIC interface 'enp23s0f1np1' used by the Ethernet transmitter 2024-05-29T20:24:54.341216 [OFH ] [I] Starting the operation of the Open Fronthaul interface 2024-05-29T20:24:54.341217 [OFH ] [I] Starting the realtime timing worker 2024-05-29T20:24:54.341225 [OFH ] [I] Started the realtime timing worker 2024-05-29T20:24:54.341227 [OFH ] [I] Starting the ethernet frame receiver 2024-05-29T20:24:54.341244 [OFH ] [I] Started the ethernet frame receiver 2024-05-29T20:24:54.341244 [OFH ] [I] Started the operation of the Open Fronthaul interface 2024-05-29T20:24:54.357017 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '450.19' and sector#0 2024-05-29T20:24:54.367022 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '451.19' and sector#0 2024-05-29T20:24:54.377026 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '452.19' and sector#0 2024-05-29T20:24:54.387013 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '453.19' and sector#0 2024-05-29T20:24:54.397022 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '454.19' and sector#0 2024-05-29T20:24:54.407023 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '455.19' and sector#0 2024-05-29T20:24:54.417016 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '456.19' and sector#0 2024-05-29T20:24:54.427024 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '457.19' and sector#0 2024-05-29T20:24:54.437018 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '458.19' and sector#0 2024-05-29T20:24:54.447016 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '459.19' and sector#0 2024-05-29T20:24:54.457018 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '460.19' and sector#0 2024-05-29T20:24:54.467023 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '461.19' and sector#0 2024-05-29T20:24:54.477021 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '462.19' and sector#0 2024-05-29T20:24:54.487025 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '463.19' and sector#0 2024-05-29T20:24:54.497025 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '464.19' and sector#0 2024-05-29T20:24:54.507021 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '465.19' and sector#0 2024-05-29T20:24:54.517019 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '466.19' and sector#0 2024-05-29T20:24:54.527022 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '467.19' and sector#0 2024-05-29T20:24:54.537020 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '468.19' and sector#0 2024-05-29T20:24:54.547016 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '469.19' and sector#0 2024-05-29T20:24:54.557018 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '470.19' and sector#0 2024-05-29T20:24:54.567014 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '471.19' and sector#0 2024-05-29T20:24:54.577015 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '472.19' and sector#0 2024-05-29T20:24:54.587022 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '473.19' and sector#0 2024-05-29T20:24:54.597016 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '474.19' and sector#0 2024-05-29T20:24:54.607021 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '475.19' and sector#0 2024-05-29T20:24:54.617014 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '476.19' and sector#0 2024-05-29T20:24:54.627020 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '477.19' and sector#0 2024-05-29T20:24:54.637018 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '478.19' and sector#0 2024-05-29T20:24:54.647013 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '479.19' and sector#0 2024-05-29T20:24:54.657021 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '480.19' and sector#0 2024-05-29T20:24:54.667016 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '481.19' and sector#0 2024-05-29T20:24:54.677025 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '482.19' and sector#0 2024-05-29T20:24:54.687025 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '483.19' and sector#0 2024-05-29T20:24:54.697015 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '484.19' and sector#0 2024-05-29T20:24:54.707018 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '485.19' and sector#0 2024-05-29T20:24:54.717020 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '486.19' and sector#0 2024-05-29T20:24:54.727019 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '487.19' and sector#0 2024-05-29T20:24:54.737017 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '488.19' and sector#0 2024-05-29T20:24:54.747021 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '489.19' and sector#0 2024-05-29T20:24:54.757017 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '490.19' and sector#0 2024-05-29T20:24:54.767022 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '491.19' and sector#0 2024-05-29T20:24:54.777026 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '492.19' and sector#0 2024-05-29T20:24:54.787025 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '493.19' and sector#0 2024-05-29T20:24:54.797018 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '494.19' and sector#0 2024-05-29T20:24:54.807016 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '495.19' and sector#0 2024-05-29T20:24:54.817016 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '496.19' and sector#0 2024-05-29T20:24:54.827016 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '497.19' and sector#0 2024-05-29T20:24:54.837027 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '498.19' and sector#0 2024-05-29T20:24:54.847019 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '499.19' and sector#0 2024-05-29T20:24:54.857020 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '500.19' and sector#0 2024-05-29T20:24:54.867017 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '501.19' and sector#0 2024-05-29T20:24:54.877023 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '502.19' and sector#0 2024-05-29T20:24:54.887025 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '503.19' and sector#0 2024-05-29T20:24:54.897019 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '504.19' and sector#0 2024-05-29T20:24:54.907016 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '505.19' and sector#0 2024-05-29T20:24:54.917025 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '506.19' and sector#0 2024-05-29T20:24:54.927019 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '507.19' and sector#0 2024-05-29T20:24:54.937017 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '508.19' and sector#0 2024-05-29T20:24:54.947019 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '509.19' and sector#0 2024-05-29T20:24:54.957020 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '510.19' and sector#0 2024-05-29T20:24:54.967021 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '511.19' and sector#0 2024-05-29T20:24:54.977023 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '512.19' and sector#0 2024-05-29T20:24:54.987025 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '513.19' and sector#0 2024-05-29T20:24:54.997022 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '514.19' and sector#0 2024-05-29T20:24:55.002228 [OFH ] [I] Real-time timing worker woke up late, skipped '28' symbols 2024-05-29T20:24:55.002229 [OFH ] [W] Real-time timing worker woke up late, sleep time has been '1000us', or equivalently, '28' symbols 2024-05-29T20:24:55.007021 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '515.19' and sector#0 2024-05-29T20:24:55.017017 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '516.19' and sector#0 2024-05-29T20:24:55.027018 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '517.19' and sector#0 2024-05-29T20:24:55.037026 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '518.19' and sector#0 2024-05-29T20:24:55.047019 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '519.19' and sector#0 2024-05-29T20:24:55.057018 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '520.19' and sector#0 2024-05-29T20:24:55.067022 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '521.19' and sector#0 2024-05-29T20:24:55.077018 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '522.19' and sector#0 2024-05-29T20:24:55.087015 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '523.19' and sector#0 2024-05-29T20:24:55.097024 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '524.19' and sector#0 2024-05-29T20:24:55.107017 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '525.19' and sector#0 2024-05-29T20:24:55.117019 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '526.19' and sector#0 2024-05-29T20:24:55.127017 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '527.19' and sector#0 2024-05-29T20:24:55.137025 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '528.19' and sector#0 2024-05-29T20:24:55.147026 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '529.19' and sector#0 2024-05-29T20:24:55.157021 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '530.19' and sector#0 2024-05-29T20:24:55.167020 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '531.19' and sector#0 2024-05-29T20:24:55.177016 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '532.19' and sector#0 2024-05-29T20:24:55.187013 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '533.19' and sector#0 2024-05-29T20:24:55.197024 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '534.19' and sector#0 2024-05-29T20:24:55.207026 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '535.19' and sector#0 2024-05-29T20:24:55.217019 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '536.19' and sector#0 2024-05-29T20:24:55.227021 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '537.19' and sector#0 2024-05-29T20:24:55.237022 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '538.19' and sector#0 2024-05-29T20:24:55.247014 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '539.19' and sector#0 2024-05-29T20:24:55.257018 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '540.19' and sector#0 2024-05-29T20:24:55.267015 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '541.19' and sector#0 2024-05-29T20:24:55.277016 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '542.19' and sector#0 2024-05-29T20:24:55.287022 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '543.19' and sector#0 2024-05-29T20:24:55.297025 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '544.19' and sector#0 2024-05-29T20:24:55.307026 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '545.19' and sector#0 2024-05-29T20:24:55.317014 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '546.19' and sector#0 2024-05-29T20:24:55.327021 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '547.19' and sector#0 2024-05-29T20:24:55.337018 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '548.19' and sector#0 2024-05-29T20:24:55.341215 [OFH ] [I] Received packets: rx_total=0 rx_early=0, rx_on_time=0, rx_late=0 2024-05-29T20:24:55.347023 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '549.19' and sector#0 2024-05-29T20:24:55.357022 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '550.19' and sector#0 2024-05-29T20:24:55.367025 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '551.19' and sector#0 2024-05-29T20:24:55.377013 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '552.19' and sector#0 2024-05-29T20:24:55.387017 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '553.19' and sector#0 2024-05-29T20:24:55.397022 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '554.19' and sector#0 2024-05-29T20:24:55.407018 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '555.19' and sector#0 2024-05-29T20:24:55.417023 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '556.19' and sector#0 2024-05-29T20:24:55.427020 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '557.19' and sector#0 2024-05-29T20:24:55.437017 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '558.19' and sector#0 2024-05-29T20:24:55.447022 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '559.19' and sector#0 2024-05-29T20:24:55.457024 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '560.19' and sector#0 2024-05-29T20:24:55.467017 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '561.19' and sector#0 2024-05-29T20:24:55.477018 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '562.19' and sector#0 2024-05-29T20:24:55.487016 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '563.19' and sector#0 2024-05-29T20:24:55.497015 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '564.19' and sector#0 2024-05-29T20:24:55.507016 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '565.19' and sector#0 2024-05-29T20:24:55.517015 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '566.19' and sector#0 2024-05-29T20:24:55.527021 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '567.19' and sector#0 2024-05-29T20:24:55.537024 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '568.19' and sector#0 2024-05-29T20:24:55.547017 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '569.19' and sector#0 2024-05-29T20:24:55.557022 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '570.19' and sector#0 2024-05-29T20:24:55.567024 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '571.19' and sector#0 2024-05-29T20:24:55.577015 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '572.19' and sector#0 2024-05-29T20:24:55.587020 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '573.19' and sector#0 2024-05-29T20:24:55.597016 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '574.19' and sector#0 2024-05-29T20:24:55.607012 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '575.19' and sector#0 2024-05-29T20:24:55.617020 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '576.19' and sector#0 2024-05-29T20:24:55.627019 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '577.19' and sector#0 2024-05-29T20:24:55.637015 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '578.19' and sector#0 2024-05-29T20:24:55.647016 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '579.19' and sector#0 2024-05-29T20:24:55.657015 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '580.19' and sector#0 2024-05-29T20:24:55.667021 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '581.19' and sector#0 2024-05-29T20:24:55.677017 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '582.19' and sector#0 2024-05-29T20:24:55.687020 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '583.19' and sector#0 2024-05-29T20:24:55.697014 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '584.19' and sector#0 2024-05-29T20:24:55.707017 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '585.19' and sector#0 2024-05-29T20:24:55.717025 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '586.19' and sector#0 2024-05-29T20:24:55.727023 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '587.19' and sector#0 2024-05-29T20:24:55.737014 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '588.19' and sector#0 2024-05-29T20:24:55.742093 [OFH ] [I] Stopping the operation of the Open Fronthaul interface 2024-05-29T20:24:55.742094 [OFH ] [I] Requesting stop of the realtime timing worker 2024-05-29T20:24:55.743148 [OFH ] [I] Stopped the realtime timing worker 2024-05-29T20:24:55.743149 [OFH ] [I] Requesting stop of the ethernet frame receiver 2024-05-29T20:24:55.843203 [OFH ] [I] Stopped the ethernet frame receiver 2024-05-29T20:24:55.843203 [OFH ] [I] Stopped the operation of the Open Fronthaul interface

Steps to reproduce the problem

Here is the config file I am using (same as the xeon server, baring the big server uses a remote core while it is local in that setup)

amf: addr: 127.0.0.5 # The address or hostname of the AMF. bind_addr: 127.0.0.5 #172.20.25.67 # A local IP that the gNB binds to for traffic from the>

ru_ofh: t1a_max_cp_dl: 500 t1a_min_cp_dl: 258 t1a_max_cp_ul: 465 t1a_min_cp_ul: 285 t1a_max_up: 250 t1a_min_up: 50 ta4_max: 200 ta4_min: 0 is_prach_cp_enabled: true is_dl_broadcast_enabled: true compr_method_ul: bfp compr_bitwidth_ul: 9 compr_method_dl: bfp compr_bitwidth_dl: 9 compr_method_prach: bfp compr_bitwidth_prach: 9 enable_ul_static_compr_hdr: false enable_dl_static_compr_hdr: false iq_scaling: 4.5 cells:

  • network_interface: enp23s0f1np1 # Ethernet interface name used to communicate with the R> ru_mac_addr: 70:b3:d5:e1:5d:ca # RU MAC address. du_mac_addr: 50:7c:6f:55:d2:c9 # DU MAC address. vlan_tag_cp: 2 # VLAN tag value for C-Plane. vlan_tag_up: 2 # VLAN tag value for U-Plane. prach_port_id: [4, 5] # PRACH eAxC port value. dl_port_id: [0, 1] # Downlink eAxC port values. ul_port_id: [0, 1] # Uplink eAxC port values.

cell_cfg: dl_arfcn: 640000 band: 78 # The NR band. channel_bandwidth_MHz: 20 # Bandwith in MHz. Number of PRBs will be automatically derived. common_scs: 30 # Subcarrier spacing in kHz used for data. plmn: "00101" # PLMN broadcasted by the gNB. tac: 1 # Tracking area code (needs to match the core configuration). pci: 1 # Physical cell ID. nof_antennas_dl: 2 nof_antennas_ul: 2 prach: prach_config_index: 159 prach_root_sequence_index: 1 zero_correlation_zone: 0 prach_frequency_start: 2 tdd_ul_dl_cfg: dl_ul_tx_period: 10 # Optional INT (10). Sets the TDD pattern periodicity in slots. The combination of this value and the chosen numerology must lead to a TDD periodicity of 0.5, > nof_dl_slots: 7 # Optional INT (6). Number of consecutive full Downlink slots. Supported: [0-80]. nof_dl_symbols: 6 # Optional INT (0). Number of Downlink symbols at the beginning of the slot following full Downlink slots. Supported: [0-13]. nof_ul_slots: 2 # Optional INT (3). Number of consecutive full Uplink slots. Supported: [0 - 80]. nof_ul_symbols: 4 ssb: ssb_period: 20 pdsch: mcs_table: qam256

log: filename: log.log # Path of the log file. all_level: warning ofh_level: info

Additional Information

i used the srs_performance script

andrepuschmann commented 3 months ago

No traffic is coming back from the RU so the DU complains. Double check MAC addresses, VLAN, settings, etc.

melkael commented 3 months ago

Thank you, I fixed the MAC but still am getting some of the error messages and can't attach a UE (or even see in the core logs that there is a registration attenmpt). Here is the log:

2024-05-29T21:40:20.882960 [OFH ] [I] Opened successfully the NIC interface 'enp23s0f1np1' used by the Ethernet receiver 2024-05-29T21:40:20.882966 [OFH ] [I] Opened successfully the NIC interface 'enp23s0f1np1' used by the Ethernet transmitter 2024-05-29T21:40:20.972919 [OFH ] [I] Starting the operation of the Open Fronthaul interface 2024-05-29T21:40:20.972921 [OFH ] [I] Starting the realtime timing worker 2024-05-29T21:40:20.972928 [OFH ] [I] Started the realtime timing worker 2024-05-29T21:40:20.972931 [OFH ] [I] Starting the ethernet frame receiver 2024-05-29T21:40:20.972946 [OFH ] [I] Started the ethernet frame receiver 2024-05-29T21:40:20.972946 [OFH ] [I] Started the operation of the Open Fronthaul interface 2024-05-29T21:40:21.002228 [OFH ] [I] Real-time timing worker woke up late, skipped '28' symbols 2024-05-29T21:40:21.002228 [OFH ] [W] Real-time timing worker woke up late, sleep time has been '1000us', or equivalently, '28' symbols 2024-05-29T21:40:21.972934 [OFH ] [I] Received packets: rx_total=2400 rx_early=0, rx_on_time=2338, rx_late=62 2024-05-29T21:40:22.002200 [OFH ] [I] Real-time timing worker woke up late, skipped '28' symbols 2024-05-29T21:40:22.002201 [OFH ] [W] Real-time timing worker woke up late, sleep time has been '1000us', or equivalently, '28' symbols 2024-05-29T21:40:22.972930 [OFH ] [I] Received packets: rx_total=2400 rx_early=0, rx_on_time=2318, rx_late=82 2024-05-29T21:40:23.002200 [OFH ] [I] Real-time timing worker woke up late, skipped '28' symbols 2024-05-29T21:40:23.002201 [OFH ] [W] Real-time timing worker woke up late, sleep time has been '1000us', or equivalently, '28' symbols 2024-05-29T21:40:23.972932 [OFH ] [I] Received packets: rx_total=2400 rx_early=0, rx_on_time=2351, rx_late=49 2024-05-29T21:40:24.002198 [OFH ] [I] Real-time timing worker woke up late, skipped '28' symbols 2024-05-29T21:40:24.002199 [OFH ] [W] Real-time timing worker woke up late, sleep time has been '1000us', or equivalently, '28' symbols 2024-05-29T21:40:24.972932 [OFH ] [I] Received packets: rx_total=2400 rx_early=0, rx_on_time=2351, rx_late=49 2024-05-29T21:40:25.002190 [OFH ] [I] Real-time timing worker woke up late, skipped '28' symbols 2024-05-29T21:40:25.002191 [OFH ] [W] Real-time timing worker woke up late, sleep time has been '1000us', or equivalently, '28' symbols 2024-05-29T21:40:25.375340 [OFH ] [I] Stopping the operation of the Open Fronthaul interface 2024-05-29T21:40:25.375341 [OFH ] [I] Requesting stop of the realtime timing worker 2024-05-29T21:40:25.376395 [OFH ] [I] Stopped the realtime timing worker 2024-05-29T21:40:25.376396 [OFH ] [I] Requesting stop of the ethernet frame receiver 2024-05-29T21:40:25.476451 [OFH ] [I] Stopped the ethernet frame receiver 2024-05-29T21:40:25.476451 [OFH ] [I] Stopped the operation of the Open Fronthaul interface

andrepuschmann commented 3 months ago

You have a few lates, but not too many. This can be solved with thread affinities and/or use of DPDK. Please check KPI on the RU as well and check if it's emitting a signal.

melkael commented 3 months ago

I checked the kpi.sh and i have a few RX_late (between 34 and 37). I tried various configs for the CPU pinning without much success. I verified with Network Signal Guru and I can't see the SIB. Which of the pinning parameters typically help in those cases?

ibrahimshikdaher commented 3 weeks ago

Hi @melkael , did you manage to solve the Issue? I am having something similar.

waddem01 commented 1 week ago

I am wondering if @melkael or @ibrahimshikdaher have been able to make any progress with this?

I am seeing exactly the same behaviour on a very similar system (i9-10900 2.8Ghz / 10 cores. Ubuntu 22.04, 6.8 kernel, Local Open5Gs Core, srsRAN 24.04.0, RAN650-1v1.0.4-dda1bf5, FibroLAN Falcon-RX/812/G, E810 O-DU NIC)

gnb.log fills with these messages:

2024-09-11T15:02:01.807015 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '91.19' and sector#0 2024-09-11T15:02:01.817014 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '92.19' and sector#0 2024-09-11T15:02:01.827017 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '93.19' and sector#0 2024-09-11T15:02:01.837011 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '94.19' and sector#0 2024-09-11T15:02:01.847018 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '95.19' and sector#0 2024-09-11T15:02:01.857024 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '96.19' and sector#0 2024-09-11T15:02:01.867019 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '97.19' and sector#0 2024-09-11T15:02:01.877013 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '98.19' and sector#0 2024-09-11T15:02:01.887022 [OFH ] [W] Missed incoming User-Plane PRACH messages for slot '99.19' and sector#0

I am seeing the odd rumble of RX_LATE on the kpi.sh output, but not too many ( up to 10 or so).

Things improved when I changed the option on phc2sys on the O-DU: phc2sys -s enp2s0f0np0 -c CLOCK_REALTIME -w -m -R 8 -n 24 -f /etc/linuxptp/default.cfg

The corresponding ptp4l service runs:: ptp4l -2 -f /etc/linuxptp/default.cfg -m -I enp2s0f0np0

default.cfg is the srsRAN configuration file (https://docs.srsran.com/projects/project/en/latest/_downloads/76d8d0c734c4c4652b38a3032dc56d7b/default.cfg) but I have to comment out quite a few parameters before ptp4l will accept this

clientOnly 0

delay_response_timeout 0

serverOnly

power_profile.2011.networkTimeInaccuracy 0

power_profile.version none

clock_class_threshold 248

refclock_sock_address /var/run/refclock.ptp.sock

uds_file_mode 0660

uds_ro_address /var/run/ptp4lro

uds_ro_file_mode 0666

phc_index -1

I switched from the 100MHz 4x2 modes and I am now running the 20MHz 2x2 mode. The RAN650 produces RF, but UEs do not attach.

I have this in my gnb.yml:

gnb_id: 1 amf: addr: 127.0.0.5 # The address or hostname of the AMF. bind_addr: 127.0.0.1 # A local IP that the gNB binds to for traffic from the AMF (local Open5GS core).

ru_ofh: t1a_max_cp_dl: 500 # Maximum T1a on Control-Plane for Downlink in microseconds. t1a_min_cp_dl: 450 # Minimum T1a on Control-Plane for Downlink in microseconds. t1a_max_cp_ul: 350 # Maximum T1a on Control-Plane for Uplink in microseconds. t1a_min_cp_ul: 290 # Minimum T1a on Control-Plane for Uplink in microseconds. t1a_max_up: 360 # Maximum T1a on User-Plane in microseconds. t1a_min_up: 300 # Minimum T1a on User-Plane in microseconds. ta4_max: 200 # Maximum Ta4 on User-Plane in microseconds. ta4_min: 0 # Minimum Ta4 on User-Plane in microseconds. is_prach_cp_enabled: true # Configures if Control-Plane messages should be used to receive PRACH messages. compr_method_ul: bfp # Uplink compression method. compr_bitwidth_ul: 9 # Uplink IQ samples bitwidth after compression. compr_method_dl: bfp # Downlink compression method. compr_bitwidth_dl: 9 # Downlink IQ samples bitwidth after compression. compr_method_prach: bfp # PRACH compression method. compr_bitwidth_prach: 9 # PRACH IQ samples bitwidth after compression. enable_ul_static_compr_hdr: false # Configures if the compression header is present for uplink User-Plane messages (false) or not present (true). enable_dl_static_compr_hdr: false # Configures if the compression header is present for downlink User-Plane messages (false) or not present (true). iq_scaling: 4.5 # IQ samples scaling factor applied before compression, should be a positive value smaller than 10. cells: network_interface: enp2s0f0np0 # Ethernet interface name used to communicate with the RU. ru_mac_addr: 70:b3:d5:e1:5e:0f # RU MAC address. du_mac_addr: 30:3e:a7:05:62:64 # DU MAC address. vlan_tag_cp: 3
vlan_tag_up: 3 prach_port_id: [4,5] # PRACH eAxC port value. dl_port_id: [0,1] # Downlink eAxC port values. ul_port_id: [0,1] # Uplink eAxC port values.

cell_cfg: dl_arfcn: 668000 # ARFCN of the downlink carrier (center frequency). frequency is 4020,000 KHz band: 77 # The NR band. channel_bandwidth_MHz: 20 # Bandwith in MHz. Number of PRBs will be automatically derived. common_scs: 30 # Subcarrier spacing in kHz used for data. plmn: "99999" # PLMN broadcasted by the gNB. tac: 7 # Tracking area code (needs to match the core configuration). pci: 1 # Physical cell ID. nof_antennas_dl: 2 nof_antennas_ul: 2 prach: prach_config_index: 159 # PRACH configuration index. prach_root_sequence_index: 1 # PRACH root sequence index. zero_correlation_zone: 0 # Zero correlation zone. prach_frequency_start: 2 # Offset in PRBs of lowest PRACH transmission occasion in frequency domain respective to PRB 0. preamble_trans_max: 50 power_ramping_step_db: 4 tdd_ul_dl_cfg: dl_ul_tx_period: 10 # Optional INT (10). Sets the TDD pattern periodicity in slots. The combination of this value and the chosen numerology must lead to a TDD periodicity of 0.5, > nof_dl_slots: 7 # Optional INT (6). Number of consecutive full Downlink slots. Supported: [0-80]. nof_dl_symbols: 6 # Optional INT (0). Number of Downlink symbols at the beginning of the slot following full Downlink slots. Supported: [0-13]. nof_ul_slots: 2 # Optional INT (3). Number of consecutive full Uplink slots. Supported: [0 - 80]. nof_ul_symbols: 4 ssb: ssb_block_power_dbm: -12 # For 15dBm output power, this would be -12dBm (20MHz carrier)

pdsch: mcs_table: qam256 pusch: p0_nominal_with_grant: -76 log: filename: /var/log/srsRAN/gnb.log # Path of the log file. du_level: debug # Debug DU to understand why UE is not attaching

all_level: warning

pcap: mac_enable: false # Set to true to enable MAC-layer PCAPs. mac_filename: /tmp/gnb_mac.pcap # Path where the MAC PCAP is stored. ngap_enable: false # Set to true to enable NGAP PCAPs. ngap_filename: /tmp/gnb_ngap.pcap # Path where the NGAP PCAP is stored.

andrepuschmann commented 6 days ago

@waddem01 The issue you're having is that the DU doesn't get any PRACH uplane back from the RU. So there is clearly a misconfiguration. Double check MACs, eAxC values, etc.

@melkael @ibrahimshikdaher Have you tried with a spectrum analyzer/USRP to see if the RU is emitting any signal at all?