search

cnti-testcatalog / testsuite

📞📱☎️📡🌐 Cloud Native Telecom Initiative (CNTI) Test Catalog is a tool to check for and provide feedback on the use of K8s + cloud native best practices in networking applications and platforms
https://wiki.lfnetworking.org/display/LN/Test+Catalog
Apache License 2.0
169 stars 70 forks source link

[BUG] Unstable 5G tests(suci_enabled) in github actions #2072

Open martin-mat opened 3 weeks ago

martin-mat commented 3 weeks ago

Describe the bug 5G tests fail frequently in github actions. Example: https://github.com/cnti-testcatalog/testsuite/actions/runs/9439225077/job/25997107636

To Reproduce run github actions few times. Failures are quite frequent.

Expected behavior Tests should pass if there is no error/change

svteb commented 6 days ago

EDIT: This comment is entirely incorrect, the issue likely stems from timing issues with tshark (yet there is some interesting info)

The issue stems from ueransim not having all SUCI components working during the test:

From task suci_enabled in src/tasks/workload/5g_validator.cr:

if K8sTshark.regex_tshark_log(/"nas_5gs.mm.type_id": "1"/, tshark_log_name) &&
            !K8sTshark.regex_tshark_log(/"nas_5gs.mm.suci.scheme_id": "0"/, tshark_log_name) &&
            !K8sTshark.regex_tshark_log(/"nas_5gs.mm.suci.pki": "0"/, tshark_log_name)

During failed tests the values nas_5gs.mm.suci.scheme_id and nas_5gs.mm.suci.pki are equal to 0. This seems to happen because the CNF sample_open5gs does not actually deploy "successfully" although all of its components are running. In the file cnf-testsuite.yml which is part of the CNFs directory are key values to enable encryption:

protectionScheme: 1
publicKey: 0ac95ceeb93308df01be82ff9994d8330e38804ece1700ee4b972d8028796275
publicKeyId: 1

These sometimes don't register (for as of yet unknown reasons) and thus ueransim does not use them, as can be seen in the logs from actions:

Successful run:

I, [2024-06-09 23:07:08 +00:00 #432707]  INFO -- cnf-testsuite: ue_values: amf:
  hostname: open5gs-amf-ngap

mcc: '999'
mnc: '70'
sst: 1
sd: "0x111111"
tac: '0001'

protectionScheme: 1
publicKey: '0ac95ceeb93308df01be82ff9994d8330e38804ece1700ee4b972d8028796275'
publicKeyId: 1
routingIndicator: '0000'
# protectionScheme: 0
# publicKey:
# publicKeyId: 1
# routingIndicator: '0000'

Unsuccessful run:

I, [2024-06-09 20:25:19 +00:00 #360248]  INFO -- cnf-testsuite: ue_values: amf:
  hostname: open5gs-amf-ngap

mcc: '999'
mnc: '70'
sst: 1
sd: "0x111111"
tac: '0001'

# protectionScheme: 0
# publicKey:
# publicKeyId: 1
# routingIndicator: '0000'

Will investigate further.