search

lowRISC / opentitan

OpenTitan: Open source silicon root of trust
https://www.opentitan.org
Apache License 2.0
2.58k stars 777 forks source link

[test-triage] Timeouts in nightly regression #24425

Open alees24 opened 2 months ago

alees24 commented 2 months ago

Hierarchy of regression failure

Mostly chip level.

Failure Description

A number of the nightly regression tests show a timeout (wall clock time), mostly reporting a timeout after a run time of 60n minutes; the exception being an explicitly-specified timeout of 220m.

This appears to be an infrastructure problem and the issue has been created just to track their progress. It is anticipated that when the infrastructure change is completely properly these tests will be resurrected.

Steps to Reproduce

Tests with similar or related failures

chip_csr_aliasing chip_sw_alert_handler_reverse_ping_in_deep_sleep chip_sw_coremark chip_sw_exit_test_unlocked_bootstrap chip_sw_inject_scramble_seed hip_sw_keymgr_key_derivation chip_sw_keymgr_sideload_aes chip_sw_otp_ctrl_dai_lock chip_sw_rv_timer_systick_test flash_ctrl_error_prog_type flash_ctrl_phy_ack_consistency

HU90m commented 2 months ago

The following are additional chip level tests not listed in the issue description that are flaky due to timeouts. The timeouts were observed in https://reports.opentitan.org/hw/top_earlgrey/dv/2024.09.10_08.17.33/report.html