adafruit / circuitpython

CircuitPython - a Python implementation for teaching coding with microcontrollers
https://circuitpython.org
Other
4.08k stars 1.21k forks source link

silabs builds intermittently fail in GitHub Actions CI #8603

Open dhalbert opened 11 months ago

dhalbert commented 11 months ago

@silabs-ChatNguyen @silabs-BelaV

Silabs port builds fail intermittently during GitHub Actions CI runs. Re-running the failed job usually succeeds.

Examples (which will disappear within 3 months, I think): https://github.com/adafruit/circuitpython/actions/runs/6778521612 https://github.com/adafruit/circuitpython/actions/runs/6723361060

This is not a showstopper, since re-running is easy.

We're wondering if there's a race condition in the silabs makefiles, such as some missing dependency. The jobs get run with -j2.

silabs-ChatNguyen commented 11 months ago

Hi, It seems the problem occurs when building boards with parallelism. I will check the makefiles again.

silabs-ChatNguyen commented 11 months ago

Silabs uses the SLC tool for generate code before build process. In 2 examples above, code generation failed on first build: image I don't understand why, but i can try to update new version of SLC tool.

dhalbert commented 11 months ago

I think someone commented recently that it might have to do with a race condition about signature generation (?) in the slc setup, though I cannot find that comment right now

silabs-ChatNguyen commented 11 months ago

I have a workaround for race condition issue. Run slc-generate with single core -j1. After slc-generate done, the Makefile continues to run with multiple core normally. https://github.com/adafruit/circuitpython/commit/0a6e92d001dbc416bacda09b5ff568dd1a407018

dhalbert commented 11 months ago

Sounds good; a simple PR would be fine. Later you can work with the slc folks (or is that you :) ) to see what is going on there.

jepler commented 10 months ago

I am not sure #8615 resolved the problem. A similar error occured in https://github.com/adafruit/circuitpython/actions/runs/6906615046/job/18792200464?pr=8628 which included the change from this PR:

devkit_xg24_brd2601b: Building languages: en_US, de_DE, en_GB, en_x_pirate, es, fil, fr, ID, it_IT, ja, nl, pl, pt_BR, ru, sv, tr, zh_Latn_pinyin
Error: Build devkit_xg24_brd2601b for en_US took 19.95s and failed
make: Entering directory '/home/runner/work/circuitpython/circuitpython/ports/silabs'
- Verbosity options: any combination of "steps commands rules", as `make V=...` or env var BUILD_VERBOSE
GEN build-devkit_xg24_brd2601b/genhdr/mpversion.h
make[1]: Entering directory '/home/runner/work/circuitpython/circuitpython/ports/silabs'
make[1]: warning: -j1 forced in submake: resetting jobserver mode.
- Verbosity options: any combination of "steps commands rules", as `make V=...` or env var BUILD_VERBOSE
SLC generates project
sdk was set to /home/runner/work/circuitpython/circuitpython/ports/silabs/gecko_sdk
The extension /home/runner/work/circuitpython/circuitpython/ports/silabs/cp_efr32_extension/cp_efr32.slce is now trusted.
The sdk /home/runner/work/circuitpython/circuitpython/ports/silabs/gecko_sdk/ is now trusted. 
Slcp parse warnings for /home/runner/work/circuitpython/circuitpython/ports/silabs/circuitpython_efr32.slcp
    Referenced SDK Extension 'cp_efr32:1.0.0' could not be found.
Warnings loading project: 
 - Referenced SDK Extension 'cp_efr32:1.0.0' could not be found.
Found valid toolchain component toolchain_gcc for gcc
Generation failed: /home/runner/work/circuitpython/circuitpython/ports/silabs/circuitpython_efr32.slcp contains parse issues. As of .slcs spec version 8, projects must not contain any warnings to generate.
To override this behaviour, opt in using --force.
make[1]: *** [Makefile:173: slc-generate] Error 1
make[1]: Leaving directory '/home/runner/work/circuitpython/circuitpython/ports/silabs'
make: *** [Makefile:142: build-devkit_xg24_brd2601b/circuitpython_efr32.Makefile] Error 2
make: Leaving directory '/home/runner/work/circuitpython/circuitpython/ports/silabs'