search

lopsided98 / nix-ros-overlay

ROS overlay for the Nix package manager
Apache License 2.0
173 stars 69 forks source link

CI is broken ? #353

Closed movefasta closed 3 months ago

movefasta commented 4 months ago

Hello!

About several months CI doesn't work on all distros because of

(for examble this action) Cached Failed Build JOB: Cache not found for input keys: failed-builds-rolling-x86_64-linux-c8afb864b67d2c77b4a523953be4cb73cb8f4d4a, failed-builds-rolling-x86_64-linux- Run ./.github/actions/nix-ros-build-action JOB: Warning: Failed to load failed builds cache: Error: ENOENT: no such file or directory, open 'failed-builds/failed-builds.json'.

@lopsided98 how we can fix it?

lopsided98 commented 4 months ago

That warning is normal with no cache (because none of the builds have succeeded recently). I suspect some number of packages are being built because it runs for a few hours before failing, but then something goes wrong. Build log output is not printed because it would produce an absurd amount of interleaved output. It is supposed to be dumped in an organized manner at the end, but something causes the action to fail before that. Even if all builds fail, the action is still supposed to succeed.

movefasta commented 4 months ago

That warning is normal with no cache (because none of the builds have succeeded recently). I suspect some number of packages are being built because it runs for a few hours before failing, but then something goes wrong. Build log output is not printed because it would produce an absurd amount of interleaved output. It is supposed to be dumped in an organized manner at the end, but something causes the action to fail before that. Even if all builds fail, the action is still supposed to succeed.

Thank you for explanation. it's very good that binary cache still uploads. But I was just alarmed that when the continuous integration system log constantly returns an error, then we may miss something really important in the future.

lopsided98 commented 4 months ago

I'm keeping this open because there is definitely something wrong with the CI. I just enabled debug logging so maybe that will give some hints.

lopsided98 commented 3 months ago

It is working correctly again as far as I can tell. It was running out of memory while evaluating and there was a bug uploading to cachix.