Build checks fail if multiple (versions of) crates are deployed at once.

[Jellix]

Summary of the issues that might be lingering there:

• PR build checks generally seem to get confused when multiple crates are involved:
  • This seems to result in the behaviour that for the final result only the last build is taken into account.
  • Hence in a scenario where a PR changes crate A and B, building of crate A fails, but B is being built successfully after, the final check result may be considered successful.
• PR build checks seem to build the latest version of a crate, possibly regardless of the version of the crate that changed.

If you compare these runs:

https://github.com/alire-project/alire-index/runs/1270111342?check_suite_focus=true (failed) https://github.com/alire-project/alire-index/runs/1270157219?check_suite_focus=true (success)

The first run ends with:

stderr: ADA.IO_EXCEPTIONS.USE_ERROR
stderr: new name "alire/alire.toml.upstream" designates a file that already exists
stderr: raised ADA.IO_EXCEPTIONS.USE_ERROR : new name "alire/alire.toml.upstream" designates a file that already exists
Call stack traceback locations:
0x5644729b977a 0x5644726e6d43 0x564472872098 0x5644728778c0 0x564472830e79 0x564472830121 0x56447224db6f 0x56447225054b 0x7f1081a89099 0x56447224d748 0xfffffffffffffffe

ERROR at line 113 (code: 1)

From the logs you see there are two changed crate files (both new, just different versions):

Changed files: index/ai/aicwl/aicwl-3.24.0.toml index/ai/aicwl/aicwl-3.24.1.toml

but it seems then that alr will always try to build the latest version of the crate. That is probably intended for the general use case, I still think it should build the exact crate that has been specified, one could imagine a scenario where an existing crate will be modified. Or maybe alr just got utterly confused, because it was the same crate with two different versions.

If I understand the log correctly, the result of the first failing run (search for USE_ERROR) does not immediately lead to a failing build, alire continues after the first failure, and only the second run leads to the build failing.

Which could indicate that even in a scenario where multiple different crates are tested at once (which seems a more likely use case than my attempt here), only the result of the last build counts.

[mosteo]

Thanks for the report, Vincent.

It's true that only the last release for a crate is built. This was a limitation of the previous format which made much harder to identify the releases being submitted. I should fix it so the precise releases being changed are built.

On the issue of only the last check mattering, I'd be surprised about that, but I'll double check.

Finally, for submissions that are not rebased against the tip of the branch, things can get bonkers because many files are reported as changed that have nothing to do with the specific crate being submitted. I'm not sure at this time if there's an easy way around that other than rebasing the PR and force-pushing it. It's possible some git-fu could do the trick here but I haven't had the time to explore this issue in depth yet.

[Jellix]

On the issue of only the last check mattering, I'd be surprised about that, but I'll double check.

I'm not sure, either, maybe the single results somehow get collected into a final result.

I was just inferring from the log:

In line 2954 .. 2963 you see the first deployment fail.

Just to see in line 3009 the second attempt starting.

I would have expected the whole thing to abort after the first fail already.

[mosteo]

Thanks for the pointer, that will save me some digging. I may have missed a -o errexit or some pipe check.