alire-project / alire

Command-line tool from the Alire project and supporting library
GNU General Public License v3.0
288 stars 49 forks source link

post-fetch action doesn't run when enclosed by `[[actions."case(distribution)"."..."]]` #1544

Closed dalybrown closed 8 months ago

dalybrown commented 9 months ago

This may not be an Alire issue and may be an issue with the crate: I couldn't tell by the documentation whether this case statement is supported for actions or not.

In the xmlada crate, there is a post-fetch action which is enclosed by [[actions."case(distribution)"."..."]]. This action does not run on deployment of the crate (whether as a dependency of another crate or by deploying the crate itself) nor does it run when alr update is executed. The crate must be built for this action to run. See: https://github.com/alire-project/alire-index/blob/stable-1.2.1/index/xm/xmlada/xmlada-24.0.0.toml#L23.

This is problematic because it is a dependency of many crates within the ecosystem. Consequently commands can't be executed by alire on this crate unless the crate is built. This causes issues in CI pipelines when we run jobs in parallel to the build (and we don't want to always have to build crates prior to running every job).

According to the docs, post-fetch actions should be run on deployment of the crate and on updates to the environment. This doesn't happen with this crate.

dalybrown commented 9 months ago

A bite more testing and it seems like post-fetch actions are run during builds when I don't expect them to be - this might be another issue.

mosteo commented 9 months ago

What alr version are you using? Behaviors have slightly changed after 2.0, and the documentation may be obsolete. With 2.0, post-fetch will run once per unique build of a release, the first time its sources are synced.

(Actions do understand conditional expressions.)

dalybrown commented 9 months ago

I am using a version after 2.0.

If the action only runs on a build, then it's not really a post-fetch action then. There should be some actions that are only run on deployment of the crate (e.g., generating source files automatically).

Sometimes we want to run commands on a crate without having to build the whole crate but still require files to be generated. If post-fetch isn't the action to accomplish that, I think a new one should be added or the behaviour reverted!

mosteo commented 9 months ago

Actually, post-fetch in 2.0 should work as you describe, but bearing in mind that the action is not run on the pristine copy kept after download (<cache>/releases), but on the one synced for build (<cache>/builds). That is, the location reported by alr printenv. Your cache location is given by alr version.

In case it helps with your workflow, you can force the running of an action in isolation with alr action.

dalybrown commented 9 months ago

Thanks for the reply. We do force the action at the moment to work around it.

mosteo commented 9 months ago

OK, I've checked the current implementation details, and things are working as intended but with the consequences you have seen, so let's see why and what would help you.

With shared builds, we cannot generate the release build dirs until the full configuration is known. But, a full configuration is only required at build time, for reasons related to configurations without defaults. For that reason, post-fetch now lazily runs only once, just before a build, because it is the only time at which we are sure everything is properly configured and the action can always succeed. So, currently, the only difference between post-fetch and pre-build is that the latter is run every time, and post-fetch only once.

alr update is removing the post-fetch-completed flag, so it is indeed run again in the next build, but that doesn't help you.

I understand from your post that you'd like to run post-fetch on everything but without starting a build. The way to achieve that now is manually doing an alr action --recursive post-fetch, but that is suboptimal because it will redo post-fetch steps every time, instead of only once.

Would it work for you something like alr build --prepare that stopped after post-fetch? I'm not sure about the implications of the jobs in parallel with the build that you mention.

dalybrown commented 9 months ago

That seems reasonable ... we basically want some hook to run actions once that don't need to be executed again (e.g., code generation within a dependency).

I'm not so concerned about the semantics of it. The workaround at the moment forcing the actions is sufficient, but I suspect others will have a similar use case where they want to run actions only once upon deployment. Having that capability built into Alire would be nice and having an intuitive way of specifying it is also nice.

What about a post-deploy action and make it explicit? Or, how about defining actions separate from their trigger, and having a table associate actions with triggers so you can reuse them (e.g., have an action table that associates a key with an action, and then have trigger tables that associate a trigger with actions).

mosteo commented 9 months ago

These are plausible ideas but they go quite beyond the scope of the current issue. I'll document what we have discussed here to clarify how actions interact with the new shared builds, and implement a natural way of triggering post-fetch in the context of a build configuration without requiring the full build.

dalybrown commented 9 months ago

Sounds good - thanks!