search

arcalot / arcaflow-engine

Arcaflow is a highly-portable workflow engine enabling modular and validated pipelines through containerized plugins.
https://arcalot.io/arcaflow/
Apache License 2.0
6 stars 9 forks source link

Change build to use reusable workflows #169

Closed webbnh closed 2 months ago

webbnh commented 2 months ago

Changes introduced with this PR

This PR reworks the build workflow

By contributing to this repository, I agree to the contribution guidelines.

webbnh commented 2 months ago

Now that https://github.com/arcalot/arcaflow-reusable-workflows/pull/16 is merged, we're ready to try this.

webbnh commented 2 months ago

This PR is having the same problem as https://github.com/arcalot/arcaflow-docsgen/pull/23#issuecomment-2040749623 with the "release" check.

Also, it's having the same problem that https://github.com/arcalot/arcaflow-docsgen/pull/22 had, with the organization variable values not being available in the called workflow. Unfortunately, unlike the Go release workflow which blunders on without it, the Python release workflow dies.

So, @jaredoconnell, unless someone has an objection, you should use your Admin override to merge it; and, if it doesn't run properly after that, we'll fix it then.

jaredoconnell commented 2 months ago

Now that I think of it, I think we need to pass the org variables in from the calling workflow, and have them as input variables in the reusable workflows. I now remember Justin running into this with secrets.

webbnh commented 2 months ago

I think we need to pass the org variables in from the calling workflow, and have them as input variables in the reusable workflows. I now remember Justin running into this with secrets.

Org variables, environment variables, inputs, and secrets are all separate, individual things with individual handling (and frustrations).

Org variables do work, once the workflow is merged in -- we already have evidence of that in go_lint_and_test.yaml. (Secrets, on the other hand -- except for the GITHUB_TOKEN 😛, which is special -- do have to be passed in.)

I suspect that passing the values of the org variables as inputs would get us past the problem of not being able to test prior to merging (and, it would better encapsulate/uncouple the reusable workflows), so it wouldn't be a bad thing. I had contemplated using that approach, but go_lint_and_test.yaml was already using the organization variable directly, and I didn't really want the two new workflows to be different, so I had them use the variables directly as well.

I s'pose I'll retool all three of them.

webbnh commented 2 months ago

This PR is now back in draft awaiting https://github.com/arcalot/arcaflow-reusable-workflows/pull/17.

webbnh commented 2 months ago

This PR has been superseded by #171, in the hopes that, because that's pulling from a branch in the upstream repo, it will test cleanly.