[bug] v4: multiple artifacts with the same name

Borda commented 6 months ago

What happened?

It seems the action created multiple artifacts with the same name within one workflow.

The reason why we build such workflows is to be able to aggregate outputs from several jobs as GH does not offer any other way of communication among distanced jobs...

What did you expect to happen?

it will overwrite the past artifact or raise an error...

How can we reproduce it?

See the linked workflow - https://github.com/Lightning-AI/utilities/actions/runs/7253160143?pr=209

Anything else we need to know?

These actions are wrapped in composite action; I'm not sure if it may have any impact...

the main workflow
  |- shared workflow / job A (upload)
  |- shared workflow / job B
  |   |- composite action (download & upload)

What version of the action are you using?

4.0.0

What are your runner environments?

linux

Are you on GitHub Enterprise Server? If so, what version?

No response

robherley commented 6 months ago

👋 @Borda you have a mix of v3 and v4 Artifacts in your job (they are not compatible with each other per the breaking changes). I'd suggest checking any composite workflows that might be creating additional Artifacts.

Borda commented 6 months ago

you have a mix of v3 and v4 Artifacts in your job

I have fixed the download version before opening this issue but did not expect that dependabot does not bump all occurrences... :(

Borda commented 6 months ago

Ok, now I got the existence error, which is progress, but how can I overwrite it with a new version or delete it and create a new one?

Error: Failed to CreateArtifact: Received non-retryable error: Failed request: (409) Conflict: an artifact with this name already exists on the workflow run

robherley commented 6 months ago

Please see this reply: https://github.com/actions/upload-artifact/issues/481#issuecomment-1862899981

As part of v4, you are not able to upload to the same Artifact multiple times.

If you must recreate the Artifact, I have an example here: https://github.com/actions/upload-artifact/issues/471#issuecomment-1858236639

Borda commented 6 months ago

If you must recreate the Artifact, I have an example here: #471 (comment)

Thank you. This is close, but I am afraid it will not work as you can't pass artifact-id from one job to another if they are running as strategy parametrization

yaleman commented 6 months ago

I'm having a similar issue with matrix'd jobs creating duplicate artifacts, or being unable to use the name because it's got invalid values (ie, the parent container name, etc) how are we supposed to name things given the really limited templating capabilities?

bwoodsend commented 4 months ago

how are we supposed to name things given the really limited templating capabilities?

There is an enumeration in ${{ strategy.job-index }}. e.g. If you have a matrix of 10 jobs then that variable will be an integer from 0 to 9. So it looks like the most foolproof way to collect artifacts from multiple jobs is to append that variable to the artifact name then, when downloading, use pattern: with merge-multiple: true instead of name::

      - uses: actions/upload-artifact@v4
        with:
          name: my-artifacts-${{ strategy.job-index }}
          path: ...
      ...
      - uses: actions/download-artifact@v4
        with:
          pattern: my-artifacts-*
          merge-multiple: true

Even with a workaround though, I still think this was a change for the worse. The only motivation I could find for it was that people pushing to the same file (but not the same artifact name) concurrently would end up with broken artifacts which is a totally self inflicted, self explanatory and easily workaround-able problem whereas collecting artifacts from multiple jobs is a pretty key reason for this action's existence. Unless perhaps it would allow you to remove the restriction that artifacts can't be downloaded via the UI or the REST API whilst the workflow is still running, it feels like you've just made an intuitive common thing much harder for everyone in favour of making an invalid thing marginally less confusing for a very small group of people.

actions / upload-artifact