[bug] v4: overwrite: true fails with parallel jobs writing to the artifact

[yury-s]

What happened?

When setting overwrite: true, upload sometimes fails with the following output:

Run actions/upload-artifact@v4
  with:
    name: pull-request-number
    path: pull_request_number.txt
    overwrite: true
    if-no-files-found: warn
    compression-level: 6
  env:
    FORCE_COLOR: 1
    FLAKINESS_CONNECTION_STRING: 
    ELECTRON_SKIP_BINARY_DOWNLOAD: 1
    PWTEST_BOT_NAME: ubuntu-latest-node18-1
With the provided path, there will be 1 file uploaded
Artifact name is valid!
Root directory input is valid!
Error: Failed to CreateArtifact: Received non-retryable error: Failed request: (409) Conflict: an artifact with this name already exists on the workflow run

What did you expect to happen?

One of the job succeeded and other silently continue doing nothing as documentation of the option suggests.

How can we reproduce it?

Configure several parallel jobs within a workflow and matrixes to write into the same artifact with overwrite: true, something like this:

    - name: Upload artifact with the pull request number
      if: always() && github.event_name == 'pull_request'
      uses: actions/upload-artifact@v4
      with:
        name: pull-request-number
        path: pull_request_number.txt
        overwrite: true

The upload action will fail sometimes, e.g. see this run.

Anything else we need to know?

No response

What version of the action are you using?

v4.3.0

What are your runner environments?

linux, window, macos

Are you on GitHub Enterprise Server? If so, what version?

No response

[melloware]

I get this error as well https://github.com/open-sce/fluent-cli/actions/runs/7254116343/job/19762158254

Error: Failed to CreateArtifact: Received non-retryable error: Failed request: (409) Conflict: an artifact with this name already exists on the workflow run

[darthcloud]

Updated to v4 as well due to Node.js warning, added overwrite: true and still getting the error: https://github.com/darthcloud/BlueRetro/actions/runs/7768402627/workflow

[melloware]

I even tried v4.3.0 just to make sure it was using the latest version and the same error happens.

[melloware]

I was able to fix my problem by following the migration guide:https://github.com/actions/upload-artifact/blob/main/docs/MIGRATION.md

See my commit: https://github.com/open-sce/fluent-cli/commit/900fed54e6680276ffbc62365843f005ceb7e990

[moos3]

I'm running into the this issue Failed to CreateArtifact: Received non-retryable error: Failed request: (409) Conflict: an artifact with this name already exists on the workflow run only on some builds, not always. I'm doing this as the upload

- name: Upload meta bake definition
        uses: actions/upload-artifact@v4
        with:
          name: ${{ matrix.version }}-onbuild-poetry-bake-meta
          path: /tmp/bake-meta.json
          if-no-files-found: error
          overwrite: true

Isn't the whole point of overwrite, to overwrite it and not care if that artifact is already there?

[JonathanAtCenterEdge]

Getting this exact same issue, randomly with overrwrite: true my workflows are failing with a 409 conflict

[darthcloud]

I don't understand why you are deprecating v3: https://github.blog/changelog/2024-04-16-deprecation-notice-v3-of-the-artifact-actions/

When v4 is still broken on parallel job setup.

[robherley]

👋 I want to clarify that the overwrite operation is not atomic. It simply is a helper to delete the artifact before creating a new one.

The intended purpose of this overwrite feature was not for parallel jobs. It was meant for serial overwriting (like uploading a binary, then downloading it, signing it, then reuploading it).

If you are trying to upload to the same artifact name across parallel jobs, you will hit race conditions. This does not merge artifacts across jobs. If you have jobs A, B and C and they all try to upload my-artifact with overwrite: true, only the contents from one of the job's artifacts will be contained, the last one that wrote. The uploads from the other jobs would be just wasted time. You would be better off skipping the other jobs with a conditional if you do not care about the artifact contents to save runtime costs.

However, if you do care about the artifact contents from multiple concurrent jobs, you simply need to give the artifacts different names (like variables of your parallel matrix) and call actions/upload-artifact/merge which is outlined in the migration guide: https://github.com/actions/upload-artifact/blob/main/docs/MIGRATION.md#merging-multiple-artifacts

This is exactly what @melloware stated above and implemented in their workflow, and is the correct solution for v4 due to the key differences of how this new major version works.

Hope this helps!