Publish Artifacts CI build step does not support retries

[MichaelSimons]

The Publish Artifacts CI build step will fail if rerun when a previous attempt has uploaded any artifacts. This is because the publish build step is set to run on success or failure and that it does not support overwriting artifacts. When the occurs, the failure will will look similar to the following:

##[error]Artifact CentOSStream9_Online_PreviousSourceBuiltSdk_x64_Artifacts already exists for build 2502184.

There are a couple options to solve this but they have some cons:

1. Only publish on success. This would work for normal build failures but if the 1ES steps fail which are run after publish, re-run will still encounter this issue. Furthermore, in some instance it is desirable to inspect the artifacts that were produced during a failed build. This change would prevent that.
2. Modify the Publish Artifacts step to include the Job Attempt number. This would mean each run would have it's artifacts uploaded to a distinct artifact name. The downside to this, is there are some dependent jobs (e.g. the source-build sdk content tests) which download the artifacts. They would need to account for the possibility of multiple artifacts existing.

[ellahathaway]

A very similar issue was occurring in docker: https://github.com/dotnet/docker-tools/issues/1323

Here is a summary of the options & their respective challenges:

• Force overwrite the published artifact
  • Azdo pipelines don't support force overwriting of artifacts or deleting artifacts
• Upload multiple versions of the artifacts by appending $(System.JobAttempt) to the artifact name
  • This complicates the consumption of the artifacts by other pipelines. Since many of our pipelines download these artifacts, we should avoid making a change that creates a ripple affect like this.
• Don't publish a new artifact when one already exists
  • A successful run should publish artifacts whenever possible. Otherwise, we risk not having artifacts for certain releases/rebootstraps.
• Condition the publishing of the artifact on the success of a previous step (e.g. a successful build)
  • We risk not having artifacts for failing builds. This is not ideal when we are trying to investigate build failures.

Frankly, none of the options are great. In an ideal world, we could delete the old artifacts and upload new ones. But since that's not possible, I think that the best way forward is to condition the publishing of the artifacts on the success of previous step. It poses the least risk and complications, and we'd still have the logs and binlogs from previous attempts so if there's a failure on a previous attempt, we'd still be able to investigate. This is also the solution that was decided upon in https://github.com/dotnet/docker-tools/issues/1323.

[MichaelSimons]

I agree @ellahathaway with your assessment.