Limit the number of pipeline artifacts that are published where possible

[mthalman]

When using 1ES pipeline templates, each time a pipeline artifact is published, it executes a set of auto-injected steps to scan the artifact (8 of these steps, currently). In the publishing pipeline, multiple artifacts are published even from the same job. For example, both the build and publish jobs publish two artifacts each and some upcoming changes to the publish stage will increase that one to three. So there are multiple sets of these steps that get injected into the pipeline. This is problematic for a few reasons:

• It makes it difficult to find a specific step in the UI because of all the noise from these auto-injected steps.
• It increases the risk of a random failure occurring from one of these steps, forcing a retry.
• It's likely not efficient from a performance perspective to execute these steps across multiple artifacts when it could execute them once on a consolidated artifact.

We should consider trying to consolidate the files we want to store as artifacts and publish them only once. I expect that there are some cases where this might not be possible based on other constraints.

[dotnet-issue-labeler[bot]]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[dotnet-issue-labeler[bot]]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[lbussell]

Completely agree. Also related is https://github.com/dotnet/docker-tools/issues/1331 (specifically https://github.com/dotnet/docker-tools/issues/1331#issuecomment-2166524315). Disabling SBOM generation for most/all artifacts that don't ship to users will greatly cut down on the number of extra steps and extra artifacts that are produced by the pipeline.