Adding `--skip-existing` on upload

[hmaarrfk]

Comment:

I'm trying to build a few different combinations of split packages.

However, some outputs are the same between builds which is causing them to fail upload as shown in the message bolow.

I know that anaconda upload has an option --skip-existing would it be acceptable to add it?

The alternative is to add the package to every output, which is both redundant, and "unecessary". Example logs:

• linux_64_libprotobuf4.25.3.log
• libopenvino_protobuf_4.24.4.log

xref: https://github.com/conda-forge/openvino-feedstock/pull/86

[ERROR] 
Traceback (most recent call last):
  File "/opt/conda/bin/anaconda", line 10, in <module>
    sys.exit(main())
  File "/opt/conda/lib/python3.10/site-packages/binstar_client/scripts/cli.py", line 257, in main
    binstar_main(commands, args, exit_)
  File "/opt/conda/lib/python3.10/site-packages/binstar_client/scripts/cli.py", line 206, in binstar_main
    return arguments.main(arguments)
  File "/opt/conda/lib/python3.10/site-packages/binstar_client/commands/upload.py", line 61, in main
    uploader.upload(filename)
  File "/opt/conda/lib/python3.10/site-packages/binstar_client/commands/upload.py", line 504, in upload
    return self.upload_package(filename, package_meta)
  File "/opt/conda/lib/python3.10/site-packages/binstar_client/commands/upload.py", line 528, in upload_package
    if not self._check_file(meta):
  File "/opt/conda/lib/python3.10/site-packages/binstar_client/commands/upload.py", line 579, in _check_file
    raise errors.Conflict(f'file {basename} already exists for package {meta.name} version {meta.version}', 409)
binstar_client.errors.Conflict: ('file linux-64/libopenvino-intel-cpu-plugin-2024.0.0-h2e90f83_3.conda already exists for package libopenvino-intel-cpu-plugin version 2024.0.0', 409)
Failed to upload due to Command '['anaconda', '--quiet', '--show-traceback', '-t', '/tmp/tmprhlx6woi/binstar.token', 'upload', '/home/conda/feedstock_root/build_artifacts/linux-64/libopenvino-intel-cpu-plugin-2024.0.0-h2e90f83_3.conda', '--user=cf-staging', '--channel=main', '--force-metadata-update']' returned non-zero exit status 1.. Trying again in 10 seconds
Using STAGING_BINSTAR_TOKEN for anaconda.org uploads to cf-staging.
INFO:binstar:Using Anaconda API: https://api.anaconda.org
Distribution /home/conda/feedstock_root/build_artifacts/linux-64/libopenvino-intel-cpu-plugin-2024.0.0-h2e90f83_3.conda already exists on cf-staging. Waiting another 15 seconds to try uploading again.
Distribution /home/conda/feedstock_root/build_artifacts/linux-64/libopenvino-intel-cpu-plugin-2024.0.0-h2e90f83_3.conda already exists on cf-staging. Waiting another 30 seconds to try uploading again.

cc: @ilya-lavrenov

[hmaarrfk]

It might be something like this patch:

diff --git a/recipe/conda_forge_ci_setup/upload_or_check_non_existence.py b/recipe/conda_forge_ci_setup/upload_or_check_non_existence.py
index bd8c0d2..8188ed6 100644
--- a/recipe/conda_forge_ci_setup/upload_or_check_non_existence.py
+++ b/recipe/conda_forge_ci_setup/upload_or_check_non_existence.py
@@ -105,7 +105,7 @@ def built_distribution_already_exists(cli, name, version, fname, owner):
 def upload(token_fn, path, owner, channels, private_upload=False, force_metadata_update=True):
     cmd = ['anaconda', '--quiet', '--show-traceback', '-t', token_fn,
            'upload', path, '--user={}'.format(owner),
-           '--channel={}'.format(channels)]
+           '--channel={}'.format(channels), '--skip-existing']
     if private_upload:
         cmd.append("--private")
     if force_metadata_update:

[jaimergp]

I wonder if we rely on this to fix partial uploads that didn't finish due to network errors. Regardless, it could be added as a conda-forge.yml option via conda-smithy? We only need to support that optional flag here.