minio / mc

Unix like utilities for object store
https://min.io/download
GNU Affero General Public License v3.0
2.86k stars 548 forks source link

mc --continue does not seem to work on Docker Image #4981

Closed antoineroux closed 3 months ago

antoineroux commented 3 months ago

Expected behavior

I am using minio and mc in a Docker Compose file to set up a test environment containing an S3 bucket. I configured my mc service like this:

  mc:
    image: quay.io/minio/mc:RELEASE.2024-07-08T20-59-24Z
    entrypoint: sh -c "mc alias set local http://mock-s3:9000 minioadmin minioadmin && mc mb --ignore-existing local/jobs-product && mc cp --continue --recursive /ci_reference_files/* local/jobs-product"
    volumes:
      - ./mock-s3/ci_reference_files:/ci_reference_files
    depends_on:
      mock-s3:
        condition: service_started

I am expecting mc not to try and copy the files if they were already created in a previous session, thanks to the --continue flag in mc cp --continue --recursive /ci_reference_files/* local/jobs-product". In any case it should complete succesfully.

According to the documentation, --continue is a valid flag.

Actual behavior

mc fails with the following error:

mc: <ERROR> Invalid command usage, flag provided but not defined: -continue

SUPPORTED FLAGS:
   --rewind value                     roll back object(s) to current version at specified time
   --version-id value, --vid value    select an object version to copy
   --recursive, -r                    copy recursively
   --older-than value                 copy objects older than value in duration string (e.g. 7d10h31s)
   --newer-than value                 copy objects newer than value in duration string (e.g. 7d10h31s)
   --storage-class value, --sc value  set storage class for new object(s) on target
   --attr value                       add custom metadata for the object
   --preserve, -a                     preserve filesystem attributes (mode, ownership, timestamps)
   --disable-multipart                disable multipart upload feature
   --md5                              force all upload(s) to calculate md5sum checksum
   --tags value                       apply one or more tags to the uploaded objects
   --retention-mode value             retention mode to be applied on the object (governance, compliance)
   --retention-duration value         retention duration for the object in d days or y years
   --legal-hold value                 apply legal hold to the copied object (on, off)
   --zip                              Extract from remote zip file (MinIO server source only)
   --enc-c value                      encrypt/decrypt objects using client provided keys. (multiple keys can be provided) Formats: RawBase64 or Hex.
   --enc-kms value                    encrypt/decrypt objects using specific server-side encryption keys. (multiple keys can be provided) [$MC_ENC_KMS]
   --enc-s3 value                     encrypt/decrypt objects using server-side default keys and configurations. (multiple keys can be provided). [$MC_ENC_S3]
   --config-dir value, -C value       path to configuration folder (default: "/root/.mc") [$MC_CONFIG_DIR]
   --quiet, -q                        disable progress bar display [$MC_QUIET]
   --disable-pager, --dp              disable mc internal pager and print to raw stdout [$MC_DISABLE_PAGER]
   --no-color                         disable color theme [$MC_NO_COLOR]
   --json                             enable JSON lines formatted output [$MC_JSON]
   --debug                            enable debug output [$MC_DEBUG]
   --insecure                         disable SSL certificate verification [$MC_INSECURE]
   --limit-upload value               limits uploads to a maximum rate in KiB/s, MiB/s, GiB/s. (default: unlimited) [$MC_LIMIT_UPLOAD]
   --limit-download value             limits downloads to a maximum rate in KiB/s, MiB/s, GiB/s. (default: unlimited) [$MC_LIMIT_DOWNLOAD]
   --help, -h                         show help

Steps to reproduce the behavior

Run Docker Compose on this example (I am using /tmp as a source of files to copy, you can change it to something else):

services:

  mock-s3:
    image: quay.io/minio/minio:RELEASE.2024-06-29T01-20-47Z
    ports:
      - 9000:9000
      - 9001:9001
    environment:
      - MINIO_ROOT_USER=minioadmin
      - MINIO_ROOT_PASSWORD=minioadmin
    command: server /data --address :9000 --console-address :9001

  mc:
    image: quay.io/minio/mc:RELEASE.2024-07-08T20-59-24Z
    entrypoint: sh -c "mc alias set local http://mock-s3:9000 minioadmin minioadmin && mc mb --ignore-existing local/jobs-product && mc cp --recursive -c /ci_reference_files/* local/jobs-product"
    volumes:
      - /tmp:/ci_reference_files
    depends_on:
      mock-s3:
        condition: service_started

mc --version

quay.io/minio/mc:RELEASE.2024-07-08T20-59-24Z

harshavardhana commented 3 months ago

--continue has been long deprecated and removed, there is no such thing as --continue anymore.

harshavardhana commented 3 months ago

I am expecting mc not to try and copy the files if they were already created in a previous session, thanks to the --continue flag

if you want this use mc mirror instead of cp

antoineroux commented 3 months ago

Thanks a lot for the answer. The documentation still mentions --continue, hence this request. I will try mc mirror it looks like it does exactly what I want.

harshavardhana commented 3 months ago

Thanks a lot for the answer. The documentation still mentions --continue, hence this request. I will try mc mirror it looks like it does exactly what I want.

Will get rid of the docs