fema-ffrd / cloud-mock

Docker network with minio and stac-browser to mock cloud environment for development activities
MIT License
1 stars 0 forks source link

decide on mount vs recursive copy #2

Open HenryGeorgist opened 8 months ago

HenryGeorgist commented 8 months ago

two approaches for minio have been proposed:

  1. mounting a local directory for payloads and model data
  2. recursively copying local data into minio

for 1, a developer can make a local change and see it in minio, which is useful. for 2, the workflow keeps the local copy pristine, only making edits in the minio instance, allowing for the developer to take down minio and relaunch it removing any edits a plugin may have made. it requires remembering to restart minio if you want a local file change to be up in the minio instance, but it does provide a benefit of starting from a clean slate each time.

HenryGeorgist commented 8 months ago

recursive copy example docker compose file

version: '3'
services:
  minio:
    container_name: s3
    image: minio/minio
    environment:
      - MINIO_ROOT_USER=AKIAIOSFODNN7EXAMPLE
      - MINIO_ROOT_PASSWORD=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
    command: server /data --console-address ":9001"
    ports:
      - '9000:9000'
      - '9001:9001'
  # configure minio on startup (create buckets, etc)
  # inspired by https://github.com/minio/minio/issues/4769
  # and         https://gist.github.com/haxoza/22afe7cc4a9da7e8bdc09aad393a99cc
  minio_init:
    container_name: s3_init
    image: minio/mc
    depends_on:
      - minio
    entrypoint: >
      /bin/sh -c "
      /usr/bin/mc config host add minio http://minio:9000 AKIAIOSFODNN7EXAMPLE wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY;
      /usr/bin/mc rb --force  minio/model-library;
      /usr/bin/mc mb minio/model-library;
      /usr/bin/mc policy set public minio/model-library;
      /usr/bin/mc cp --recursive /models/ minio/model-library/;
      /usr/bin/mc rb --force  minio/cc-store;
      /usr/bin/mc mb minio/cc-store;
      /usr/bin/mc policy set public minio/cc-store;
      /usr/bin/mc cp --recursive /payloads/ minio/cc-store/cc_store/;
      exit 0;
      "
    volumes:
      - ./models:/models
      - ./payloads:/payloads