kouprlabs / voltaserve

⚡️ Cloud Storage for Creators
https://voltaserve.com
Other
271 stars 14 forks source link

How can I mount an existing local folder as a volume? #295

Open stancubed opened 2 months ago

stancubed commented 2 months ago

Question:

I'm trying to add a local directory so that I can work with it inside a workspace and have those changes reflected in the directory as well. At the moment, it looks like Voltaserve is built to serve fresh bucket content, but how can I get existing data into it? Are there any ways to do this currently or any plans to add this capability?


Context:

I've got this deployed via Portainer on Docker in Linux and everything is working very well so far!


What have you tried:

I've tried adding my local dir to minio as a volume, but that doesn't seem to change anything on the service.

minio:
    image: minio/minio:RELEASE.2024-04-06T05-26-02Z
    container_name: volta_minio
    networks:
      - voltaserve_default
    ports:
      - ${VOLTASERVE_MINIO_PORT}:9000
      - ${VOLTASERVE_MINIO_CONSOLE_PORT}:9001
    environment:
      MINIO_ROOT_USER: voltaserve
      MINIO_ROOT_PASSWORD: voltaserve
      MINIO_REGION: us-east-1
    volumes:
      - /home/user/storage/digital/ebooks:/data
    command: server /data --console-address ":9001"
bouassaba commented 2 months ago

@stancubed what about using WebDAV to import your existing data into Voltaserve? You can use Cyberduck, WinSCP or Rclone.

stancubed commented 2 months ago

I need the data to remain in its current structure because it's also used by other containers, so I was hoping I could keep the existing storage infra but mount it into Voltaserve to interact with it as well. If I import it into Voltaserve, it'll disappear into Minio and won't be accessible by the host machine or other containers that have them mounted as volumes, right?

bouassaba commented 2 months ago

Right, the data used by Voltaserve will be living in MinIO. A workaround would be to run a periodic sync from time to time with WebDAV, between your source folder and Voltaserve. Would that work for you?

stancubed commented 2 months ago

I think that would double the space occupied, which is an expensive endeavor.

On Tue, Sep 3, 2024, 5:32 PM Anass Bouassaba @.***> wrote:

Right, the data used by Voltaserve will be living in MinIO. A workaround would be to run a periodic sync from time to time with WebDAV, between your source folder and Voltaserve. Would that work for you?

— Reply to this email directly, view it on GitHub https://github.com/kouprlabs/voltaserve/issues/295#issuecomment-2327554567, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA5SDSEVM7N6R72K6RVZEVTZUY2GZAVCNFSM6AAAAABNLRWDPWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMRXGU2TINJWG4 . You are receiving this because you were mentioned.Message ID: @.***>

snapshotleisure commented 1 month ago

Hi @bouassaba, is it possible to import an existing Minio bucket in? or it has to be fresh? Also if there were any changes that were done to the file outside Voltaserve, would those changes be picked up?

bouassaba commented 1 month ago

Hi @bouassaba, is it possible to import an existing Minio bucket in? or it has to be fresh? Also if there were any changes that were done to the file outside Voltaserve, would those changes be picked up?

@snapshotleisure it has to be a fresh bucket. But as long as you follow the folder structure of Voltaserve inside MinIO - external changes will be picked up. Create a fresh bucket, and import some files using Voltaserve, then have a look at the structure to get familiar with it. I will be happy to answer any questions regarding this.

bouassaba commented 1 month ago

@snapshotleisure btw the IDs you see in the folder names inside MinIO's bucket are snapshot IDs, same as in the database table snapshot and snapshot_file (this one is the association between snapshots and files). A file can have multiple snapshots.

snapshotleisure commented 1 month ago

Ok, thank you for that