cvat-ai / cvat

Annotate better with CVAT, the industry-leading data engine for machine learning. Used and trusted by teams at any scale, for data of any scale.
https://cvat.ai
MIT License
12.75k stars 3.03k forks source link

how to mount a share path for the big dataset in windows os. #1472

Open ishumann opened 4 years ago

ishumann commented 4 years ago

Hi, I am sorry for a noob question but I want to know how to mount a shared path for the big dataset. I tried to follow the documentation on the Share Path

version: "2.3"

services:
  cvat:
    environment:
      CVAT_SHARE_URL: "Mounted from /mnt/share host directory"
    volumes:
      - cvat_share:/home/django/share:ro

volumes:
  cvat_share:
    driver_opts:
      type: none
      device: /mnt/share
      o: bind

but I am getting this error while building

$ docker-compose -f docker-compose.yml -f docker-compose.override.yml -f components/openvino/docker-compose.openvino.yml -f cvat/apps/dextr_segm
entation/docker-compose.dextr.yml  -f components/auto_segmentation/docker-compose.auto_segmentation.yml -f components/tf_annotation/docker-compo
se.tf_annotation.yml -f components/analytics/docker-compose.analytics.yml up -d

WARNING: The no_proxy variable is not set. Defaulting to a blank string.
WARNING: The http_proxy variable is not set. Defaulting to a blank string.
WARNING: The https_proxy variable is not set. Defaulting to a blank string.
cvat_redis is up-to-date
cvat_db is up-to-date
cvat_elasticsearch is up-to-date
cvat_kibana is up-to-date
cvat_logstash is up-to-date
Creating cvat ... error                                                                                                                          
ERROR: for cvat  Cannot create container for service cvat: failed to mount local volume: mount /mnt/share:/var/lib/docker/volumes/cvat_cvat_share/_data, flags: 0x1000: no such file or directory

ERROR: for cvat  Cannot create container for service cvat: failed to mount local volume: mount /mnt/share:/var/lib/docker/volumes/cvat_cvat_share/_data, flags: 0x1000: no such file or directory
ERROR: Encountered errors while bringing up the project.

Docker version Client: Docker Engine - Community Version: 19.03.8 API version: 1.40 Go version: go1.12.17 Git commit: afacb8b Built: Wed Mar 11 01:23:10 2020 OS/Arch: windows/amd64 Experimental: true

I also tried to post my problem on Gitter no reply what procedure do I need to follow to work this out please help me.

nmanovic commented 4 years ago

@MannAgrawal , does /mnt/share path exists on your host machine?

ishumann commented 4 years ago

@nmanovic Unfortunately, I don't... I am sorry I am very new to docker and cvat but don't know how to do it... could you please give me instruction to how to mount the share folder. and thank you for your quick response.

nmanovic commented 4 years ago

@MannAgrawal , you need to create the directory. Please run mkdir /mnt/share or sudo mkdir /mnt/share (or choose any other directory and update docker-compose.override.yml accordingly). After that run your up -d command again. It should be without errors. Put your data into /mnt/share directory and inside CVAT you will be able to use it.

nmanovic commented 4 years ago

I will close the issue. A lot of people on gitter can help you with the question. Don't expect that people will answer immediately. Find people which solved the problem in the past and ask them directly.

barondra commented 3 years ago

Hi, I am using Windows. If I want to cvat to connect local directory at C:\mnt\share Then I should not change anything in override.yml file copied in the guide right? But it still doesnt connect. What am I missing here? It has been frustrating

ronzhin-dmitry commented 3 years ago

Hi, I am using Windows. If I want to cvat to connect local directory at C:\mnt\share Then I should not change anything in override.yml file copied in the guide right? But it still doesnt connect. What am I missing here? It has been frustrating

Hello! I'm having similar issues, I cannot mount directory in docker from WSL. Here are some illustrations of how this whole process looks like:

cvat-error

Did you manage to solve the problem?

ronzhin-dmitry commented 3 years ago

Hi, I am using Windows. If I want to cvat to connect local directory at C:\mnt\share Then I should not change anything in override.yml file copied in the guide right? But it still doesnt connect. What am I missing here? It has been frustrating

Hello! I'm having similar issues, I cannot mount directory in docker from WSL. Here are some illustrations of how this whole process looks like:

cvat-error

Did you manage to solve the problem?

Ok, so thanks to issue #2315 (https://github.com/openvinotoolkit/cvat/issues/2315) and particularly answer of https://github.com/ChrisCurrin I found out that it can be fixed manually on WSL. The actions he described could be performed after cleaning up containers and volumes, make external volume in docker-compose.override.yml and add volume manually via docker volume create --name cvat_share --opt type=none --opt device=YOUR-PATH --opt o=bind

After adding it manually everything seemed to be working fine

P.S.: One thing to mention: under WSL in Windows I had to write win-style path, i.e. YOUR-PATH = C:\someFolder Otherwise powershell still resulted in mounting error.

nmanovic commented 3 years ago

@ronzhin-dmitry , thanks for your reply. If you can contribute into documentation and add necessary information to simplify life of other users, it will be awesome.

jimwormold commented 3 years ago

Hi, I am using Windows. If I want to cvat to connect local directory at C:\mnt\share Then I should not change anything in override.yml file copied in the guide right? But it still doesnt connect. What am I missing here? It has been frustrating

Hello! I'm having similar issues, I cannot mount directory in docker from WSL. Here are some illustrations of how this whole process looks like: cvat-error Did you manage to solve the problem?

Ok, so thanks to issue #2315 (#2315) and particularly answer of https://github.com/ChrisCurrin I found out that it can be fixed manually on WSL. The actions he described could be performed after cleaning up containers and volumes, make external volume in docker-compose.override.yml and add volume manually via docker volume create --name cvat_share --opt type=none --opt device=YOUR-PATH --opt o=bind

After adding it manually everything seemed to be working fine

P.S.: One thing to mention: under WSL in Windows I had to write win-style path, i.e. YOUR-PATH = C:\someFolder Otherwise powershell still resulted in mounting error.

Finally, a solution that worked! Thanks

ronzhin-dmitry commented 2 years ago

Hello! Still did not put it into documentation, but have one more thing to mention: if you did all the tricks I described above and still do not see any files in the task-creation file-share tab -- just check that you have some files in the directory you're trying to share. Somehow if the directory is empty you will see the same view of the tab as if share was not connected at all. That made me really mad recently, maybe will be useful for someone.

lschaupp commented 2 years ago

For Reference (copied from Chris in another thread). This worked for me:

https://github.com/openvinotoolkit/cvat/issues/2263#issuecomment-719040863

kimlia545 commented 2 years ago

https://openvinotoolkit.github.io/cvat/docs/administration/basics/installation/#share-path

# docker-compose.override.yml
version: '3.3'

services:
  cvat:
    environment:
      CVAT_SHARE_URL: 'Mounted from /mnt/share host directory'
    volumes:
      - cvat_share:/home/django/share:ro

volumes:
  cvat_share:
    driver_opts:
      type: none
      device: /mnt/share
      o: bind
samiechan commented 1 year ago

docker-compose.override.yml

services:
  cvat_server:
    volumes:
      - cvat_share:/home/django/share:ro
  cvat_worker_import:
    volumes:
      - cvat_share:/home/django/share:ro

volumes:
  cvat_share:

Create volume: docker volume create --name cvat_cvat_share --opt type=none --opt device=C:\your_folder_path --opt o=bind

Run docker compose