Closed virajprakash closed 6 months ago
Hmmm, thanks for the thorough explanation! I haven't encountered this before but I can suggest some things to try:
One would be mounting one directory deeper, so you're passing a "normal" directory to docker rather than a mount point. So mounting s3fs idemo /root/opentopodata/data/s3mount/
then the config file would point to data/s3mount/dataset
.
The next step would be execing into the running docker container and having a look around.
docker run
command abovedocker ps
to get the container ID for opentopodatadocker exec -it XXXXX bash
where XXXX is the container IDcd /app/
ls -lh
and make sure the data
folder is there with normal permissionsls -lh /app/data/
and make sure the dataset
folder is there with normal permissionsls -lgh /app/data/dataset/
and make sure the tifs are there.I've never used s3fs, but have used rclone mount to mount cloud buckets then have those as docker volumes for opentopodata and gpxz: you could look into that if you're having lots of issues.
Hey I am still stuck unfortunately! We have switched to rclone since you seem to be reporting success with using it to mount a cloud bucket specifically on opentopodata. Here are my exact steps:
start rclone to mount from our s3 bucket into /data/s3mount Leaving this process running, I start a new terminal and use the docker run command from above
Below are the permissions for all the folders and also all the rest of the commands you mentioned: ls -lh
ls -lh /app/data/
and lastly ls -lgh /app/data/dataset returns all of the tiff files
config.yaml:
name: ned10m path: data/s3mount/dataset
Thank you so much for your help
I have also experimented with multiple config paths including the one you suggested and I also did make build before testing any of them using make run. However I am still seeing 'no dataset folder' when i try to run
We also tried using s3fs and no luck with that unfortunately
It looks like the files are at data/dataset
rather than data/s3mount/dataset
, could you try updating config.yaml
to match?
You could also try an absolute path: /app/data/dataset
It looks like the files are at
data/dataset
rather thandata/s3mount/dataset
, could you try updatingconfig.yaml
to match?You could also try an absolute path:
/app/data/dataset
Closing assuming this was the issue.
Hi, I am getting the following error when I use 'make run' ERROR:root:Invalid config: No dataset folder found at location 'data/dataset' However, I already made sure that the S3 bucket is setup correctly using s3fs and verified on my local file system s3fs idemo /root/opentopodata/data -o passwd_file=${HOME}/.passwd-s3fs
Also attached are relevant portions of my config.yaml and makefile.
Makefile portion
run: docker run --rm -it --volume /root/opentopodata/data:/app/data:ro -p 5000:5000 opentopodata:$(VERSION)
config.yaml:
Thanks in advance !!!
Yes I know it is not recommended to use root and I will be sure to change that but I am almost positive this is a separate issue.