search

coiled / feedback

A place to provide Coiled feedback
14 stars 3 forks source link

Error writing to bucket with the new `--mount-bucket` feature #294

Closed rsignell closed 3 months ago

rsignell commented 3 months ago

I tried out the new --mount-bucket feature, and while I can read from a coiled-generated Jupyter notebook instance, I'm getting an error writing.

I created a new bucket s3://esip-coiled with the same AWS credentials I used for coiled, and verified I could write a test file using the AWS CLI.

I then started a Jupyter notebook instance with Coiled:

coiled notebook start --region us-west-2 --name rsignell-notebook --vm-type m5.xlarge --software pangeo-notebook --workspace esip-lab --mount-bucket s3://esip-coiled/rsignell

and when I opened a notebook, I could see my test file, but got an error on write:

$ pwd
/mount/esip-coiled-rsignell
$ ls
coiled_pangeo_notebook_env.yml
$ git clone https://github.com/fs-jbzambon/opendata-coawst.git
Cloning into 'opendata-coawst'...
error: unable to write symref for HEAD: Function not implemented

I did notice this environment variable is set in the Jupyter instance:

AWS_CONTAINER_CREDENTIALS_FULL_URI=http://localhost:3000/creds/aws-token-creds.json
ntabris commented 3 months ago

Yeah, it unfortunately looks like git doesn't work (or at least, doesn't easily work) inside a FUSE mount, which is what we're using.

@rsignell do you want to use git in the notebook, or is the primary goal just to get the files from the git repo in the persistent storage?

This very much isn't a great solution, but at present I got the files from the git repo by cloning the repo locally, then using

aws s3 sync ./opendata-coawst/ s3://coiled-sandbox-nat/notebook-git/opendata-coawst/

(or in your case it would be s3://esip-coiled/rsignell/opendata-coawst/)

to copy the local checkout files to the S3 bucket, which can then be accessed from the notebook.

ntabris commented 3 months ago

To be clear, I'm moderately confident that the issue is the special types of file operations that git needs. You should be able to write to the bucket—you can confirm this with touch test-write or echo "hello" > test-write or something like that.

rsignell commented 3 months ago

Indeed, echo "test" > text.txt works fine. I should have thought to try that!