search

fsspec / gcsfs

Pythonic file-system interface for Google Cloud Storage
http://gcsfs.readthedocs.io/en/latest/
BSD 3-Clause "New" or "Revised" License
330 stars 142 forks source link

"Forbidden" for writing, but not reading, when using `token='cache'` and `token='cloud'` #230

Open arokem opened 4 years ago

arokem commented 4 years ago

We are using gcsfs inside a pangeo-powered cluster on GCP. One of our users has run into the following somewhat-mysterious behavior: initializing a gcfs.GCSFileSystem object with token='cache' and token='cloud' allows him to read items that are in a bucket on our project, but he hits a 403 and gets Forbidden: https://www.googleapis.com/upload/storage/v1/b/learning2learn/o Insufficient Permission when trying to write to the bucket. This is resolved when using token='browser' and going through that process. Beyond fixing the particular issue, I am also a bit confused on when/how/whether a token gets cached in the user's home directory and how we can examine whether a cached token works. Thanks!

martindurant commented 4 years ago

The last bit I can answer simply: a token is only cached by gcsfs when successfully using the "browser" method. Note to be sure that you are specifying the project parameter every time (except "cloud", where it might have been specified with the gcloud config set command), this can cause problems.