fsspec / gcsfs

Pythonic file-system interface for Google Cloud Storage
http://gcsfs.readthedocs.io/en/latest/
BSD 3-Clause "New" or "Revised" License
320 stars 141 forks source link

Strange error message when using cp instead of put #599

Closed alex-hh closed 6 months ago

alex-hh commented 6 months ago

I was trying to upload a local file using cp, and found the following error message rather misleading (the problem was solved by using put instead):

Traceback (most recent call last):
  File "/Users/alex/proteins/evo-gvp/evogvp/benchmarking/run.py", line 53, in <module>
    run(
  File "/Users/alex/proteins/evo-gvp/evogvp/benchmarking/run.py", line 26, in run
    metrics = pipeline.run(
              ^^^^^^^^^^^^^
  File "/Users/alex/proteins/evo-gvp/evogvp/benchmarking/pipelines/pipelines.py", line 121, in run
    raise e
  File "/Users/alex/proteins/evo-gvp/evogvp/benchmarking/pipelines/pipelines.py", line 111, in run
    self.run_validation_on_instance(
  File "/Users/alex/proteins/evo-gvp/evogvp/benchmarking/pipelines/pipelines.py", line 274, in run_validation_on_instance
    metrics = validation(
              ^^^^^^^^^^^
  File "/Users/alex/proteins/evo-gvp/evogvp/benchmarking/validations/ardca.py", line 75, in __call__
    self.upload_msa(target_id, output_msa_file)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/alex/proteins/evo-gvp/evogvp/benchmarking/validations/ardca.py", line 97, in upload_checkpoint
    print(params_file)
    ^^^^^^^^^^^^^^^^^^^
  File "/Users/alex/envs/devo/lib/python3.11/site-packages/fsspec/spec.py", line 1506, in cp
    return self.copy(path1, path2, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/alex/envs/devo/lib/python3.11/site-packages/fsspec/asyn.py", line 118, in wrapper
    return sync(self.loop, func, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/alex/envs/devo/lib/python3.11/site-packages/fsspec/asyn.py", line 103, in sync
    raise return_result
  File "/Users/alex/envs/devo/lib/python3.11/site-packages/fsspec/asyn.py", line 56, in _runner
    result[0] = await coro
                ^^^^^^^^^^
  File "/Users/alex/envs/devo/lib/python3.11/site-packages/fsspec/asyn.py", line 390, in _copy
    raise ex
  File "/Users/alex/tools/homebrew/Cellar/python@3.11/3.11.6_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/tasks.py", line 452, in wait_for
    return await fut
           ^^^^^^^^^
  File "/Users/alex/envs/devo/lib/python3.11/site-packages/gcsfs/core.py", line 1126, in _cp_file
    out = await self._call(
          ^^^^^^^^^^^^^^^^^
  File "/Users/alex/envs/devo/lib/python3.11/site-packages/gcsfs/core.py", line 437, in _call
    status, headers, info, contents = await self._request(
                                      ^^^^^^^^^^^^^^^^^^^^
  File "/Users/alex/envs/devo/lib/python3.11/site-packages/decorator.py", line 221, in fun
    return await caller(func, *(extras + args), **kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/alex/envs/devo/lib/python3.11/site-packages/gcsfs/retry.py", line 123, in retry_request
    return await func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/alex/envs/devo/lib/python3.11/site-packages/gcsfs/core.py", line 430, in _request
    validate_response(status, contents, path, args)
  File "/Users/alex/envs/devo/lib/python3.11/site-packages/gcsfs/retry.py", line 104, in validate_response
    raise OSError(f"Forbidden: {path}\n{msg}")
OSError: Forbidden: b/tmp/o/1a1xA%2F1703792249%2Fardca.conf/rewriteTo/b/evogvp/o/benchmark_artifacts%2FCATH_example%2Fardca%2F1a1xA%2Fardca.conf
The billing account for the owning project is disabled in state closed
martindurant commented 6 months ago

It basically means "you don't own that bucket". Since the whole point of fsspec/gcsfs is to make paths appear similar on whatever backend, there isn't a clear way to distinguish that the path you used is not meant to be interpreted as bucket/path.

alex-hh commented 6 months ago

Understood!