search

seung-lab / cloud-files

Threaded Python and CLI client library for AWS S3, Google Cloud Storage (GCS), in-memory, and the local filesystem.
BSD 3-Clause "New" or "Revised" License
38 stars 8 forks source link

Tests for GCS #9

Open william-silversmith opened 4 years ago

william-silversmith commented 4 years ago

Right now I'm sort of getting this tested by proxy via CloudVolume's tests. It's ugly though.

fcollman commented 4 years ago

there is a bug in the current implementation using the latest versions of crc32c cloud-volume and cloud-files.


metadata.py", line 71, in fetch_info
    info = self.cache.download_json(self.meta.join(self.mesh_path, 'info'))
  File "/usr/local/lib/python3.7/site-packages/cloudvolume/cacheservice.py", line 354, in download_json
    res = self.download( [ path ], compress=compress, progress=False )
  File "/usr/local/lib/python3.7/site-packages/cloudvolume/cacheservice.py", line 477, in download
    raise frag['error']
  File "/usr/local/lib/python3.7/site-packages/cloudfiles/cloudfiles.py", line 176, in download
    check_crc32c(path, content, server_hash)
  File "/usr/local/lib/python3.7/site-packages/cloudfiles/cloudfiles.py", line 153, in check_crc32c
    crc = crc32c(content)
  File "/usr/local/lib/python3.7/site-packages/cloudfiles/lib.py", line 155, in crc32c
    return crc32clib.value(binary) # an integer
AttributeError: module 'crc32c' has no attribute 'value'
william-silversmith commented 4 years ago

There was an issue with google-resumable-media and google-crc32c where their versions were conflicting with crc32c.

https://github.com/googleapis/google-resumable-media-python/pull/162#issuecomment-682082258

Can you double check to see if your version of google-crc32c is at 1.0.0?