search

fsspec / filesystem_spec

A specification that python filesystems should adhere to.
BSD 3-Clause "New" or "Revised" License
1.03k stars 360 forks source link

google DRIVE implementation #98

Open rabernat opened 5 years ago

rabernat commented 5 years ago

Most of us university folks have essentially unlimited free storage in google drive. It would be great to be able to work with this storage space in a cloud-native way.

The google drive SDK includes a python API: https://developers.google.com/drive/api/v3/about-sdk

In principle, it should be straightforward to create a google-drive implementation for fsspec.

I believe this would open up a whole world of cool possibilities. In particular, it would lower the bar for putting data on the "cloud." More people are comfortable with google drive than they are with S3 or GCS.

martindurant commented 5 years ago

Certainly a good idea! Without reading the docs, I imagine it's not too hard; I wonder how we would go about testing this? I would hate to have to invoke vcrpy...

rabernat commented 5 years ago

Are you trolling me Martin?!? Since when was test coverage ever a dealbreaker for fsspec? 😈

But seriously, isn't it the same issue for all of the cloud services (credentials needed to access the service)?

martindurant commented 5 years ago

No! There are things like moto and azurite that are miles easier to use than vcrpy. In this repo, we use docker for testing ssh and webhdfs (and dask tests hdfs for us). Coverage isn't too bad considering hdfs and fuse don't show up here...

andersy005 commented 3 years ago

Presuming https://github.com/intake/gdrivefs was created in response to this issue, I was wondering if this issue is still relevant and/or whether it is safe to close it?