search

fsspec / gcsfs

Pythonic file-system interface for Google Cloud Storage
http://gcsfs.readthedocs.io/en/latest/
BSD 3-Clause "New" or "Revised" License
320 stars 141 forks source link

Question: aiohttp vs. gRPC API #625

Open keunhong opened 1 month ago

keunhong commented 1 month ago

Hello! I was curious if there was a reason gcsfs uses aiohttp with the JSON API to query GCS rather than using the Google Python client which calls the gRPC API. AFAIK the gRPC API is a lot more efficient.

martindurant commented 1 month ago

I don't think grpc-async was a thing when this repo switched to async.

Honestly, I doubt for the size of the encoded messages, it makes any difference at all, and actually json is pretty well optimized for python (whereas protobufs are certainly not). Any bytes payload (i.e., file down/uploads) would be the same in either model.