Add tests for file processors with Google Cloud Storage backend

m3dev / gokart

Gokart solves reproducibility, task dependencies, constraints of good code, and ease of use for Machine Learning Pipeline.

https://gokart.readthedocs.io/en/latest/

MIT License

305 stars 57 forks source link

Add tests for file processors with Google Cloud Storage backend #363

Open mamo3gr opened 5 months ago

mamo3gr commented 5 months ago

relates https://github.com/m3dev/gokart/pull/361

We have tests for file processors with S3 backend. The ones with Google Cloud Storage backend would be needed.
There is moto, a good mocking library for S3, while there is no counterpart for GCS.
I found a good candidate cloud-storage-mocker but it only mocks GCS Python client, not GCS itself.