Google-v2 gcs fuse support

DataBiosphere / dsub

Open-source command-line tool to run batch computing tasks and workflows on backend services such as Google Cloud.

Apache License 2.0

265 stars 44 forks source link

Google-v2 gcs fuse support #115

Closed eap closed 5 years ago

eap commented 6 years ago

Some of our internal users have expressed interest in adding support for GCS FUSE mounting of buckets. The pipelines team has published an example of doing this in the v2alpha1 API. It should be possible to do this with dsub's "google-v2" provider.

mbookman commented 6 years ago

Cloud Storage Fuse support was included for the google-v2 provider in release 0.2.1. Will look into support for the local provider as well. Will not be supported in the (deprecated) google provider.

mbookman commented 5 years ago

Rather than supporting gcsfuse in the local provider, we have added support for mounting a local directory with the --mount flag. This enables local development and testing prior to scaling up running in cloud. This was added to the local provicer in release 0.2.4.