This new issue consolidates several existing issues.
Currently there are 4 independent sets of credentials used by Google Cloud.
We use the gcloud commandline tool in at least one place, adding ssh keys to instances (since ansible gce module does not edit metadata after an instance is started). This uses credentials set by "gcloud auth login".
We also use google.cloud.storage, which relies on the application default credentials ("gcloud auth application-default login").
Ansible relies on the service account provided in Loom settings, and passes these credentials to the underlying libcloud libraries.
The Loom server and workers always use the default service account, since ansible does not support setting the service account on an instance.
To fix this:
Eliminate use of ansible. Work directly with libcloud instead. That allows us to stop using gcloud for setting metadata and lets us correctly set service accounts on new instances (items 1 and 4 above)
Stop using google.cloud libraries in the client. To handle storage, force the client to post files directly to the server or to authenticated URLs generated by the server. If google.cloud libraries can't be used without first running "gcloud auth application-default log" in the shell, eliminate them completely, and use libcloud for the server to query storage and generate signed URLs.
This new issue consolidates several existing issues.
Currently there are 4 independent sets of credentials used by Google Cloud.
To fix this: