search

tensorflow / cloud

The TensorFlow Cloud repository provides APIs that will allow to easily go from debugging and training your Keras and TensorFlow code in a local environment to distributed training in the cloud.
https://github.com/tensorflow/cloud
Apache License 2.0
377 stars 91 forks source link

Allow passing in Google Cloud Credentials (instead of application default credentials) #51

Open vimota opened 4 years ago

vimota commented 4 years ago

Kaggle Notebooks supports integrations with several Google Cloud services by authenticating the user via OAuth and providing a credentials object (for example https://www.kaggle.com/product-feedback/163416).

We'd like to provide support for tensorflow_cloud on Kaggle using user's existing auth via this mechanism. One way to do that would be allowing the library to have a credential object passed in on initialization (like other Google Cloud client libraries).

Another approach, which I've sort of hacked together below[1] is to take the credential object and write it to the filesystem as the GOOGLE_APPLICATION_CREDENTIALS file. This actually works and is implemented by the google_auth library but it always returns a None project id (https://github.com/googleapis/google-auth-library-python/blob/9058f1fea680613d9717a62ee37dc294c11b9c8a/google/auth/_default.py#L126) so tensorflow_cloud throws an error: https://github.com/tensorflow/cloud/blob/dd5c18856fcae4948f638392488e94adce90577f/tensorflow_cloud/gcp.py#L27. Would it be possible to support this approach, but have another mechanism to pass in the project id, in a constructor for example?

Is there any constraint that disallows using user credentials (and requires service accounts to be used)?

[1]

import json
import os
from kaggle_secrets import UserSecretsClient
user_secrets = UserSecretsClient()
user_credential = user_secrets.get_gcloud_credential()
with open('/tmp/key.json', 'w', encoding='utf-8') as f:
    f.write(str(user_credential))
os.environ['GOOGLE_APPLICATION_CREDENTIALS']="/tmp/key.json"
!gcloud config set project vimota-project2
5shreya commented 1 month ago

Hey I would like to work on this issue. Can you assign me