Importing keyfile on heroku

[leantony]

Iam deploying an app to heroku, and am using the google cloud storage bucket. All works well on my local pc, but when I deploy the app to heroku, and set the config vars, it doesnt work. I've seen that the auth client expects an absolute file path to the keyfile that contains the credentials. For platforms such as heroku, it seems abit of a challenge, since I can't commit add the keyfile to git How do I get around this?

[jdpedrie]

Hi @leantony,

One option is to store the contents of your credentials keyfile in a Heroku configuration variable.

Your code in such a case would look something like this: (Assuming a keyfile configuration variable named GOOGLE_CREDENTIALS.

use Google\Auth\CredentialsLoader;

$credentials = CredentialsLoader::makeCredentials($scope, json_decode(getenv('GOOGLE_CREDENTIALS'), true));

[leantony]

I have done that but keep getting an error. Okay, I pasted the json content as the variable. Then gave it a key name. GOOGLE_APPLICATION_CREDENTIALS The exception says that a file wasn't found

[leantony]

How do I really provide a path to a file in heroku. Unless I make the file available on git during commit, which I don't think is okay, since this is are credentials

[jdpedrie]

Would you mind sharing the code you're using? (With any sensitive information removed of course)

You are right that committing the keyfile to git isn't the right approach.

[tmatsuo]

One trick we're using for travis build is to base64 encode the file and set the encoded string as a secret environment variable like "GOOGLE_CREDENTIALS_BASE64", then use a script like this to dump it to a file on the server.

[bshaffer]

@leantony the variable GOOGLE_APPLICATION_CREDENTIALS is a reserved environment variable that is supposed to be the path to a json file. The suggestion here is to not use a file at all, but instead load the JSON contents of your credentials file directly from an environment variable. Something like this (using the environment variable GOOGLE_CREDENTIALS) should work:

use Google\Auth\CredentialsLoader;
use Google\Auth\Middleware\AuthTokenMiddleware;
use GuzzleHttp\Client;
use GuzzleHttp\HandlerStack;

$scope = 'https://www.googleapis.com/auth/taskqueue';
$credentials = CredentialsLoader::makeCredentials($scope, json_decode(getenv('GOOGLE_CREDENTIALS'), true));

$stack = HandlerStack::create();
$stack->push(new AuthTokenMiddleware($credentials));

$client = new Client([
    'handler' => $stack,
    'base_uri' => 'https://www.googleapis.com/taskqueue/v1beta2/projects/',
    'auth' => 'google_auth' // authorize all requests
]);

$res = $client->get('myproject/taskqueues/myqueue');

[leantony]

Its a laravel app. Am using this package => https://github.com/Superbalist/laravel-google-cloud-storage to push files to google cloud. Its only a service provider. Anyway, It makes use of version 0.9 of google auth php library. Here is the config on heroku. sample Then, the error afterwards. As you can see, the environment variable is loaded

[jdpedrie]

What Brent was saying is that GOOGLE_APPLICATION_CREDENTIALS cannot be used for your keyfile contents. If GOOGLE_APPLICATION_CREDENTIALS is set, it must be a path to the location of the keyfile.

Change the name of your configuration variable (to GOOGLE_CREDENTIALS, or anything else you want) and try again. If you're having trouble getting things working with the laravel provider you're using, share your code and we'll try and help.

[leantony]

Okay, the service provider expects a config variable named GOOGLE_CLOUD_KEY_FILE that points to an absolute path to the json credentials file. That works on my machine On heroku however, since I cant really provide a path to the file (or rather don't know how to) I just used the same variable with the value as the json string itself That failed. I get an exception that the file cannot be found. From /app/vendor/google/cloud/src/ClientTrait.php line 82 So I did abit of digging and found that the auth client can also work with an environment variable named GOOGLE_APPLICATION_CREDENTIALS. Did the same, but didnt work too. On my local pc, I have this in the .env file

GOOGLE_CLOUD_KEY_FILE="/home/google_auth_key.json"

I think ill have to try @tmatsuo's approach for dumping the file as base 64.But use a laravel command to dump the file at the root of the project Then refrence it as a path on heroku config like this

GOOGLE_CLOUD_KEY_FILE="google_auth_key.json"

Would this work?

[jdpedrie]

Writing the keyfile to disk from a heroku config var should work, yes.

[lorelei522]

@leantony What did you end up doing? I am having the same problem. I need to deploy to Heroku and I am using the Google Vision API and I have the JSON file.

Please let me know

[mgnaresh]

I would also be very intrested in heroku + bigquery

Not yet tried heroku + bigquery as my local machine it self has an issue.

Is Python+BigQuery really working ?

Error processing line 3 of /Library/Python/2.7/site-packages/google_cloud_bigquery-0.22.1-py2.7-nspkg.pth:

Traceback (most recent call last): File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site.py", line 152, in addpackage exec line File "", line 1, in KeyError: 'google'

[wapnen]

@lorelei522 hey did you get it to work? how?

[wapnen]

@tmatsuo i did what you suggested, i get the following error GoogleException in ClientTrait.php line 120: Given keyfile was invalid

[fnmendez]

With my partner @agmardones we're developing a Koa application with Google Cloud Storage services. To get it work we only needed to set private_key and client_email from the json. In Heroku, under GOOGLE_PRIVATE_KEY env var, we inserted the key value exactly as in the json but without quotes. Finally, to import them we used the following solution that a classmate proposed.

module.exports = {
  private_key: process.env.GOOGLE_PRIVATE_KEY.replace(/\\n/g, '\n'),
  client_email: process.env.GOOGLE_CLIENT_EMAIL,
}

Hope it helps.

[abhibeta]

I am looking for solution too , how to use google api key in heroku, nothing is working

[arokanto]

In case you need to access Google Analytics API, I finally got this to work without having to write anything to disk on Heroku. This solution should apply to other use cases as well with small modifications.

Follow the instructions on Google's own Hello Analytics page here: https://developers.google.com/analytics/devguides/reporting/core/v3/quickstart/service-php

Set up three environment variables:

• GOOGLE_API_PRIVATE_KEY_BASE64
• GOOGLE_API_CLIENT_EMAIL
• GOOGLE_API_CLIENT_ID

Populate them with values from your service-account-credentials.json file. The private key should be copied without the surrounding quotes as mentioned by @fnmendez and then converted to base64. On MacOS and probably Linux you can do it with this command: openssl base64 -A -in <in file> -out <out file>. Note that the base64 conversion is optional but makes your env a bit less ugly, although it's still pretty ugly with that base64 vomit.

Then replace the initializeAnalytics function with the following:

function initializeAnalytics() {
    $private_key = base64_decode( getenv('GOOGLE_API_PRIVATE_KEY_BASE64') );
    $private_key = str_replace('\n', "\n", $private_key);

    $client_parameters = array(
        'client_email'        => getenv('GOOGLE_API_CLIENT_EMAIL'),
        'signing_algorithm'   => 'HS256',
        'signing_key'         => $private_key
    );

    $client = new Google_Client( $client_parameters );
    $client->useApplicationDefaultCredentials();
    $client->setClientId( getenv('GOOGLE_API_CLIENT_ID') ); 
    $client->setScopes(['https://www.googleapis.com/auth/analytics.readonly']);
    $analytics = new Google_Service_Analytics($client);

    return $analytics;
}

The gist here is that you there is no "setPrivateKey" or "setClientEmail" method in Google_Client class, but you can pre-populate those in the constructor.

[tybro0103]

If you're landing here because of this same issue, but with firebase & heroku, see this: https://firebase.google.com/docs/reference/admin/node/admin.credential.html#cert

[jacobnuno]

@abhibeta @wapnen @leantony I did it as @jdpedrie said it before. I store the contents of my credentials keyfile in a Heroku configuration variable.

So in my laravel configuration I load the JSON content of my credentials file directly from an environment variable as @bshaffer recommended json_decode(env('GCS_CREDENTIALS'), true)

   'gcs' => [
            'driver' => 'gcs',
            'project_id' => env('GOOGLE_CLOUD_PROJECT_ID'),
            'key_file' => json_decode(env('GCS_CREDENTIALS'), true),
            'bucket' => env('GOOGLE_CLOUD_STORAGE_BUCKET'),
            'path_prefix' => env('GOOGLE_CLOUD_STORAGE_PATH_PREFIX', null),
            'storage_api_uri' => env('GOOGLE_CLOUD_STORAGE_API_URI', null), 
            'visibility' => 'public', 
        ],

I use the laravel package https://github.com/Superbalist/laravel-google-cloud-storage

[fabswt]

I don't want to use a credentials file, because I don't want to commit credentials to git. Instead, I want to use an environment variable as it's easy to set both for local dev and production on Heroku, and won't end up in git.

However, the Google Cloud is forcing us to use a JSON file. So here's how I did it:

First, encoded the content of the credentials file provided by Google Cloud's console. This way we can store it in .env locally and as as config var on Heroku Ended up encoding the JSON as base64, because flattening the file was not enough*.

/**
 * base64-encode the content of a JSON file.
 *
 * This is used to encode Google Cloud credential files, for storage inside
 * .env or Heroku's config vars.
 *
 * KEEP IT! Do NOT delete!
 */
$jsonpath = ABSPATH .  "Testing-a6cdf0d82200.json";
$jsonstring = file_get_contents( $jsonpath);
$encoded = base64_encode( $jsonstring );
echo $encoded;
exit;
// copy the result in .env or set it on Heroku

Then decoded the environment variable, for use with the Google Cloud SDK client (here is an example with Google Translate):

if ( ! getenv('GOOGLE_CREDENTIALS') ) {
    throw new ErrorException( "Missing environment variable GOOGLE_CREDENTIALS" );
}

$jsonstring = base64_decode( getenv('GOOGLE_CREDENTIALS') );
$credentials = json_decode( $jsonstring, true );

$translationServiceClient = new TranslationServiceClient(
    ['credentials' => $credentials]
);

/** Uncomment and populate these variables in your code */
// $text = 'Hello, world!';
// $targetLanguage = 'fr';
// $projectId = 'some-project-1234'; // can't i get rid of it?
$contents = [$text];
$formattedParent = $translationServiceClient->locationName($projectId, 'global');

try {
    $response = $translationServiceClient->translateText(
        $contents,
        $targetLanguage,
        $formattedParent
    );
    // Display the translation for each input text provided
    foreach ($response->getTranslations() as $translation) {
        printf('Translated text: %s' . PHP_EOL, $translation->getTranslatedText());
    }
} finally {
    $translationServiceClient->close();
}

Took a while, but at least now it's all set up.

Hopefully this'll help someone!

---

*[Without base64 encoding: removing spaces and line-breaks from the JSON would break the private key with BEGIN PRIVATEKEY becoming BEGINPRIVATEKEY, while keepig the spaces lead to my app not being able to load the .env]:

[huzaifaiftikhar]

1. Convert the firebase config JSON (serviceAccountKey) to base64 encoded string, this can be done in many ways like using openssl command openssl base64 -in <firebaseConfig.json> -out <firebaseConfigBase64.txt> or using Nodejs as illustrated below

   Buffer.from(JSON.stringify({
   "type": "",
   "project_id": "",
   "private_key_id": "",
   "private_key": "",
   "client_email": "",
   "client_id": "",
   "auth_uri": "",
   "token_uri": "",
   "auth_provider_x509_cert_url": "",
   "client_x509_cert_url": ""
   })).toString('base64')

2. Save the output base64 encoded string of the above step in an env variable like GOOGLE_CONFIG_BASE64. (Save it to ConfigVars in case of Heroku)
3. Initialise the firebaseSdk in your node application

   const firebaseAdminSdk = require('firebase-admin'),
     firebaseAdminApp = firebaseAdminSdk.initializeApp({credential: firebaseAdminSdk.credential.cert(
       JSON.parse(Buffer.from(process.env.GOOGLE_CONFIG_BASE64, 'base64').toString('ascii')))
   });