Systems tests are failing due to Google auth issue.

[jthomas]

All the Nimbella CLI tests which use the storage bucket started failing last week with this error:

# -- command failed --
# status : 1
# output (3 lines):
#    ›   Error: Unable to detect a Project Id in the current environment. 
#    ›   To learn more about authentication and Google APIs, visit: 
#    ›   https://cloud.google.com/docs/authentication/getting-started
# --

This error is thrown from here: https://github.com/googleapis/google-auth-library-nodejs/blob/0dfb42962bf364dccd00205741a4cf387aec619b/src/auth/googleauth.ts#L213-L217

I cannot replicate the error locally and debugging this yields no further clues as to why it is failing in the CI/CD (and why it started...). I've tried setting the PROJECT env var manually as a workout but no joy.

I'm going to have to set up a minimal test example using the library and run that on Github CI/CD in hope of replicating the bug at least...

[jthomas]

Workout: If I correctly pass through the env var to manually set the Google Project Id (https://github.com/nimbella/nimbella-cli/commit/21e9a1c03d1889bb02a069cce7c594c02296a690) - it works again. It's not clear why the project Id isn't picked up from the credentials (as it's passed in) and works in other local builds.

I haven't seen or heard anyone else report this issue apart from in the CI/CD pipeline. It's difficult to debug due to not having remote access to the Github Actions envs. I'd tried various things to reproduce it but had no luck.

I'm going to keep this issue open in case it comes up again but don't want to spend more time than I already have on this if we aren't witnessing it anywhere else. The workout will be enough for now.

[bhageena]

I am able to reproduce this error in portal deployment after clearing out _nodemodules and package-lock.json.

Suspecting a change in recent releases of @google-cloud/storage to have caused this error, attempting pinning it to version 5.8.3 It is resolving to 5.10.0 in the portal.

portal@1.0.0 /home/crai/projects/portal
└─┬ nimbella-cli@1.16.0
  └─┬ @nimbella/nimbella-deployer@1.18.0
    ├─┬ @nimbella/sdk@1.3.1
    │ └─┬ @nimbella/storage@0.0.2
    │   └── @google-cloud/storage@5.10.0  deduped
    └─┬ @nimbella/storage@0.0.3
      └── @google-cloud/storage@5.10.0 

Also the latest release of @nimbella/storage, 0.0.4 had unpinned dependency whereas in the master branch it is pinned to 5.8.3 We might need to make a fresh release of @nimbella/storage from master.

[bhageena]

Added explicit dependency of @google-cloud/storage 5.8.3, in nimbella-cli, tried deploying portal but still failed. Storage dependency resolving to:

nimbella-cli@1.17.0 /home/crai/projects/nim/nimbella-cli
├── @google-cloud/storage@5.8.3 
└─┬ @nimbella/nimbella-deployer@2.2.0
  ├─┬ @nimbella/sdk@1.3.1
  │ └─┬ @nimbella/storage@0.0.2
  │   └── @google-cloud/storage@5.8.3  deduped
  └─┬ @nimbella/storage@0.0.4
    └── @google-cloud/storage@5.8.3  deduped

Deployment of ocr demo also failed, whereas calc passed. Again ocr with striped bucket specs passed. It has to do with the bucket specs in project.yml, a project with bucket specs is failing.

[jthomas]

@bhageena Do you think you can get a minimal working test case (which fails with the error) working locally to share? I did try to do this but never got the error to occur. From my investigations - the error is thrown deep in the google auth library.

[bhageena]

requires new release of storage https://github.com/nimbella/storage/pull/4

[bhageena]

I tried putting together a minimal script to run independently and reproduce the issue, but it's now insignificant as, we are able to reproduce it as in nimbella/storage#4

const { getStorageProvider } = require('@nimbella/storage')
const { authPersister, getCredentials } = require('@nimbella/nimbella-deployer')
const { readFileSync } = require('fs')

async function upload() {
    credentials = await getCredentials(authPersister)
    const provider = getStorageProvider('@nimbella/storage-gcs')
    const client = provider.getClient(credentials.namespace, credentials.ow.apihost, true, credentials)
    const fileTOUpload = require.resolve('./test.html')
    uploadStream = readFileSync(fileTOUpload)
    const remoteFile = client.file('test/test.html')
    await remoteFile.save(uploadStream, { contentType: 'text/html' })
}

upload().then(console.log).catch(console.log) 

[jthomas]

I've now released v0.0.5 of the storage library. This will close this issue as the patch update is automaticaly picked up by the deployer library (and therefore the CLI). I'll close this as "fixed" unless we find out otherwise. https://github.com/nimbella/storage/releases/tag/v0.0.5