gdm autoscale template onboards devices as standalone - error no such object

Do you already have an issue opened with F5 support?

Github Issues are consistently monitored by F5 staff, but should be considered as best effort only and you should not expect to receive the same level of response as provided by F5 Support. Please open an case with F5 if this is a critical issue.

Description

Autoscale template does not place BIG-IP devices in cluster. It fails onboarding of the autoscale script. The devices end up as standalone.

YAML used...

imports:
- path: f5-payg-autoscale-bigip-waf.py
resources:
- name: f5-payg-autoscale-bigip-waf
  type: f5-payg-autoscale-bigip-waf.py
  properties:
   region: us-west1
   availabilityZone1: us-west1-b
   mgmtNetwork: jgiroux-net-mgmt
   mgmtSubnet: jgiroux-subnet-mgmt
   imageName: f5-bigip-15-1-0-2-0-0-9-payg-best-25mbps-200321032524
   instanceType: n1-standard-8
   manGuiPort: 8443
   bigIpModules: ltm:nominal,asm:nominal
   serviceAccount: svc-jgiroux@xxxxx.iam.gserviceaccount.com
   targetSize: 2
   minReplicas: 2
   maxReplicas: 4
   cpuUtilization: 0.8
   coolDownPeriod: 600
   policyLevel: high
   applicationPort: 80
   applicationDnsName: httpbin.org
   allowUsageAnalytics: yes
   declarationUrl: default

Error in logs

2020-06-04T18:11:25.402Z info: [pid: 18357] [scripts/runScript.js] 2020-06-04T18:11:25.402Z silly: [pid: 32457] [lib/util.js] typeof err object 
2020-06-04T18:11:25.402Z info: [pid: 18357] [scripts/runScript.js] 2020-06-04T18:11:25.402Z verbose: [pid: 32457] [lib/util.js] tryUntil error: No such object: f5-bigip-jg-f5-autoscale1-89559/public_keys/jg-f5-autoscale1-bigip-4p4h tries left: 1 
2020-06-04T18:11:25.402Z silly: [pid: 32457] [lib/util.js] tryUntil: retryOrReject: numRemaining: 1 , code: 404 , message: No such object: f5-bigip-jg-f5-autoscale1-89559/public_keys/jg-f5-autoscale1-bigip-4p4h 
2020-06-04T18:11:27.458Z info: [pid: 18357] [scripts/runScript.js] 2020-06-04T18:11:27.458Z info: [pid: 32457] [lib/gceCloudProvider.js] getData error No such object: f5-bigip-jg-f5-autoscale1-89559/public_keys/jg-f5-autoscale1-bigip-4p4h 
2020-06-04T18:11:27.499Z info: [pid: 18357] [scripts/runScript.js] 2020-06-04T18:11:27.499Z silly: [pid: 32457] [lib/util.js] tryUntil: got error {"message":"No such object: f5-bigip-jg-f5-autoscale1-89559/public_keys/jg-f5-autoscale1-bigip-4p4h","name":"ApiError","stack":"ApiError: No such object: f5-bigip-jg-f5-autoscale1-89559/public_keys/jg-f5-autoscale1-bigip-4p4h\n    at Object.parseHttpRespMessage (/config/cloud/gce/node_modules/@f5devcentral/f5-cloud-libs-gce/node_modules/@google-cloud/storage/node_modules/@google-cloud/common/src/util.js:155:33)\n    at Object.handleResp (/config/cloud/gce/node_modules/@f5devcentral/f5-cloud-libs-gce/node_modules/@google-cloud/storage/node_modules/@google-cloud/common/src/util.js:130:18)\n    at Duplexify.self.requestStream.on.on.res (/config/cloud/gce/node_modules/@f5devcentral/f5-cloud-libs-gce/node_modules/@google-cloud/storage/src/file.js:579:21)\n    at emitOne (events.js:116:13)\n    at Duplexify.emit (events.js:211:7)\n    at emitOne (events.js:116:13)\n    at DestroyableTransform.emit (events.js:211:7)\n    at onResponse (/config/cloud/gce/node_modules/@f5devcentral/f5-cloud-libs-gce/node_modules/retry-request/index.js:192:19)\n    at Request.<anonymous> (/config/cloud/gce/node_modules/@f5devcentral/f5-cloud-libs-gce/node_modules/retry-request/index.js:147:11)\n    at emitOne (events.js:116:13)","code":404,"errors":[],"response":{"statusCode":404,"headers":{"x-guploader-uploadid":"AAANsUntU1P8YkNMskmJT73jFJxoDHvbh-GXIvr1NkBg7TqAs_ZtYlwOlK0CKqQXQHL7jlxsG1XotfsEa_jcWneEclNEORil_A","content-type":"text/html; charset=UTF-8","date":"Thu, 04 Jun 2020 18:11:27 GMT","vary":"Origin, X-Origin","expires":"Thu, 04 Jun 2020 18:11:27 GMT","cache-control":"private, max-age=0","content-length":"87","server":"UploadServer"},"request":{"uri":{"protocol":"https:","slashes":true,"auth":null,"host":"www.googleapis.com","port":null,"hostname":"www.googleapis.com","hash":null,"search":"?alt=media","query":"alt=media","pathname":"/storage/v1/b/f5-bigip-jg-f5-autoscale1-89559/o/public_keys%2Fjg-f5-autoscale1-bigip-4p4h","path":"/storage/v1/b/f5-bigip-jg-f5-autoscale1-89559/o/public_keys%2Fjg-f5-autoscale1-bigip-4p4h?alt=media","href":"https://www.googleapis.com/storage/v1/b/f5-bigip-jg-f5-autoscale1-89559/o/public_keys%2Fjg-f5-autoscale1-bigip-4p4h?alt=media"},"method":"GET","headers":{"Accept-Encoding":"gzip","User-Agent":"gcloud-node-storage/1.7.0","x-goog-api-client":"gl-node/8.11.1 gccl/1.7.0","Authorization":"Bearer ya29.c.Km3NB4ajd0oSfZLty0bdqNr95M6qEdOPzMjqkWJuwnwMHxhyqzdZoQPJLuBd0ndPxyzZSiHYWI2RHLT03N-x2D2er1kBTtZCCYRd8zETEMctO3rYCZkU6kxB4Swp0ih37mNNIqNSQ3H0fawAnjvZ"}}}} 
2020-06-04T18:11:27.539Z info: [pid: 18357] [scripts/runScript.js] 2020-06-04T18:11:27.539Z silly: [pid: 32457] [lib/util.js] typeof err object 
2020-06-04T18:11:27.540Z info: [pid: 18357] [scripts/runScript.js] 2020-06-04T18:11:27.539Z verbose: [pid: 32457] [lib/util.js] tryUntil error: No such object: f5-bigip-jg-f5-autoscale1-89559/public_keys/jg-f5-autoscale1-bigip-4p4h tries left: 0 
2020-06-04T18:11:27.539Z silly: [pid: 32457] [lib/util.js] tryUntil: retryOrReject: numRemaining: 0 , code: 404 , message: No such object: f5-bigip-jg-f5-autoscale1-89559/public_keys/jg-f5-autoscale1-bigip-4p4h 
2020-06-04T18:11:27.539Z verbose: [pid: 32457] [lib/util.js] Max tries reached.

Template

https://github.com/F5Networks/f5-google-gdm-templates/tree/master/experimental/autoscale/waf/via-lb/existing-stack/payg

Severity Level

This appears to only affect versions 15.x. I tried 14.x and 13.x and both the latter versions successfully clustered.

Here's the 15.0.1 output. Different messages from 15.1.0.2...but still...not cluster.

2020-06-04T18:48:11.923Z info: [pid: 29117] [scripts/autoscale.js] Cluster action UPDATE 
2020-06-04T18:48:11.924Z debug: [pid: 29117] [lib/bigIp.js] list localhost /tm/cm/trust-domain/Root 
2020-06-04T18:48:11.940Z debug: [pid: 29117] [lib/bigIp.js]  {"kind":"tm:cm:trust-domain:trust-domainstate","name":"Root","fullPath":"Root","generation":789,"selfLink":"https://localhost/mgmt/tm/cm/trust-domain/Root?ver=15.0.1","caCert":"/Common/dtca.crt","caCertReference":{"link":"https://localhost/mgmt/tm/cm/cert/~Common~dtca.crt?ver=15.0.1"},"caCertBundle":"/Common/dtca-bundle.crt","caCertBundleReference":{"link":"https://localhost/mgmt/tm/cm/cert/~Common~dtca-bundle.crt?ver=15.0.1"},"caDevices":["/Common/jg-f5-autoscale150-bigip-46l6.c.f5-4136-mspteam-dev.internal"],"caDevicesReference":[{"link":"https://localhost/mgmt/tm/cm/device/~Common~jg-f5-autoscale150-bigip-46l6.c.f5-4136-mspteam-dev.internal?ver=15.0.1"}],"caKey":"/Common/dtca.key","caKeyReference":{"link":"https://localhost/mgmt/tm/cm/key/~Common~dtca.key?ver=15.0.1"},"status":"standalone","trustGroup":"/Common/device_trust_group","trustGroupReference":{"link":"https://localhost/mgmt/tm/cm/device-group/~Common~device_trust_group?ver=15.0.1"}}
2020-06-04T18:48:11.941Z info: [pid: 29117] [scripts/autoscale.js] This instance is not in cluster. Requesting join. 
2020-06-04T18:48:11.942Z silly: [pid: 29117] [scripts/autoscale.js] Join request is too soon after last join request. 32509 ms 
2020-06-04T18:48:11.943Z info: [pid: 29117] [scripts/autoscale.js] Checking for messages 
2020-06-04T18:48:14.119Z debug: [pid: 29117] [scripts/autoscale.js] Handling 0 message(s) 
2020-06-04T18:48:14.124Z info: [pid: 29117] [lib/util.js] Autoscale finished. 
2020-06-04T18:48:14.120Z silly: [pid: 29117] [scripts/autoscale.js] number of messages to read: 
2020-06-04T18:48:14.121Z silly: [pid: 29117] [scripts/autoscale.js] number of read messages:

These versions work fine... f5-bigip-13-1-3-2-0-0-4-payg-best-1gbps-20191105210022 f5-bigip-14-1-2-3-0-0-5-payg-best-1gbps-191218142340

These versions do not cluster... f5-bigip-15-0-1-0-0-11-payg-best-25mbps-190803012348 f5-bigip-15-1-0-2-0-0-9-payg-best-25mbps-200321032524

F5Networks / f5-google-gdm-templates