search

F5Networks / f5-azure-arm-templates-v2

Azure Resource Manager Templates for quickly deploying BIG-IP services in Azure
22 stars 45 forks source link

Deploying the BIG-IP VE in Azure - Fails #25

Closed hondocalrissian closed 1 year ago

hondocalrissian commented 1 year ago

Describe the bug

onBoarder script fails with the following error "Cannot set db variable (restjavad.useextramb = true)."

Expected behavior

Deployment to complete and have a working environment configured per the ARM template deployed

Current behavior

Deployment error is displayed in Azure Resource manager with the complete error being:

2023-07-25T19:28:30.842Z [20846]: info: Configuration file: /config/cloud/runtime-init.conf 2023-07-25T19:28:30.860Z [20846]: info: Validating provided declaration 2023-07-25T19:28:30.856Z [20846]: info: Processing controls parameters 2023-07-25T19:28:30.914Z [20846]: info: Successfully validated declaration 2023-07-25T19:28:30.917Z [20846]: info: Executing custom pre_onboard_enabled commands 2023-07-25T19:28:30.929Z [20846]: info: Executing inline shell command: /usr/bin/setdb provision.extramb 1000 2023-07-25T19:28:31.046Z [20846]: info: Shell command: /usr/bin/setdb provision.extramb 1000 execution completed; response: 2023-07-25T19:28:31.050Z [20846]: info: Executing inline shell command: /usr/bin/setdb restjavad.useextramb true 2023-07-25T19:28:31.157Z [20846]: error: Command failed: /usr/bin/setdb restjavad.useextramb true Cannot set db variable (restjavad.useextramb = true).

2023-07-25T19:28:31.158Z [20846]: info: Sending F5 Teem report for failure case. 2023-07-25T19:28:41.846Z [20846]: info: {"id":"98ad5c92-e50e-4f42-c256a986cb3c","product":"BIG-IP","cpuCount":8,"diskSize":41984,"memoryInMb":32176,"version":"15.1.9.1","nicCount":3,"regKey":"TYQMM-LOPNP-VXQMW-GPQCP-VDJRGXU","platformId":"Z100","hostname":"bigip1","management":"10.X.X.X/24","provisionedModules":{"ltm":"nominal"},"installedPackages":{},"environment":{"pythonVersion":"Python 2.7.5","pythonVersionDetailed":"2.7.5 (default, Jul 11 2023, 10:14:43) \n[GCC 4.8.5 20150623 (Red Hat 4.8.5-16)]","nodeVersion":"v6.9.1","libraries":{"ssh":"OpenSSH_7.4p1, OpenSSL 1.0.2s-fips 28 May 2019"}}} 2023-07-25T19:28:51.561Z [20846]: info: F5 Teem report was successfully sent for failure case. 2023-07-25T19:28:51.562Z [20846]: info: Command failed: /usr/bin/setdb restjavad.useextramb true Cannot set db variable (restjavad.useextramb = true).

Possible solution

I suspect that this issue may be version related. The version attempting to be deployed is 15.1.9.1

Steps to reproduce

Use the Azure deploy button at the link https://github.com/F5Networks/f5-azure-arm-templates-v2/tree/main/examples/failover

Fill out the template info as requested

Screenshots

Screenshot 2023-07-26 at 11 10 14 AM

Context

I am trying to deploy a HA pair per the Azure deploy template with 3 NIC's. Attempting to replicate a pre-existing environment in one Azure region to another.

Your Environment

mikeshimkus commented 1 year ago

@starkeyjhonnold Hi, the template link you shared is from the v1 ARM template project, have you tried the v2 failover template?

hondocalrissian commented 1 year ago

Sorry. My fault. I copy and pasted the wrong link. I corrected the comment. This error is seen with the v2 template.

I have been through the process several times and it fails at the same location everytime.

mikeshimkus commented 1 year ago

I created internal issue #288 to track updating the templates to use the correct db variables for newer versions of BIG-IP.

In the meantime, please try again. I made a change to the template in the default artifact location that should allow you to deploy.

hondocalrissian commented 1 year ago

@shyawnkarim

Good day.
I tried again several minutes ago and am getting the same error in the onBoarder portion. Take a look and let me know if I can get around this error or not.

Here is the error output. <= {"code":"DeploymentFailed","message":"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.","details":[{"code":"VMExtensionProvisioningError","message":"VM has reported a failure when processing extension 'onboarder' (publisher 'Microsoft.Azure.Extensions' and type 'CustomScript'). Error message: \"Enable failed: failed to execute command: command terminated with exit status=1\n[stdout]\ninit-azure/src/lib/cloud/cloudFactory.d.ts\n./f5-bigip-runtime-init-azure/src/lib/cloud/cloudFactory.js\n./f5-bigip-runtime-init-azure/src/lib/cloud/gcp\n./f5-bigip-runtime-init-azure/src/lib/cloud/gcp/cloudClient.d.ts\n./f5-bigip-runtime-init-azure/src/lib/cloud/gcp/cloudClient.js\n./f5-bigip-runtime-init-azure/src/lib/logger.d.ts\n./f5-bigip-runtime-init-azure/src/lib/logger.js\n./f5-bigip-runtime-init-azure/src/lib/provider\n./f5-bigip-runtime-init-azure/src/lib/provider/secret\n./f5-bigip-runtime-init-azure/src/lib/provider/secret/hashicorpVaultClient.d.ts\n./f5-bigip-runtime-init-azure/src/lib/provider/secret/hashicorpVaultClient.js\n./f5-bigip-runtime-init-azure/src/lib/provider/secret/index.d.ts\n./f5-bigip-runtime-init-azure/src/lib/provider/secret/index.js\n./f5-bigip-runtime-init-azure/src/lib/resolver\n./f5-bigip-runtime-init-azure/src/lib/resolver/index.d.ts\n./f5-bigip-runtime-init-azure/src/lib/resolver/index.js\n./f5-bigip-runtime-init-azure/src/lib/resolver/resolverClient.d.ts\n./f5-bigip-runtime-init-azure/src/lib/resolver/resolverClient.js\n./f5-bigip-runtime-init-azure/src/lib/telemetry\n./f5-bigip-runtime-init-azure/src/lib/telemetry/index.d.ts\n./f5-bigip-runtime-init-azure/src/lib/telemetry/index.js\n./f5-bigip-runtime-init-azure/src/lib/telemetry/telemetryClient.d.ts\n./f5-bigip-runtime-init-azure/src/lib/telemetry/telemetryClient.js\n./f5-bigip-runtime-init-azure/src/lib/utils.d.ts\n./f5-bigip-runtime-init-azure/src/lib/utils.js\n./f5-bigip-runtime-init-azure/src/lib/validator.d.ts\n./f5-bigip-runtime-init-azure/src/lib/validator.js\n./f5-bigip-runtime-init-azure/src/schema\n./f5-bigip-runtime-init-azure/src/schema/base_schema.json\n./f5-bigip-runtime-init-azure/src/version\n74618 blocks\n2023-07-28T10:08:22 - Getting lastest AT metadata at https://cdn.f5.com/product/cloudsolutions/f5-extension-metadata/latest/metadata.json\n % Total % Received % Xferd Average Speed Time Time Time Current\n Dload Upload Total Spent Left Speed\n\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r100 46195 100 46195 0 0 129k 0 --:--:-- --:--:-- --:--:-- 130k\n2023-07-28T10:08:24 - Creating command utility.\n2023-07-28T10:08:24 - RPM installation is completed.\n2023-07-28T17:08:25.147Z [20756]: info: Configuration file: /config/cloud/runtime-init.conf\n2023-07-28T17:08:25.158Z [20756]: info: Processing controls parameters\n2023-07-28T17:08:25.159Z [20756]: info: Validating provided declaration\n2023-07-28T17:08:25.212Z [20756]: info: Successfully validated declaration\n2023-07-28T17:08:25.212Z [20756]: info: Executing custom pre_onboard_enabled commands\n2023-07-28T17:08:25.223Z [20756]: info: Executing inline shell command: /usr/bin/setdb provision.extramb 1000\n2023-07-28T17:08:25.339Z [20756]: info: Shell command: /usr/bin/setdb provision.extramb 1000 execution completed; response: \n2023-07-28T17:08:25.343Z [20756]: info: Executing inline shell command: /usr/bin/setdb restjavad.useextramb true\n2023-07-28T17:08:25.448Z [20756]: error: Command failed: /usr/bin/setdb restjavad.useextramb true\nCannot set db variable (restjavad.useextramb = true).\n\n2023-07-28T17:08:25.453Z [20756]: info: Sending F5 Teem report for failure case.\n2023-07-28T17:08:36.180Z [20756]: info: {\"id\":\"e47c88b7-fedf-5d4c-ff53eaa57c10\",\"product\":\"BIG-IP\",\"cpuCount\":8,\"diskSize\":41984,\"memoryInMb\":32176,\"version\":\"15.1.9.1\",\"nicCount\":3,\"regKey\":\"TYQMM-LOPNP-VXQMW-GPQCP-VDJRGXU\",\"platformId\":\"Z100\",\"hostname\":\"bigip1\",\"management\":\"10.X.X.X/24\",\"provisionedModules\":{\"ltm\":\"nominal\"},\"installedPackages\":{},\"environment\":{\"pythonVersion\":\"Python 2.7.5\",\"pythonVersionDetailed\":\"2.7.5 (default, Jul 11 2023, 10:14:43) \n[GCC 4.8.5 20150623 (Red Hat 4.8.5-16)]\",\"nodeVersion\":\"v6.9.1\",\"libraries\":{\"ssh\":\"OpenSSH_7.4p1, OpenSSL 1.0.2s-fips 28 May 2019\"}}}\n2023-07-28T17:08:46.714Z [20756]: info: F5 Teem report was successfully sent for failure case.\n2023-07-28T17:08:46.716Z [20756]: info: Command failed: /usr/bin/setdb restjavad.useextramb true\nCannot set db variable (restjavad.useextramb = true).\n\n\n[stderr]\n\". More information on troubleshooting is available at https://aka.ms/VMExtensionCSELinuxTroubleshoot. "}]} <=

mikeshimkus commented 1 year ago

Can you confirm that the contentVersion of the template you're using is 2.8.0.0? Example: https://github.com/F5Networks/f5-azure-arm-templates-v2/blob/002470aceb1192b0524495e5e393a18d958d4453/examples/failover/azuredeploy.json#L3

What value did you use for the templateBaseUrl parameter?

hondocalrissian commented 1 year ago

Thanks for the follow-up.

It seems the Azure Deploy in the Existing stack on the Github page is still using version 2.7.0.0. I will see what I can do to use version 2.8.0.0. If the Azure Deploy button link can be updated that would be great too!

"https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2FF5Networks%2Ff5-azure-arm-templates-v2%2Fv2.7.0.0%2Fexamples%2Ffailover%2Fazuredeploy-existing-network.json”

Let me know Thanks!

On Aug 1, 2023, at 10:11 AM, Mike Shimkus @.***> wrote:

Can you confirm that the contentVersion of the template you're using is 2.8.0.0? Example: https://github.com/F5Networks/f5-azure-arm-templates-v2/blob/002470aceb1192b0524495e5e393a18d958d4453/examples/failover/azuredeploy.json#L3

What value did you use for the templateBaseUrl parameter?

— Reply to this email directly, view it on GitHub https://github.com/F5Networks/f5-azure-arm-templates-v2/issues/25#issuecomment-1660523218, or unsubscribe https://github.com/notifications/unsubscribe-auth/A5Y6YTYYOVPL7WEWIHI7LM3XTEMD5ANCNFSM6AAAAAA2Y2OC3M. You are receiving this because you were mentioned.

hondocalrissian commented 1 year ago

@mikeshimkus @shyawnkarim

I retried with this link and verified that it was using version 2.8.0.0 https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2FF5Networks%2Ff5-azure-arm-templates-v2%2Fv2.8.0.0%2Fexamples%2Ffailover%2Fazuredeploy-existing-network.json

Same error:
2023-08-01T17:42:26.179Z [25751]: info: Command failed: /usr/bin/setdb restjavad.useextramb true Cannot set db variable (restjavad.useextramb = true).

hondocalrissian commented 1 year ago

@shyawnkarim @mikeshimkus I redeployed again using the 2.8.0.0 template and using "f5-networks:f5-big-ip-good:f5-bigip-virtual-edition-200m-good-hourly:15.1.801000".
It worked without issue. My previous deployments were using a later version of 15.1. "f5-networks:f5-big-ip-good:f5-bigip-virtual-edition-200m-good-hourly:15.1.901000"

15.1.8.1 is listed as a validated version for the runtime-init. https://github.com/F5Networks/f5-bigip-runtime-init#validated-big-ip-versions.

It seems a validated version for the runtime-init is the key. . .

shyawnkarim commented 1 year ago

Closing. This issue was resolved with Release 2.9.0.0.