search

F5Networks / f5-aws-cloudformation-v2

AWS Cloudformation Templates for quickly deploying BIG-IP services in AWS
13 stars 22 forks source link

Template does not function with PAYG-Good #33

Closed sabooker closed 1 year ago

sabooker commented 1 year ago

I changed this value in the parameters file and do not get a successful boot. {Old "ParameterKey": "bigIpImage", "ParameterValue": "16.1.3.3-0.0.3PAYG-Best Plus 25Mbps"}{New "ParameterKey": "bigIpImage", "ParameterValue": "16.1.3.3-0.0.3PAYG-Good 25Mbps"},

I see these errors in the logs: startup-script.log 2023-03-28T16:10:43 - Getting lastest AT metadata at https://cdn.f5.com/product/cloudsolutions/f5-extension-metadata/latest/metadata.json % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed ^M 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0^M 0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0^M 0 0 0 0 0 0 0 0 --:--:-- 0:00:02 --:--:-- 0^M 0 0 0 0 0 0 0 0 --:--:-- 0:00:03 --:--:-- 0^M 0 0 0 0 0 0 0 0 --:--:-- 0:00:04 --:--:-- 0^M 0 0 0 0 0 0 0 0 --:--:-- 0:00:05 --:--:-- 0 Warning: Transient problem: timeout Will retry in 60 seconds. 3 retries left. ^M 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0^M 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0^M 0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0^M 0 0 0 0 0 0 0 0 --:--:-- 0:00:02 --:--:-- 0^M 0 0 0 0 0 0 0 0 --:--:-- 0:00:03 --:--:-- 0^M 0 0 0 0 0 0 0 0 --:--:-- 0:00:04 --:--:-- 0^M 0 0 0 0 0 0 0 0 --:--:-- 0:00:05 --:--:-- 0 Warning: Transient problem: timeout Will retry in 60 seconds. 2 retries left. ^M 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0^M 0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0^M 0 0 0 0 0 0 0 0 --:--:-- 0:00:02 --:--:-- 0^M 0 0 0 0 0 0 0 0 --:--:-- 0:00:03 --:--:-- 0^M 0 0 0 0 0 0 0 0 --:--:-- 0:00:04 --:--:-- 0^M 0 0 0 0 0 0 0 0 --:--:-- 0:00:05 --:--:-- 0 Warning: Transient problem: timeout Will retry in 60 seconds. 1 retries left. ^M 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0^M 0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0^M 0 0 0 0 0 0 0 0 --:--:-- 0:00:02 --:--:-- 0^M 0 0 0 0 0 0 0 0 --:--:-- 0:00:03 --:--:-- 0^M 0 0 0 0 0 0 0 0 --:--:-- 0:00:04 --:--:-- 0^M 0 0 0 0 0 0 0 0 --:--:-- 0:00:05 --:--:-- 0 curl: (28) Resolving timed out after 5515 milliseconds

ltm log Mar 28 16:10:17 ip-10-157-176-156 notice icr_eventd_schema_helper[13239]: 13239,13239, icr eventd schema helper, INFO,Transaction execution timeout is out of range [120..300].Resetting to 300 Mar 28 16:10:39 ip-10-157-176-156 warning icr_eventd_schema_helper[13239]: 01420010:4: General exception during query for initial master key value Mar 28 16:10:42 ip-10-157-176-156 info systemd-journal[1062]: Suppressed 421 messages from /system.slice/cloud-final.service Mar 28 16:10:43 ip-10-157-176-156 notice icr_eventd[7947]: 01a10012:5: Startup: MCP subscriber Mar 28 16:10:43 ip-10-157-176-156 notice icr_eventd[7947]: 01a10012:5: Startup: Invalidating down stream cache Mar 28 16:10:43 ip-10-157-176-156 notice icr_eventd[7947]: 01a10012:5: Startup: Beginning MCP message processing Mar 28 16:10:46 ip-10-157-176-156 warning chmand[4334]: 012a0004:4: CHMAN request (from get_dossier) for tag:19 failed Mar 28 16:10:49 ip-10-157-176-156 err chmand[4334]: 012a0003:3: DossierReq exception: VirtDossier Service: Instance identity retrieval from the metadata failed. Check network connectivity to the instance metadata before retrying Mar 28 16:10:49 ip-10-157-176-156 warning get_dossier[13331]: 012a0004:4: hal_request_dossier: request failed Mar 28 16:10:49 ip-10-157-176-156 err get_dossier[13331]: 01170003:3: halGetDossier returned error (1): Dossier generation failed. Mar 28 16:10:49 ip-10-157-176-156 err get_dossier[13331]: 01170003:3: halGetDossier returned error (1): Dossier generation failed. Mar 28 16:11:02 ip-10-157-176-156 err mcpd[7952]: 01020036:3: The requested parent profile (/Common/serverssl) was not found. Mar 28 16:11:54 ip-10-157-176-156 info systemd-journal[1062]: Suppressed 205 messages from /system.slice/cloud-final.service

Additionally I see these errors around the appdata volume, which when I boot with a BEST Plus image, it does not require more than 700mb, but see expected a volume of 20+ gig.

Errors surround appdata size issue Mar 29 08:23:38 ip-10-157-176-156 info mprov:10416:[10416]: 'There are NO extra disks.' Mar 29 08:23:38 ip-10-157-176-156 warning mprov:10416:[10416]: 'Disk limit exceeded. 20338 MB are required to provision these modules, but only 24 MB are available.- will mark potential space as free and retry..' Mar 29 08:23:38 ip-10-157-176-156 info mprov:10416:[10416]: 'Changing available disk space from 24 to 30' Mar 29 08:23:38 ip-10-157-176-156 err mprov:10416:[10416]: 'Disk limit exceeded. 20338 MB are required to provision these modules, but only 30 MB are available.' Mar 29 08:23:38 ip-10-157-176-156 info mprov:10416:[10416]: 'Provisioning (validation) failed.' Mar 29 08:23:38 ip-10-157-176-156 err mcpd[6178]: 01071008:3: Provisioning failed with error 1 - 'Disk limit exceeded. 20338 MB are required to provision these modules, but only 30 MB are available.' Mar 29 08:23:38 ip-10-157-176-156 err tmsh[10303]: 01420006:3: Loading configuration process failed.

I tested with a previous version - 16.1.3.1-0.0.11**PAYG-Good 25Mbps and still get same failures around appdata/dossier.

I have no issue booting my current config and getting a success using Best Plus.

mikeshimkus commented 1 year ago

Hi @sabooker, can you provide the name of the template you're using, as well as the full list of parameters? thanks

sabooker commented 1 year ago

Hello @mikeshimkus, See below info, thanks for your help!

Template: examples/autoscale/payg/autoscale-existing-network.yaml Params:

[
  {
    "ParameterKey": "allowUsageAnalytics",
    "ParameterValue": "true"
  },
  {
    "ParameterKey": "application",
    "ParameterValue": "f5app"
  },
  {
    "ParameterKey": "artifactLocation",
    "ParameterValue": "f5-aws-cloudformation-v2/v2.7.0.0/examples/"
  },
  {
    "ParameterKey": "bigIpSubnetAz1",
    "ParameterValue": "subnet-XXXXXXXXXXXXXb1bf"
  },
  {
    "ParameterKey": "bigIpSubnetAz2",
    "ParameterValue": "subnet-XXXXXXXXXXXXXac18"
  },
  {
    "ParameterKey": "externalSubnetAz1",
    "ParameterValue": "subnet-XXXXXXXXXXXXXa282"
  },
  {
    "ParameterKey": "externalSubnetAz2",
    "ParameterValue": "subnet-XXXXXXXXXXXXXe9cc"
  },
  {
    "ParameterKey": "internalSubnetAz1",
    "ParameterValue": "subnet-XXXXXXXXXXXXXb1bf"
  },
  {
    "ParameterKey": "internalSubnetAz2",
    "ParameterValue": "subnet-XXXXXXXXXXXXXac18"
  },
  {
    "ParameterKey": "bigIpCustomImageId",
    "ParameterValue": ""
  },
  {
    "ParameterKey": "bigIpImage",
    "ParameterValue": "*16.1.3.3-0.0.3**PAYG-Good 25Mbps*"
  },
  {
    "ParameterKey": "bigIpInstanceProfile",
    "ParameterValue": ""
  },
  {
    "ParameterKey": "bigIpInstanceType",
    "ParameterValue": "m5.xlarge"
  },
  {
    "ParameterKey": "bigIpMaxBatchSize",
    "ParameterValue": "5"
  },
  {
    "ParameterKey": "bigIpMinInstancesInService",
    "ParameterValue": "1"
  },
  {
    "ParameterKey": "bigIpPauseTime",
    "ParameterValue": "480"
  },
  {
    "ParameterKey": "bigIpRuntimeInitConfig",
    "ParameterValue": "https://XXXXXXXXXXXXX.s3.amazonaws.com/f5-config/runtime-init-conf-payg-with-app.yaml"
  },
  {
    "ParameterKey": "bigIpRuntimeInitPackageUrl",
    "ParameterValue": "https://cdn.f5.com/product/cloudsolutions/f5-bigip-runtime-init/v1.6.0/dist/f5-bigip-runtime-init-1.6.0-1.gz.run"
  },
  {
    "ParameterKey": "bigIpScaleInCpuThreshold",
    "ParameterValue": "20"
  },
  {
    "ParameterKey": "bigIpScaleInThroughputThreshold",
    "ParameterValue": "10000000"
  },
  {
    "ParameterKey": "bigIpScaleOutCpuThreshold",
    "ParameterValue": "80"
  },
  {
    "ParameterKey": "bigIpScaleOutThroughputThreshold",
    "ParameterValue": "20000000"
  },
  {
    "ParameterKey": "bigIpScalingMaxSize",
    "ParameterValue": "50"
  },
  {
    "ParameterKey": "bigIpScalingMinSize",
    "ParameterValue": "1"
  },
  {
    "ParameterKey": "bigIpSecretArn",
    "ParameterValue": ""
  },
  {
    "ParameterKey": "cloudWatchLogGroupName",
    "ParameterValue": "F5-CFT-V2-test1"
  },
  {
    "ParameterKey": "cloudWatchLogStreamName",
    "ParameterValue": "logs"
  },
  {
    "ParameterKey": "cloudWatchDashboardName",
    "ParameterValue": "F5-CFT-V2-test1"
  },
  {
    "ParameterKey": "cost",
    "ParameterValue": "f5cost"
  },
  {
    "ParameterKey": "createLogDestination",
    "ParameterValue": "true"
  },
  {
    "ParameterKey": "environment",
    "ParameterValue": "f5env"
  },
  {
    "ParameterKey": "group",
    "ParameterValue": "f5group"
  },
  {
    "ParameterKey": "loggingS3BucketName",
    "ParameterValue": ""
  },
  {
    "ParameterKey": "metricNameSpace",
    "ParameterValue": "f5-scaling-metrics"
  },
  {
    "ParameterKey": "notificationEmail",
    "ParameterValue": "XXXXXXXXXXXXX@gmail.com"
  },
  {
    "ParameterKey": "owner",
    "ParameterValue": "f5owner"
  },
  {
    "ParameterKey": "provisionExternalBigipLoadBalancer",
    "ParameterValue": "false"
  },
  {
    "ParameterKey": "provisionInternalBigipLoadBalancer",
    "ParameterValue": "true"
  },
  {
    "ParameterKey": "provisionPublicIp",
    "ParameterValue": "false"
  },
  {
    "ParameterKey": "restrictedSrcAddressMgmt",
    "ParameterValue": "10.0.0.0/8"
  },
  {
    "ParameterKey": "restrictedSrcAddressApp",
    "ParameterValue": "10.0.0.0/8"
  },
  {
    "ParameterKey": "s3BucketName",
    "ParameterValue": "f5-cft-v2"
  },
  {
    "ParameterKey": "s3BucketRegion",
    "ParameterValue": "us-east-1"
  },
  {
    "ParameterKey": "snsEvents",
    "ParameterValue": "autoscaling:EC2_INSTANCE_LAUNCH,autoscaling:EC2_INSTANCE_LAUNCH_ERROR"
  },
  {
    "ParameterKey": "sshKey",
    "ParameterValue": "F5-CFT-V2-Key"
  },
  {
    "ParameterKey": "uniqueString",
    "ParameterValue": "f5v2testing2"
  },
  {
    "ParameterKey": "vpcCidr",
    "ParameterValue": "10.1.1.0/24"
  },
  {
    "ParameterKey": "vpcId",
    "ParameterValue": "vpc-XXXXXXXXXXXXXadaf"
  }
]
sabooker commented 1 year ago

hello @mikeshimkus.

Did this help, looking for any updates you have on your end. Did you find the same in your testing?

mikeshimkus commented 1 year ago

Hi @sabooker, sorry, haven't had a chance yet to have a look. I created internal issue EC-212 for tracking it.

mikeshimkus commented 1 year ago

Hey @sabooker, by default the template provisions LTM and ASM, in order to use this combo you need to choose a best or best plus image:

If you don't want to use ASM you would need to modify the runtime init config by removing ASM here: https://github.com/F5Networks/f5-aws-cloudformation-v2/blob/a4d715d3c22fed7d779e1cd693c008636a464672/examples/autoscale/bigip-configurations/runtime-init-conf-payg-with-app.yaml#L65

It looks like you're using a custom config for bigIpRuntimeInitConfig, did you remove ASM from the module provisioning there?

sabooker commented 1 year ago

Thank you Mike, this solved our issue.