Cannot read property 'nic' of undefined

[hrdina28]

Do you already have an issue opened with F5 support?

No.

Description

Hello,

I cant do failover :(. Its a new deployment, perhaps completely my mistake. But I really dont know how to get over it.

Error message: Wed, 07 Jun 2023 18:17:27 GMT - fine: [f5-cloud-failover] _discoverAddressOperations parsed nics {"mine":[{"nic":{"Attachment":{"AttachTime":"2023-06-07T07:15:03.000Z","AttachmentId":"eni-attach-0db8e2a20427dcb0c","DeleteOnTermination":true,"DeviceIndex":1,"NetworkCardIndex":0,"InstanceId":"i-08b4a91aa7b7a3cde","InstanceOwnerId":"187459682462","Status":"attached"},"AvailabilityZone":"us-west-1a","Description":"","Groups":[{"GroupName":"VirtualServer-Security-Group","GroupId":"sg-036d87aecdc84ac70"}],"InterfaceType":"interface","Ipv6Addresses":[],"MacAddress":"06:22:c4:93:2e:69","NetworkInterfaceId":"eni-00e77dca64bc51467","OwnerId":"187459682462","PrivateIpAddress":"10.0.1.200","PrivateIpAddresses":[{"Primary":true,"PrivateIpAddress":"10.0.1.200"},{"Association":{"AllocationId":"eipalloc-01121fe7dd53f5241","AssociationId":"eipassoc-02749d160265b07aa","IpOwnerId":"187459682462","PublicDnsName":"","PublicIp":"13.57.139.194"},"Primary":false,"PrivateDnsName":"","PrivateIpAddress":"10.0.1.210"}],"Ipv4Prefixes":[],"Ipv6Prefixes":[],"RequesterManaged":false,"SourceDestCheck":false,"Status":"in-use","SubnetId":"subnet-0d30ef51debc4867f","TagSet":[{"Key":"f5_cloud_failover_label","Value":"mydeployment"},{"Key":"f5_cloud_failover_nic_map","Value":"external"}],"VpcId":"vpc-0d734c156a019768f"}}],"theirs":[]} Wed, 07 Jun 2023 18:17:27 GMT - severe: [f5-cloud-failover] error in _getFailoverDiscovery: TypeError: Cannot read property 'nic' of undefined

Definition: { "class": "Cloud_Failover", "environment": "aws", "controls": { "class": "Controls", "logLevel": "silly" }, "externalStorage": { "scopingName": "jlo-bucket-f5", "encryption": { "serverSide": { "enabled": true, "algorithm": "AES256" } } }, "failoverAddresses": { "enabled": true, "scopingTags": { "f5_cloud_failover_label": "mydeployment" }, "addressGroupDefinitions": [ { "type": "networkInterfaceAddress", "scopingAddress": "10.0.1.210" } ] } }

Currently only on public facing interface with EIP.

Environment information

For bugs, enter the following information:

• Cloud Failover Extension Version: 1.14
• BIG-IP version: 17.1
• Cloud provider: AWS

Severity Level

Severity: 3

Thank you

Best regards

[shyawnkarim]

How did you create this deployment? Did you use this template: https://github.com/F5Networks/f5-aws-cloudformation-v2/tree/main/examples/failover or did you deploy it another way?

[hrdina28]

Hi, this one: https://clouddocs.f5.com/cloud/public/v1/aws/AWS_ha.html#awshaobjects So 2x https://clouddocs.f5.com/cloud/public/v1/aws/AWS_multiNIC.html#awsmultinic and then followed: https://clouddocs.f5.com/products/extensions/f5-cloud-failover/latest/userguide/aws-same-az.html#aws-same-az

[shyawnkarim]

It looks like the tag on your 2nd nic may be incorrect. Can you double check them? Sometimes a trailing whitespace will cause a tag to mismatch.

[hrdina28]

Why do you think so? I cant see any whitespace in AWS or in json output.

Thank you

[shyawnkarim]

I just meant that as an example that could create a tag to mismatch. Based off the logs, it looks like one of your nics isn't being found by CFE.

[hrdina28]

You mean that I should have those tags on all interfaces? So also mgmt and HA?

[shyawnkarim]

This section in our documentation goes over proper tagging of your AWS infrastructure objects. Can you check to see if all of your tags have been done correctly?

[hrdina28]

I have just found out that its happening only on one F5. The other one is able to process the dry-run command correctly. I checked the JSON config, tags and both are correct.

[admin@ip-10-0-0-201:Standby:In Sync] ~ # curl -su admin: -X POST -d '{"action":"dry-run"}' http://localhost:8100/mgmt/shared/cloud-failover/trigger | jq . { "message": "Cannot read property 'nic' of undefined" } [admin@ip-10-0-0-201:Standby:In Sync] ~ # curl -su admin: http://localhost:8100/mgmt/shared/cloud-failover/inspect | jq . { "instance": "i-0453959a814903525", "addresses": [ { "publicIpAddress": "13.57.139.194", "privateIpAddress": "10.0.1.210", "networkInterfaceId": "eni-069bae22b6678b286" }, { "publicIpAddress": "3.101.175.105", "privateIpAddress": "10.0.0.201", "networkInterfaceId": "eni-07c698a2e4bbfaa44" } ], "routes": [], "hostName": "ip-10-0-0-201.us-west-1.compute.internal", "deviceStatus": "standby", "trafficGroup": [ { "name": "/Common/traffic-group-1" } ] }

[shyawnkarim]

Can you try resetting the state file and test again?

[hrdina28]

Same:

[admin@ip-10-0-0-201:Standby:In Sync] ~ # curl -su admin: -X POST -d '{"resetStateFile":true}' http://localhost:8100/mgmt/shared/cloud-failover/reset | jq . { "message": "Failover state file was reset" } [admin@ip-10-0-0-201:Standby:In Sync] ~ # curl -su admin: http://localhost:8100/mgmt/shared/cloud-failover/inspect | jq . { "instance": "i-0453959a814903525", "addresses": [ { "publicIpAddress": "13.57.139.194", "privateIpAddress": "10.0.1.210", "networkInterfaceId": "eni-069bae22b6678b286" }, { "publicIpAddress": "3.101.175.105", "privateIpAddress": "10.0.0.201", "networkInterfaceId": "eni-07c698a2e4bbfaa44" } ], "routes": [], "hostName": "ip-10-0-0-201.us-west-1.compute.internal", "deviceStatus": "standby", "trafficGroup": [ { "name": "/Common/traffic-group-1" } ] } [admin@ip-10-0-0-201:Standby:In Sync] ~ # curl -su admin: -X POST -d '{"action":"dry-run"}' http://localhost:8100/mgmt/shared/cloud-failover/trigger | jq . { "message": "Cannot read property 'nic' of undefined" }

[hrdina28]

How did you create this deployment? Did you use this template: https://github.com/F5Networks/f5-aws-cloudformation-v2/tree/main/examples/failover or did you deploy it another way?

Embedded stack arn:aws:cloudformation:us-east-1:187459682462:stack/BigIp-Failx-BigIpInstance01-EK2EY6500KD1/22142fa0-062c-11ee-8c3e-122d8ab572d1 was not successfully created: The following resource(s) failed to create: [S3Bucket, BigipStaticExternalInterface].

Only filled in allowed source IPs ... :/

[shyawnkarim]

Excellent, I'm happy you were able to find the cause of the error. I'm going to close this issue but if you would still like additional assistance, leave a message here and I'll reopen it.

[hrdina28]

You missunderstood, I am sorry.

The template you mentioned at the beginning ends up with error while building up: Embedded stack arn:aws:cloudformation:us-east-1:187459682462:stack/BigIp-Failx-BigIpInstance01-EK2EY6500KD1/22142fa0-062c-11ee-8c3e-122d8ab572d1 was not successfully created: The following resource(s) failed to create: [S3Bucket, BigipStaticExternalInterface]. Then its reverting all changes. I wanted to try this one in different AWS datacenter.

The problem with "Cannot read property 'nic' of undefined" is in different datacenter. And still there.

I feel to be unlucky in both ways it seems :)

[shyawnkarim]

Can you open up another issue for this 2nd issue?

[hrdina28]

Managed to get this one: Fri, 09 Jun 2023 18:26:25 GMT - severe: [f5-cloud-failover] Cannot read property 'nic' of undefined TypeError: Cannot read property 'nic' of undefined at Promise.all.then (/var/config/rest/iapps/f5-cloud-failover/nodejs/providers/aws/cloud.js:361:42) at tryCatcher (/usr/share/rest/node/node_modules/bluebird/js/release/util.js:16:23) at Promise._settlePromiseFromHandler (/usr/share/rest/node/node_modules/bluebird/js/release/promise.js:512:31) at Promise._settlePromise (/usr/share/rest/node/node_modules/bluebird/js/release/promise.js:569:18) at Promise._settlePromise0 (/usr/share/rest/node/node_modules/bluebird/js/release/promise.js:614:10) at Promise._settlePromises (/usr/share/rest/node/node_modules/bluebird/js/release/promise.js:693:18) at Promise._fulfill (/usr/share/rest/node/node_modules/bluebird/js/release/promise.js:638:18) at PromiseArray._resolve (/usr/share/rest/node/node_modules/bluebird/js/release/promise_array.js:126:19) at PromiseArray._promiseFulfilled (/usr/share/rest/node/node_modules/bluebird/js/release/promise_array.js:144:14) at Promise._settlePromise (/usr/share/rest/node/node_modules/bluebird/js/release/promise.js:574:26) at Promise._settlePromise0 (/usr/share/rest/node/node_modules/bluebird/js/release/promise.js:614:10) at Promise._settlePromises (/usr/share/rest/node/node_modules/bluebird/js/release/promise.js:693:18) at Async._drainQueue (/usr/share/rest/node/node_modules/bluebird/js/release/async.js:133:16) at Async._drainQueues (/usr/share/rest/node/node_modules/bluebird/js/release/async.js:143:10) at Immediate.Async.drainQueues (/usr/share/rest/node/node_modules/bluebird/js/release/async.js:17:14) at runCallback (timers.js:794:20) at tryOnImmediate (timers.js:752:5) at processImmediate [as _immediateCallback] (timers.js:729:5) Fri, 09 Jun 2023 18:26:25 GMT - finest: [f5-cloud-failover] Uploading data to: f5cloudfailover/f5cloudfailoverstate.json Fri, 09 Jun 2023 18:26:28 GMT - finest: [f5-cloud-failover] Telemetry submitted successfully

Perhaps some reading issue? The example on the left is nonworking, while on the right is working one.

Thank you :)

[shyawnkarim]

@hrdina28 can you email me at s.karim@f5.com? I'd like to schedule a time to help you troubleshoot your environment.

[hrdina28]

Hello, I did :)

[shyawnkarim]

Closing. This issue was resolved with Release 1.15.0.