search

nonlive / keto

Manage kubernetes clusters
https://github.com/nonlive/keto
Apache License 2.0
14 stars 2 forks source link

Duplicate API calls (describe-subnets) in cluster build #73

Open KashifSaadat opened 7 years ago

KashifSaadat commented 7 years ago

The addition of logging has highlighted duplication of certain API calls. On creating a full cluster (master & compute nodes), the describe-subnets ec2 API is called a minimum of 4 times (can increase depending on the amount of compute pools defined). Additionally, on building computepool0 the call to the API passes 5 subnets when only 3 were initially defined (2 ids were duplicated).

Example Logs:

[debug] 2017/06/28 09:50:55 aws.go:699: describing a list of subnets [subnet-XXXXXX01 subnet-XXXXXX02 subnet-XXXXXX03]
[debug] 2017/06/28 09:50:55 aws.go:707: received subnets description: [3 subnet objects]
..
[debug] 2017/06/28 09:52:01 aws.go:699: describing a list of subnets [subnet-XXXXXX01 subnet-XXXXXX01 subnet-XXXXXX02 subnet-XXXXXX02 subnet-XXXXXX03]
[debug] 2017/06/28 09:52:01 aws.go:707: received subnets description: [3 subnet objects]

The providers could potentially have a method of maintaining a local cache for certain API calls, so if the same subnet IDs are passed the details can be returned immediately.

Is there any case where the data may change mid build, requiring a re-fetch?

vaijab commented 7 years ago

I am aware of this. This code bit could definitely be optimized. They are not entirely duplicate calls, they happen at different times for different components, but sure, some runtime state could help to minimize cloud API calls.