AWS: Agent does not update instance type changes

[sebastianvilla]

Given the following scenario:

EC2 node deployed via Docker Cloud using t2.medium instance type. After changing the instance type to t2.large via AWS console (stop, change instance type, start) the WebUI continues to list node details for the old instance instance type.

The WebUI does receive a Node Update (agent) success event, which triggers the corresponding Container update events successfully. Details about the new instance type are available via the instance metadata (http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html#instancedata-data-retrieval)

Note: I have not tested this scenario with other providers (i.e. resizing a DigitalOcean droplet, though I suspect the flow should be fairly similar).

[amegianeg]

Hi Sebastian, The agent is not telling DC that the size of an instance has changed. That task belongs to a system container deployed in the node called "metrics". DC should have been notified about this change and should have modified the info, but according to your issue, looks like there was an error here. Can you provide me more info about the node? need to troubleshoot it to know what's going on there. Thanks!

EDIT: the metrics system container has been removed, so Docker Cloud does not have any notion of the resized event.

[sebastianvilla]

@amegianeg are you saying that your guys removed the metrics system container from the codebase? or that it is removed when the AWS instance type changes? (stop EC2 instance, change type, start EC2 instance).

The issue remains the same, I just tried the following:

1. Create a new node cluser using AWS as provider, us-east-1 and using an m4-large instance (or any type)
2. Once the cluster is deployed everything works as expected
3. Now go into the AWS Console and stop the EC2 instance launched by DC.
4. At this point the node is show as "Unreachable" in DC
5. Change the instance type to something else (c4.8xlarge in my test) via AWS Console
6. Start the EC2 instance
7. New node type (and CPU & RAM) are not updated in DC

Note: this time in my test, the node remains as "Unreachable" in DC indefinitely.

I think the expected behaviour would be that if a Node is resized, the specs would be updated by DC.3

[amegianeg]

@sebastianvilla sorry for not clarifying that: yes, we no longer maintain the metrics container. Sorry for the inconvenience

[vojtasvoboda]

Same problem here - when changing EC2 type directly at AWS, Docker Cloud still showing old instance type.

Is there some way how to change instance type on-the-fly?

Thanks :-)