search

rundeck-plugins / rundeck-ec2-nodes-plugin

Get resource node data from Amazon EC2
85 stars 45 forks source link

Refresh nodes on scheduled jobs #52

Closed gdowmont closed 5 years ago

gdowmont commented 8 years ago

More of a question than an issue itself.

I am running Rundeck 2.6.7 and plugin 1.5.2. I have configured number of project and scheduled jobs that run on EC2 instances that belong to Auto Scaling Groups. Everything works as expected until the ASG scales down and terminates some of the instances. Scheduled jobs are timing out because they cannot connect to the terminated nodes because the node select filter has not been refreshed to see the updated list of the instances.

I have gone through the docs of both Rundeck and the plugin however I could not spot anything that would allow me to refresh the nodes before running those jobs. Is this intended behaviour or have I missed something when configuring jobs/projects,sources? Is there any way I can debug it any further?

Thanks Greg

gschueler commented 8 years ago

see also https://github.com/rundeck/rundeck/issues/78

Currently Node sources are also cached within Rundeck for performance reasons, but it would be nice to allow a Job to force a synchronous reload if necessary, perhaps with a new Job flag, to have the most up to date data before execution.

I know that is an ooold rundeck issue, however we've done work recently on the Node caching system, so it is useful to hear other use cases/requirements to help us improve it.

kevhu commented 8 years ago

Until this feature exists, what is the suggested workaround? I don't see how using autoscaling groups is possible if nodes are not refreshed.

What if we disabled node caching entirely in the project properties?

jinesh-choksi commented 8 years ago

@kevhu, I've suggested a workaround in rundeck/rundeck#78

gschueler commented 5 years ago

https://github.com/rundeck/rundeck/pull/2380