search

jenkinsci / ec2-fleet-plugin

The EC2 Fleet plugin launches EC2 instances as worker nodes for Jenkins CI server, automatically scaling the capacity with the load.
https://plugins.jenkins.io/ec2-fleet/
Apache License 2.0
111 stars 81 forks source link

Making changes to plugin config not reflected, jobs not scaling #153

Closed jnewton03 closed 4 years ago

jnewton03 commented 5 years ago

I have been doing some testing with this plugin to replace the EC2 plugin. I created several different fleets and have been testing with a number of builds. However, when I try to start additional builds, the fleet never scales up. Here is what I see in the logs:

currentDemand 2 availableCapacity 0 (availableExecutors 0 connectingExecutors 0 plannedCapacitySnapshot 0 additionalPlannedCapacity 0)

Nov 06, 2019 3:41:00 PM FINE com.amazon.jenkins.ec2fleet.EC2FleetCloud fine
protoclubFleet [protoclubFleet] CanProvision called on fleet: "protoclubFleet" wanting: "linux-docker-client". Returning false.

Nov 06, 2019 3:41:00 PM FINE com.amazon.jenkins.ec2fleet.NoDelayProvisionStrategy apply
Provisioning not complete, consulting remaining strategies

Nov 06, 2019 3:41:00 PM INFO com.amazon.jenkins.ec2fleet.NoDelayProvisionStrategy apply
currentDemand -1 availableCapacity 2 (availableExecutors 0 connectingExecutors 0 plannedCapacitySnapshot 2 additionalPlannedCapacity 0)

Nov 06, 2019 3:41:00 PM FINE com.amazon.jenkins.ec2fleet.NoDelayProvisionStrategy apply
Provisioning completed

Nov 06, 2019 3:41:00 PM INFO com.amazon.jenkins.ec2fleet.NoDelayProvisionStrategy apply
currentDemand 5 availableCapacity 0 (availableExecutors 0 connectingExecutors 0 plannedCapacitySnapshot 0 additionalPlannedCapacity 0)

Nov 06, 2019 3:41:00 PM FINE com.amazon.jenkins.ec2fleet.EC2FleetCloud fine
protoclubFleet [protoclubFleet] CanProvision called on fleet: "protoclubFleet" wanting: "windowsClient". Returning false.

Nov 06, 2019 3:41:00 PM FINE com.amazon.jenkins.ec2fleet.NoDelayProvisionStrategy apply
Provisioning not complete, consulting remaining strategies

Nov 06, 2019 3:41:00 PM FINE com.amazon.jenkins.ec2fleet.EC2FleetCloud fine
protoclubFleet [protoclubFleet] CanProvision called on fleet: "protoclubFleet" wanting: "windowsClient". Returning false.

Nov 06, 2019 3:41:01 PM INFO com.amazon.jenkins.ec2fleet.EC2FleetCloud info
protoclubFleet [protoclubFleet] start

Nov 06, 2019 3:41:01 PM INFO com.amazon.jenkins.ec2fleet.EC2FleetCloud info
protoclubFleet [protoclubFleet] fleet instances [i-06600ba4737431c4a]

Nov 06, 2019 3:41:01 PM INFO com.amazon.jenkins.ec2fleet.EC2FleetCloud info
protoclubFleet [protoclubFleet] described instances [i-06600ba4737431c4a]

Nov 06, 2019 3:41:01 PM INFO com.amazon.jenkins.ec2fleet.EC2FleetCloud info
protoclubFleet [protoclubFleet] jenkins nodes [i-06600ba4737431c4a]

Nov 06, 2019 3:41:01 PM INFO com.amazon.jenkins.ec2fleet.EC2FleetCloud info
protoclubFleet [protoclubFleet] jenkins nodes without instance []

Nov 06, 2019 3:41:01 PM INFO com.amazon.jenkins.ec2fleet.EC2FleetCloud info
protoclubFleet [protoclubFleet] terminated instances []

Nov 06, 2019 3:41:01 PM INFO com.amazon.jenkins.ec2fleet.EC2FleetCloud info
protoclubFleet [protoclubFleet] new instances []

Nov 06, 2019 3:41:06 PM INFO com.amazon.jenkins.ec2fleet.EC2FleetCloud info
protoclubFleet [protoclubFleet] start

Nov 06, 2019 3:41:06 PM INFO com.amazon.jenkins.ec2fleet.EC2FleetCloud info
protoclubFleet [protoclubFleet] fleet instances [i-06600ba4737431c4a]

Nov 06, 2019 3:41:06 PM INFO com.amazon.jenkins.ec2fleet.EC2FleetCloud info
protoclubFleet [protoclubFleet] described instances [i-06600ba4737431c4a]

Nov 06, 2019 3:41:06 PM INFO com.amazon.jenkins.ec2fleet.EC2FleetCloud info
protoclubFleet [protoclubFleet] jenkins nodes [i-06600ba4737431c4a]

Nov 06, 2019 3:41:06 PM INFO com.amazon.jenkins.ec2fleet.EC2FleetCloud info
protoclubFleet [protoclubFleet] jenkins nodes without instance []

Nov 06, 2019 3:41:06 PM INFO com.amazon.jenkins.ec2fleet.EC2FleetCloud info
protoclubFleet [protoclubFleet] terminated instances []

Nov 06, 2019 3:41:06 PM INFO com.amazon.jenkins.ec2fleet.EC2FleetCloud info
protoclubFleet [protoclubFleet] new instances []

However, when I show my fleet, it is never scaling up: Total target capacity 1 (1 fulfilled)

On-Demand capacity 0 (0 fulfilled)

jnewton03 commented 5 years ago

I tried again today by changing to a different fleet and continue to get messages like the below even though my fleet has capacity of 1. It seems changes to the configuration do not take effect without a Jenkins reboot:

currentDemand -1 availableCapacity 2 (availableExecutors 0 connectingExecutors 0 plannedCapacitySnapshot 2 additionalPlannedCapacity 0)

DerkSchooltink commented 4 years ago

I tried again today by changing to a different fleet and continue to get messages like the below even though my fleet has capacity of 1. It seems changes to the configuration do not take effect without a Jenkins reboot:

currentDemand -1 availableCapacity 2 (availableExecutors 0 connectingExecutors 0 plannedCapacitySnapshot 2 additionalPlannedCapacity 0)

@jnewton03 Can you verify this still occurs after installing the 1.16.2 release?

jnewton03 commented 4 years ago

@DerkSchooltink I will upgrade and if there are still issues I'll post them here. Thanks for the heads up! :)