search

cirruslabs / cirrus-ci-agent

Agent to execute Cirrus CI tasks
Mozilla Public License 2.0
13 stars 6 forks source link

RFE: auto restart for "Failed to start java blah" #303

Closed edsantiago closed 11 months ago

edsantiago commented 1 year ago

Quite often we submit PRs, come back an hour later, and find:

! Failed to start: Task java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@7d6a9ed9[Not completed, task = java.util.concurrent.Executors$RunnableAdapter@47426c98[Wrapped task = TrustedListenableFutureTask@6ef275bf[status=PENDING, info=[task=[running=[NOT STARTED YET], com.google.api.gax.rpc.CheckingAttemptCallable@17b03199]]]]] rejected from java.util.concurrent.ScheduledThreadPoolExecutor@29980b40[Shutting down, pool size = 4, active threads = 0, queued tasks = 2, completed tasks = 1523]

A hooman then has to manually click Re-run on all of those. And wait an extra hour.

Would it be possible to auto-restart these? Perhaps after a binary backoff?

Thank you!

edsantiago commented 12 months ago

Example. And another on the same CI run. And one more.

edsantiago commented 11 months ago

And here and here and here

fkorotkov commented 11 months ago

This should be fixed now! Sorry for the inconveniences and thank you for reporting!

edsantiago commented 11 months ago

Thank you so much @fkorotkov !