Cancelling deployment to polling tentacle that is no longer there does not stop RPC retries

[sburmanoctopus]

Team

• [X] I've assigned a team label to this issue

What happened?

When a deployment to a polling tentacle that has recently gone offline is cancelled, then RPC retries may continue to execute until the RPC retry time limit is reached.

Reproduction

• Ensure RPC Retries is on
• Deploy successfully to a Polling Tentacle that is online
• Take Polling Tentacle offline
• Retry the deployment
• Wait for RPC retry message to start, then cancel

At this point, RPC retry messages will continue to stream in, even though we have cancelled.

Error and Stacktrace

No response

More Information

What appears to be happening is that the cancellation is happening correctly.

But since it has gotten past get capabilities, and has called 'start script', it doesn't know if the Tentacle actually received the message or not. So now it has to call Cancel on the Tentacle just in case.

But because the Tentacle is offline, the Cancel RPC call never gets through.

Workaround

No response

[nathanwoctopusdeploy]

Server is incorrectly trying to Cancel the Script running on Tentacle in this scenario. We know tentacle has not started the script in this scenario, so can cancel quickly without trying to cancel the script on Tentacle

[nathanwoctopusdeploy]

Moved to https://github.com/OctopusDeploy/Issues/issues/8535