search

Kong / kong

🦍 The Cloud-Native API Gateway and AI Gateway.
https://konghq.com/install/
Apache License 2.0
39.42k stars 4.83k forks source link

Could not claim instance_id for {{PLUGIN_NAME}} (key: {{PLUGIN_ID}}) #11173

Closed vtlkvl closed 1 year ago

vtlkvl commented 1 year ago

Is there an existing issue for this?

Kong version ($ kong version)

3.3.0

Current Behavior

Thread for context https://discuss.konghq.com/t/frequent-building-a-new-plugins-iterator/11656.

We run a DB-less setup in k8s where backend services run on spot instances and their IP addresses are changing pretty often. This triggers declarative reconfiguration and once every few days one of the pods gets stuck on Could not claim instance_id for. A potential issue could be that reset_instance is called only for not ready or no plugin instance. If any other error occurs, non-initialized plugin instance will not be cleaned up, and other threads will not make through the while loop in get_instance_id.

Expected Behavior

No response

Steps To Reproduce

https://github.com/vtlkvl/kong-plugins-reaload-race-condition

Anything else?

No response

StarlightIbuki commented 1 year ago

Hi. I think this PR may fix the issue: https://github.com/Kong/kong/pull/11370. It's already merged to the master so you could update and try the latest version.

alyywehby commented 1 year ago

Hey! I am facing the same issue! Using kong:3.3.1-alpine docker image! @vtlkvl is it solved at your end?

aditya-kumar-quizizz commented 1 year ago

I am seeing this issue in kong:3.3.0-alpine. which kong version has the fix? @alyywehby @vtlkvl @guanlan

mrnonz commented 11 months ago

Hi, everyone

Which kong version patch this bug? Thanks

mrnonz commented 11 months ago

@alyywehby @aditya-kumar-quizizz @vtlkvl Did you try with kong:3.4.0? or which version patch this one? I try to read many change log but not found any point to confirm this already solve.

StarlightIbuki commented 11 months ago

@mrnonz Please try with Kong 3.5. Sorry, I've left the wrong link and it's not accessible. And the PR misses a changelog.

aditya-kumar-quizizz commented 11 months ago

@mrnonz : I was looking at alpine version of the kong. And looks like latest alpine version is 3.3.1-alpine. Any idea which alpine version will have this fix? Or are we planning to roll out new alpine version of kong in near future?

StarlightIbuki commented 11 months ago

@mrnonz : I was looking at alpine version of the kong. And looks like latest alpine version is 3.3.1-alpine. Any idea which alpine version will have this fix? Or are we planning to roll out new alpine version of kong in near future?

Sorry. We have dropped support for Alpine.

mrnonz commented 11 months ago

@mrnonz Please try with Kong 3.5. Sorry, I've left the wrong link and it's not accessible. And the PR misses a changelog.

Thank you, I will try with v3.5.0++