search

aws-controllers-k8s / community

AWS Controllers for Kubernetes (ACK) is a project enabling you to manage AWS services from Kubernetes
https://aws-controllers-k8s.github.io/community/
Apache License 2.0
2.39k stars 253 forks source link

RDS stale value on `DBInstance` updating `dbParameterGroups` `parameterApplyStatus` #1945

Open dweebo opened 9 months ago

dweebo commented 9 months ago

Describe the bug Sometimes our DBInstance instance have a stale value in status.dbParameterGroups.parameterApplyStatus.

I'm not sure but it seems like the rds operator stops reconciling the DBInstance resource once it thinks it is "available". I stop seeing logs for the instances that I see with this issue.

Steps to reproduce Create a DBInstance from snapshot (dbSnapshotIdentifier) and with a custom dbParameterGroupName. Wait for RDS instance to be available and for the db parameter group to be set/applied. Check the DBInstance.status.dbParameterGroups.parameterApplyStatus. 3 out of 10 times in my testing the value was not updated on the DBInstance custom resources, even after waiting for an hour after the RDS instances were fully running.

Expected outcome I expect the RDS operator to keep reconciling until after the db parameter group is applied and the value is set to in-sync.

Environment

a-hilaly commented 7 months ago

Hi @dweebo - are you configuring the reconcile frequency of the controller? by default it's 10hours.

dweebo commented 7 months ago

Hi @dweebo - are you configuring the reconcile frequency of the controller? by default it's 10hours.

No I don't believe we are changing that. I didn't want to unnecessarily be reconciling but I see that could be a workaround.

ack-bot commented 1 month ago

Issues go stale after 180d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 60d of inactivity and eventually close. If this issue is safe to close now please do so with /close. Provide feedback via https://github.com/aws-controllers-k8s/community. /lifecycle stale