Closed damianharouff closed 10 months ago
It's important to note that both the old and new endpoint IPs were live
Yes, that's because we are using http keep alive. This is a known and yet unresolved problem.
https://github.com/golang/go/issues/23427
https://abhishekvrshny.medium.com/dns-aware-persistent-connections-abda9921db34
I think a simple fix would be to just re-create a new http connection every X periodical loops.
Problem description
Sidecar does not re-resolve the Graylog server hostname after the connection is made, so if the endpoint IP changes at some point in the future, the only action is to restart Sidecar to re-resolve the hostname.
This was noted with several Graylog Cloud customers during V1 -> NG migrations, where we continued to note Sidecars still hitting the old endpoint IP several days later, and had to reach out to the affected customers to have them just restart Sidecar.
It's important to note that both the old and new endpoint IPs were live, but Sidecar never started contacting Graylog on the new IP, because it did not seem to ever re-resolve Graylog server hostname.
Environment