search

hashicorp / consul

Consul is a distributed, highly available, and data center aware solution to connect and configure applications across dynamic, distributed infrastructure.
https://www.consul.io
Other
28.4k stars 4.43k forks source link

Unregistered query non-helpful log spam #7864

Open nvx opened 4 years ago

nvx commented 4 years ago

Running Consul 1.7.3

On some hosts I've seen a bit of spam in the Consul logs that looks like this:

    2020-05-13T07:07:29.670Z [ERROR] agent.client: RPC failed to server: method=PreparedQuery.Execute server=x.x.x.x:8300 error="rpc error making call: Query not found"
    2020-05-13T07:07:29.672Z [ERROR] agent.client: RPC failed to server: method=PreparedQuery.Execute server=x.x.x.x:8300 error="rpc error making call: Query not found"

I've been able to reproduce the error in the log by running something like this:

dig foo.query.consul. @127.0.0.1 -p 8600

I can see the logs being handy for debugging (the hosts in question run Nomad jobs, and I can't figure out which job is causing it), but as stands they don't contain any helpful information like the name of the query to troubleshoot the issue. The error level should probably be dropped to be lower than ERROR too.

Seems similar to #1660, but I'd posit that the logs are potentially useful, just not in their current form (or at their current log level).

nvx commented 4 years ago

For what it's worth, tcpdump showed me in my instance the queries were being caused by _grpc_config requests so maybe of questionable utility logging these at all (at least in my environment). I suspect these logs weren't present on Consul 1.7.2.