search

boynux / squid-exporter

Squid Prometheus Exporter
https://www.boynux.com/squid-exporter
MIT License
134 stars 53 forks source link

Failed to scrape metrcis due to scrape timeout #60

Closed LeoQuote closed 1 year ago

LeoQuote commented 2 years ago

Describe the bug I set up a scrape job for squid exporter and got scrape timeout occasionally

How can I troubleshoot this? I saw no logs either in squid exporter log or squid log.

What could be the cause of this timeout, what do you suggest to set the timeout to?

To Reproduce start a exporter to exporter squid metrics

Expected behavior scrape successfully, no timeout

actual: scrape is quite slow, response time sometimes can go up to 10s or more.

OS (please complete the following information):

Additional context Add any other context about the problem here.

boynux commented 2 years ago

Is squid and exporter running on a same server?

LeoQuote commented 2 years ago

yes, in the same server , squid exporter is running in container with host network enabled

boynux commented 2 years ago

Can you try to fetch metrics locally to see if that times out as well? You can use this guide: http://etutorials.org/Server+Administration/Squid.+The+definitive+guide/Chapter+14.+Monitoring+Squid/14.2+The+Cache+Manager/

LeoQuote commented 2 years ago

squid seems to have respond with a 400 response 

<p>The following error was encountered while trying to retrieve the URL: <a href="8118">8118</a></p>

<blockquote id="error">
<p><b>Invalid URL</b></p>
</blockquote>

what should I do to fix this? how did squid exporter managed to get these metrics?

boynux commented 2 years ago

Can you post the command you tried and got this error? It's also helpful to share your squid config (redact sensitive data), maybe in pastebin.com

I'm guessing it might be you didn't enable cache manager for your instance.

LeoQuote commented 2 years ago

After checking the cache manager page in squid manual, I tried

time curl http://localhost:8118/squid-internal-mgr/info

and it looks ok, and at the same time , the exporter can be scraped, too. I'll check it again when scraping time out.

The case I described above occurs occasionall, but not constantly, I'm trying to figure out the root cause.

Is it possible to take 10s or more to scrape the metric page?

boynux commented 1 year ago

I don't think that's normal. If not mistaken that usually means either the squid server is overloaded and cannot response to the request or it's crashing.

What you could do is enabling debug logs for the squid and see what happens, is request actually gets to the server? if not, why? is the process crashed? if yes, then why the response takes so long?

I'd say the response in normal situation should not take more than a few milliseconds. Squid has most of this data already ready, the call doesn't really do much and just fetches the data.