Requests to prometheus do not follow HTTP redirects

[quolix]

We are running Prometheus on Mesos and have an HTTP redirect in place that redirects requests to http://prometheus.corp.com to the Prometheus' current HTTP port.

Promdash does not seem to follow these redirects when configuring http://prometheus.corp.com as a server.

[quolix]

The red (!) displays Expression scalar(rack_temperature): Server returned status undefined.

[juliusv]

That's strange, haven't figured out why yet. https://docs.angularjs.org/api/ng/service/$http mentions "Note that if the response is a redirect, XMLHttpRequest will transparently follow it, meaning that the error callback will not be called for such responses.".

Could you check in your browser's dev console what the actual request/response pattern/contens is like?

[quolix]

This is with Chrome 43. It gets the redirect, but both here and in Firefox it is not entirely clear to which server the request after the redirect is sent.

Remote Address:10.10.1.13:80 Request URL:http://prometheus.redir.corp.quobyte.com/api/query?expr=scalar(rack_temperature) Request Method:GET

HTTP/1.1 307 Temporary Redirect Server: SimpleHTTP/0.6 Python/2.7.5 Date: Mon, 22 Jun 2015 13:51:51 GMT Location: http://prometheus-9027-s4.marathon.mesos:31007/api/query?expr=scalar(rack_temperature) Connection: close

---

GET /api/query?expr=scalar(racktemperature) HTTP/1.1 Host: prometheus.redir.corp.quobyte.com Connection: keep-alive Accept: application/json, text/plain, /_ Origin: http://localhost:3000 User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.125 Safari/537.36 Referer: http://localhost:3000/fx Accept-Encoding: gzip, deflate, sdch Accept-Language: en-US,en;q=0.8,de;q=0.6

[quolix]

This seems to be the problem.

https://en.wikipedia.org/wiki/Same-origin_policy

If install a Chrome plug-in that disables it, it works: https://chrome.google.com/webstore/detail/allow-control-allow-origi/nlfbmbojpeacfghkpbjhddihlkkiljbi?hl=en-US

No clue about JS security, but the original request to the real Prometheus works although it is on a different domain and port. However the redirected one it seems doesn't.

[juliusv]

Ah yes. Prometheus's API sets access control headers that allow usage of the API even cross-domain:

https://github.com/prometheus/prometheus/blob/master/web/api/legacy/query.go#L33-L38

It might be that your proxy needs to return the same headers upon redirection?

[quolix]

Adding the control headers solved it. Calling you Mr. Javascript.

[juliusv]

Woot! :)

[ethicalmohit]

I was getting the same issue then I have found that the PromDash was taking up localhost:9090, Fixing localhost with remote ip as "IP:9090" works for me.