proxy settings

[niskrev]

Hi, How can I specify the proxy server through which the HTTP requests are made? After get_data call I get ERROR; HTTP/1.1 407 Proxy Authentication Required Thanks!

[micahjsmith]

I presume that you are behind some corporate firewall?

FredData.jl uses HTTP.jl to make requests. Searching for proxy in their documentation, I see

Alternatively, HTTP.jl also respects the http_proxy, HTTP_PROXY, https_proxy, HTTPS_PROXY, and no_proxy environment variables; if set, they will be used automatically when making requests.

You can search around to see whether setting these variables appropriately on your OS resolves the issue

[niskrev]

@micahjsmith Thanks for replying. I tried setting the environment variables for HTTP_PROXY and HTTPS_PROXY but still get the following error:

ERROR: HTTP.ExceptionRequest.StatusError(407, "CONNECT", "api.stlouisfed.org:443", HTTP.Messages.Response:
"""
HTTP/1.1 407 Proxy Authentication Required
Proxy-Authenticate: NEGOTIATE, NTLM, BASIC realm="EP_SRE_AD1_BDP"
Cache-Control: no-cache
Pragma: no-cache
Content-Type: text/html; charset=utf-8
Proxy-Connection: close
Connection: close
Content-Length: 849

""")

Indeed, I am behind a corporate firewall. But in python, I am able to use the get method from the requests library by passing a proxies argument, e.g

import requests
proxies = {
  'http': 'http://user:pass@10.10.1.10:3128/'
  'https': 'https://user:pass@10.10.1.10:3128/'
}
requests.get('http://example.org', proxies=proxies)

Is it possible to pass proxies explicitly with Fred? Thanks!

[micahjsmith]

FredData.jl uses HTTP.jl and makes calls to the HTTP.request method. This method takes an optional proxy argument, per the same doc page I linked above. This would be analogous to your python requests usage.

However, according to the HTTP.jl docs, the behavior from providing an explicit proxy argument is stated to be identical to setting one of those environment variables. Can I hypothesize that those environment variables may have not been set correctly or were not made available to your Julia session?

To debug,

• print ENV("HTTP_PROXY") from the Julia REPL where you are using FredData, and confirm that the value is set appropriately. I suspect that this will not be set, and you need to export the environment variable
• try to make an API call to Fred directly using HTTP.jl, and confirm that this fails

import HTTP
url = "https://api.stlouisfed.org/fred/series/observations"
params = Dict("series" => "GDPC1", "api_key" => "my_api_key")
HTTP.request("GET", url, []; query=params)

• try again and confirm that this succeeds

import HTTP
url = "https://api.stlouisfed.org/fred/series/observations"
params = Dict("series" => "GDPC1", "api_key" => "my_api_key")
HTTP.request("GET", url, []; query=params, proxy="http://your-proxy-url")

[niskrev]

Thanks for the suggestions @micahjsmith I understand that my problem is that the HTTP calls are somehow wrong. But the problem doesn't seem to be because of how I set the ENV variables. In python with

url="https://api.stlouisfed.org/fred/series?series_id=GDPC1&api_key=my_fred_api_key"
proxystr = "my_proxy"
r = requests.get(
                url
                proxies={"https": proxystr}
               )

I get the correct response. In Julia, with the same url and proxystr r = HTTP.request("GET", url, proxy=proxystr) raises "Proxy Authentication Required" error

Setting

ENV["HTTP_PROXY"] = proxystr
ENV["HTTPS_PROXY"] = proxystr

in the beginning of the session and then making the HTTP request results in the same error.