Closed marekgrajcar closed 3 years ago
It looks like another user was able to get cvdupdate working with a proxy this past weekend: https://github.com/micahsnyder/cvdupdate/issues/7
I'd like to verify their solution and (if it works) add it to the README file. If you get it working, would you consider helping me with the documentation?
I am having problems with the proxy settings as well. Our proxy requires authentication and request is not parsing it:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/requests-2.25.1-py3.6.egg/requests/adapters.py", line 412, in send
conn = self.get_connection(request.url, proxies)
File "/usr/local/lib/python3.6/site-packages/requests-2.25.1-py3.6.egg/requests/adapters.py", line 305, in get_connection
proxy_url = parse_url(proxy)
File "/usr/local/lib/python3.6/site-packages/urllib3-1.26.4-py3.6.egg/urllib3/util/url.py", line 392, in parse_url
return six.raise_from(LocationParseError(source_url), None)
File "
It seems to be using the http_proxy and https_proxy environment variables. Tried lots of different combinations but to get this far and the proxy to not reject the connection (407) this is a close as I've gotten.
Hi,
I was able to reproduce solution from https://github.com/micahsnyder/cvdupdate/issues/7. Now I am able to reach for clamav signatures being behind proxy. What I have done is:
In /root/.cvdupdate/config.json I have updated nameserver with proxy address:
"nameserver": "<proxy_ip>",
While running cvd update inside of script I set http_proxy and https_proxy and that usually does a trick:
#!/bin/bash
http_proxy=http://<proxy_ip>:<proxy_port>
export http_proxy
https_proxy=http://<proxy_ip>:<proxy_port>
export https_proxy
/usr/local/bin/cvd update -V
Use a script similar to above with our specific proxy information. We have to use authentication with our proxy. All the attempt to not use authentication string in the URL resulted in a 407:
ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 407 Proxy Authentication Required',
Iteration 1: Set the nameserver kev value in the config.json to one of the ip addresses of our proxy server and got this:
2021-03-18 06:31:50 cvdupdate-0.3.0 DEBUG Checking available versions via DNS TXT entry query of current.cvd.clamav.net 2021-03-18 06:31:50 cvdupdate-0.3.0 INFO Using nameserver specified in the config: 10.10.7.8 2021-03-18 06:31:55 cvdupdate-0.3.0 DEBUG EXCEPTION OCCURRED: The DNS operation timed out after 5.509402751922607 seconds 2021-03-18 06:31:55 cvdupdate-0.3.0 WARNING Failed to determine available version via DNS TXT query! Traceback (most recent call last): File "/usr/local/bin/cvd", line 11, in
load_entry_point('cvdupdate==0.3.0', 'console_scripts', 'cvd')() File "/usr/local/lib/python3.6/site-packages/click-7.1.2-py3.6.egg/click/core.py", line 829, in call return self.main(args, kwargs) File "/usr/local/lib/python3.6/site-packages/click-7.1.2-py3.6.egg/click/core.py", line 782, in main rv = self.invoke(ctx) File "/usr/local/lib/python3.6/site-packages/click-7.1.2-py3.6.egg/click/core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/usr/local/lib/python3.6/site-packages/click-7.1.2-py3.6.egg/click/core.py", line 1066, in invoke return ctx.invoke(self.callback, ctx.params) File "/usr/local/lib/python3.6/site-packages/click-7.1.2-py3.6.egg/click/core.py", line 610, in invoke return callback(args, kwargs) File "/usr/local/lib/python3.6/site-packages/click-7.1.2-py3.6.egg/click/decorators.py", line 21, in new_func return f(get_current_context(), args, kwargs) File "/usr/local/lib/python3.6/site-packages/cvdupdate-0.3.0-py3.6.egg/cvdupdate/main.py", line 259, in update_alias File "/usr/local/lib/python3.6/site-packages/click-7.1.2-py3.6.egg/click/core.py", line 628, in forward return self.invoke(cmd, kwargs) File "/usr/local/lib/python3.6/site-packages/click-7.1.2-py3.6.egg/click/core.py", line 610, in invoke return callback(args, kwargs) File "/usr/local/lib/python3.6/site-packages/cvdupdate-0.3.0-py3.6.egg/cvdupdate/main.py", line 101, in db_update File "/usr/local/lib/python3.6/site-packages/cvdupdate-0.3.0-py3.6.egg/cvdupdate/cvdupdate.py", line 795, in db_update KeyError: ''
Iteration 2: changed the nameserver key value in the config.json to an ip address that was a nameserver and got a little farther:
2021-03-18 06:33:09 cvdupdate-0.3.0 DEBUG Checking available versions via DNS TXT entry query of current.cvd.clamav.net 2021-03-18 06:33:09 cvdupdate-0.3.0 INFO Using nameserver specified in the config: 10.0.0.100 2021-03-18 06:33:09 cvdupdate-0.3.0 DEBUG Checking main.cvd for update from https://database.clamav.net/main.cvd 2021-03-18 06:33:09 cvdupdate-0.3.0 DEBUG Checking main.cvd version via DNS TXT advertisement. 2021-03-18 06:33:09 cvdupdate-0.3.0 DEBUG main.cvd version advertised by DNS: 59 2021-03-18 06:33:09 cvdupdate-0.3.0 DEBUG Downloading CDIFFs first... 2021-03-18 06:33:09 cvdupdate-0.3.0 DEBUG Checking for main-59.cdiff Traceback (most recent call last): File "/usr/local/lib/python3.6/site-packages/requests-2.25.1-py3.6.egg/requests/adapters.py", line 412, in send conn = self.get_connection(request.url, proxies) File "/usr/local/lib/python3.6/site-packages/requests-2.25.1-py3.6.egg/requests/adapters.py", line 305, in get_connection proxy_url = parse_url(proxy) File "/usr/local/lib/python3.6/site-packages/urllib3-1.26.4-py3.6.egg/urllib3/util/url.py", line 392, in parse_url return six.raise_from(LocationParseError(source_url), None) File "
", line 3, in raise_from urllib3.exceptions.LocationParseError: Failed to parse: http://myUsername:myPassword@proxy.domain.com:80/ During handling of the above exception, another exception occurred:``` Traceback (most recent call last): File "/usr/local/bin/cvd", line 11, in
load_entry_point('cvdupdate==0.3.0', 'console_scripts', 'cvd')() File "/usr/local/lib/python3.6/site-packages/click-7.1.2-py3.6.egg/click/core.py", line 829, in call return self.main(args, kwargs) File "/usr/local/lib/python3.6/site-packages/click-7.1.2-py3.6.egg/click/core.py", line 782, in main rv = self.invoke(ctx) File "/usr/local/lib/python3.6/site-packages/click-7.1.2-py3.6.egg/click/core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/usr/local/lib/python3.6/site-packages/click-7.1.2-py3.6.egg/click/core.py", line 1066, in invoke return ctx.invoke(self.callback, ctx.params) File "/usr/local/lib/python3.6/site-packages/click-7.1.2-py3.6.egg/click/core.py", line 610, in invoke return callback(args, kwargs) File "/usr/local/lib/python3.6/site-packages/click-7.1.2-py3.6.egg/click/decorators.py", line 21, in new_func return f(get_current_context(), args, kwargs) File "/usr/local/lib/python3.6/site-packages/cvdupdate-0.3.0-py3.6.egg/cvdupdate/main.py", line 259, in update_alias File "/usr/local/lib/python3.6/site-packages/click-7.1.2-py3.6.egg/click/core.py", line 628, in forward return self.invoke(cmd, kwargs) File "/usr/local/lib/python3.6/site-packages/click-7.1.2-py3.6.egg/click/core.py", line 610, in invoke return callback(args, kwargs) File "/usr/local/lib/python3.6/site-packages/cvdupdate-0.3.0-py3.6.egg/cvdupdate/main.py", line 101, in db_update File "/usr/local/lib/python3.6/site-packages/cvdupdate-0.3.0-py3.6.egg/cvdupdate/cvdupdate.py", line 859, in db_update File "/usr/local/lib/python3.6/site-packages/cvdupdate-0.3.0-py3.6.egg/cvdupdate/cvdupdate.py", line 845, in update File "/usr/local/lib/python3.6/site-packages/cvdupdate-0.3.0-py3.6.egg/cvdupdate/cvdupdate.py", line 661, in _download_cvd File "/usr/local/lib/python3.6/site-packages/requests-2.25.1-py3.6.egg/requests/api.py", line 76, in get return request('get', url, params=params, kwargs) File "/usr/local/lib/python3.6/site-packages/requests-2.25.1-py3.6.egg/requests/api.py", line 61, in request return session.request(method=method, url=url, kwargs) File "/usr/local/lib/python3.6/site-packages/requests-2.25.1-py3.6.egg/requests/sessions.py", line 542, in request resp = self.send(prep, send_kwargs) File "/usr/local/lib/python3.6/site-packages/requests-2.25.1-py3.6.egg/requests/sessions.py", line 655, in send r = adapter.send(request, kwargs) File "/usr/local/lib/python3.6/site-packages/requests-2.25.1-py3.6.egg/requests/adapters.py", line 414, in send raise InvalidURL(e, request=request) requests.exceptions.InvalidURL: Failed to parse: http://myUser:myPassword@proxy.domain.com:80/
Finally got it to work. had to update our proxy with the required user-agent to allow it through unauthenticated.
Set the name server to one of our real name servers; Set the httpd_proxy and https_proxy environment variables to our proxy FQDN:port (multiple proxy addresses)
Great! I'll try to summarize all of this in the Readme when time permits. I'll keep this ticket open until I've done so.
Hello, due to the latest changes I am trying to download signatures using cvdupdate tool. I am performing it from inside of company’s network and I get warning: “Failed to determine available version via DNS TXT query!”. However when I am looking for address names resolution using our internal proxy I am able to get an answer:
Proxy server can forward DNS requests but cannot act as name server. When updating “nameserver” in /root/.cvdupdate/config.json with proxy address I got timeout like below:
I am wondering if it’s and option to forward request via proxy correctly, i.e. putting proxy ip in /root/.cvdupdate/config.json as a feature.