OpenVPN Connect Client 3.download.recipe – URLDownloader error 403

[opus-nbonenberger]

The download recipe for OpenVPN Connect fails because a 403 Error (Forbidden). Maybe it's a kind of access protection? Saw a Cloudflare like "checking" screen when manually opening the URL in the browser.

URLDownloader
{'Input': {'request_headers': {'Referer': 'https://openvpn.net/client-connect-vpn-for-mac-os/;auto',
                               'user-agent': 'Mozilla/5.0 (Macintosh; Intel '
                                             'Mac OS X 10_15_7) '
                                             'AppleWebKit/605.1.15 (KHTML, '
                                             'like Gecko) Version/16.0 '
                                             'Safari/605.1.15'},
           'url': 'https://openvpn.net/downloads/openvpn-connect-v3-macos.dmg'}}
URLDownloader: No value supplied for prefetch_filename, setting default value of: False
URLDownloader: No value supplied for CHECK_FILESIZE_ONLY, setting default value of: False
URLDownloader: ERROR: (22) The requested URL returned error: 403

Traceback (most recent call last):
  File "/Library/AutoPkg/autopkglib/URLGetter.py", line 172, in execute_curl
    result = subprocess.run(
  File "/Library/AutoPkg/Python3/Python.framework/Versions/3.10/lib/python3.10/subprocess.py", line 524, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/usr/bin/curl', '--silent', '--show-error', '--no-buffer', '--dump-header', '-', '--speed-time', '30', '--location', '--url', 'https://openvpn.net/downloads/openvpn-connect-v3-macos.dmg', '--fail', '--output', '/Users/admin/Library/AutoPkg/Cache/local.munki.OpenVPN Connect Client 3_arm64/downloads/tmpjjh4vz42', '--header', 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.0 Safari/605.1.15', '--header', 'Referer: https://openvpn.net/client-connect-vpn-for-mac-os/;auto', '--header', 'If-None-Match: "661b2403608bb904ff4eeddf1665713c-13"', '--header', 'If-Modified-Since: Wed, 07 Feb 2024 13:52:47 GMT']' returned non-zero exit status 22.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Library/AutoPkg/autopkglib/__init__.py", line 840, in process
    self.env = processor.process()
  File "/Library/AutoPkg/autopkglib/__init__.py", line 626, in process
    self.main()
  File "/Library/AutoPkg/autopkglib/URLDownloader.py", line 331, in main
    raw_headers = self.download_with_curl(curl_cmd)
  File "/Library/AutoPkg/autopkglib/URLGetter.py", line 186, in download_with_curl
    proc_stdout, proc_stderr, retcode = self.execute_curl(curl_cmd, text)
  File "/Library/AutoPkg/autopkglib/URLGetter.py", line 181, in execute_curl
    raise ProcessorError(e.stderr) from e
autopkglib.ProcessorError: curl: (22) The requested URL returned error: 403

  File "/Library/AutoPkg/autopkglib/__init__.py", line 840, in process
    self.env = processor.process()
curl: (22) The requested URL returned error: 403

Failed.

[paul-cossey]

Thanks for raising, @opus-nbonenberger

I'm seeing the same. Not sure how, or even if we can get round this. 🤔

[opus-nbonenberger]

@paul-cossey I googled how to bypass Cloudflare with curl and found something on https://www.zenrows.com/blog/curl-bypass-cloudflare#http-headers.

I tested:

curl https://openvpn.net/downloads/openvpn-connect-v3-macos.dmg -v -L -H "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9" -H "Accept-Encoding: gzip, deflate" -H "Accept-Language: en-US,en;q=0.9" -H "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" --output "/Users/admin/Desktop/OpenVPN.dmg"

It seems to work here.

Can you add the additional arguments to the curl command in the recipe and test it?

[paul-cossey]

Thanks, @opus-nbonenberger

Funnily enough I had started adding all of those yesterday in testing, just didn't end up finishing.

I've now added them all it works for me: https://github.com/autopkg/dataJAR-recipes/commit/f67a7843a629b0a5e0269e4352b41be86046e393