homebysix / recipe-robot

A kick ass tool for creating AutoPkg recipes.
Apache License 2.0
300 stars 28 forks source link

Problem using Recipe-Robot to download app #122

Closed tbridge closed 7 years ago

tbridge commented 7 years ago

Hello Elliot & Team!

First - a heartfelt thank you. This app is a huge help to the community, and I appreciate all your efforts. Let's get a pint at MacADUK if you're around :)

We have an app that one of our clients relies on, that is updated frequently, and has a static install URL. The URL works in a browser, but when Recipe-Robot attempts to download it, it times out:

Antipodes:~ tom$ /Applications/Recipe\ Robot.app/Contents/Resources/scripts/recipe-robot --ignore-existing --verbose https://citadel.thinkingphones.com/api/v1/applications/fuzeapp/mac/versions/latest/download/installer
[WARNING] Will build recipes even if they already exist in "autopkg search" results. Please don't upload duplicate recipes to public AutoPkg repos on GitHub.

                      -----------------------------------
                     |  Welcome to Recipe Robot v1.0.4.  |
                      -----------------------------------
                                \   _[]_
                                 \  [oo]
                                   d-||-b
                                     ||
                                   _/  \_

Processing https://citadel.thinkingphones.com/api/v1/applications/fuzeapp/mac/versions/latest/download/installer ...
Input path looks like a download URL.
    Download URL is: https://citadel.thinkingphones.com/api/v1/applications/fuzeapp/mac/versions/latest/download/installer
[ERROR] Recipe Robot exploded with unexpected error: 
Traceback (most recent call last):
  File "/Applications/Recipe Robot.app/Contents/Resources/scripts/recipe-robot", line 95, in main
    process_input_path(facts)
  File "/Applications/Recipe Robot.app/Contents/Resources/scripts/recipe_robot_lib/inspect.py", line 131, in process_input_path
    facts = inspect_func(input_path, args, facts)
  File "/Applications/Recipe Robot.app/Contents/Resources/scripts/recipe_robot_lib/inspect.py", line 882, in inspect_download_url
    checked_url = check_url(input_path)
  File "/Applications/Recipe Robot.app/Contents/Resources/scripts/recipe_robot_lib/inspect.py", line 169, in check_url
    r = c.getresponse()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 1132, in getresponse
    response.begin()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 453, in begin
    version, status, reason = self._read_status()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 409, in _read_status
    line = self.fp.readline(_MAXLINE + 1)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 480, in readline
    data = self._sock.recv(self._rbufsize)

I'm not sure how to fix this particular error, but it is consistent between systems.

homebysix commented 7 years ago

Thanks @tbridge for the kind words!

At first glance, I thought you might be getting that error due to some sort of user-agent blocking being done on citadel.thinkingphones.com. So I ran the URL through curl -I just to see the headers:

$ curl -I "https://citadel.thinkingphones.com/api/v1/applications/fuzeapp/mac/versions/latest/download/installer"
HTTP/1.1 307 Temporary Redirect
Content-Length: 0
Date: Thu, 05 Jan 2017 18:45:36 GMT
Location: https://d27a81e2f9iafm.cloudfront.net/live/distribution/fuzeapp/mac/16.12.13601/Fuze.dmg
Server: nginx/1.10.1
Connection: keep-alive

As you can see from the "307" response code and the Location header, it's not actually blocking us, just pointing us to a different URL. So I put that URL through Recipe Robot instead, and that produced a working download recipe.

However, the DOWNLOAD_URL was tied to a static version of the app, so I replaced the version-specific URL in the download recipe:

<key>DOWNLOAD_URL</key>
<string>https://d27a81e2f9iafm.cloudfront.net/live/distribution/fuzeapp/mac/16.12.13601/Fuze.dmg</string>

With the original "latest" URL you provided earlier:

<key>DOWNLOAD_URL</key>
<string>https://citadel.thinkingphones.com/api/v1/applications/fuzeapp/mac/versions/latest/download/installer</string>

And that modified download recipe seems to work great. Give that a shot!