search

grahampugh / jamf-upload

Scripts for uploading packages to Jamf Cloud
Apache License 2.0
150 stars 38 forks source link

jamf-upload processor not handling return codes... #25

Closed everetteallen closed 3 years ago

everetteallen commented 3 years ago

So I am seeing mixed messages in a verbose run of the jamf-upload processor. Here is the confusing output:

JamfPackageUploader: HTTP response: <property object at 0x7fecd58342f0>
JamfPackageUploader: Package uploaded successfully, ID=8831
JamfPackageUploader: WARNING: Package 'UNCSystem-AutoPkg-Rosetta-2-install-1.0.signed.pkg' upload failed
JamfPackageUploader: b'<?xml version="1.0" encoding="UTF-8" standalone="yes"?><uploadResponse><id>8831</id><md5>a18574a191df0be522cf55d168c04d60</md5><message></message><receivedSize>14252</receivedSize><successful>true</successful></uploadResponse>'
ERROR: Package upload failed.
Failed.

Looks like the HTTP response is not being decoded properly but I can't tell for sure. The sanitized full output is below along with /tmp/jamf_upload/curl_output_from_jamf_upload.txt I am seeing this from 2 10.15.7 devices with Jamf Pro 10.27.0 and 10.28.0. The package does in fact upload and eventually become available. Also note that if I run the same command a second time it fails to detect that the package with the same name already exists. Thanks!

 tail -f /tmp/jamf_upload/curl_output_from_jamf_upload.txt 
   <title>Status page</title>
</head>
<body style="font-family: sans-serif;">
<p style="font-size: 1.2em;font-weight: bold;margin: 1em 0px;">Not Found</p>
<p>The server has not found anything matching the request URI</p>
<p>You can get technical details <a href="http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4.5">here</a>.<br>
Please continue your visit at our <a href="/">home page</a>.
</p>
</body>
</html>
<?xml version="1.0" encoding="UTF-8" standalone="yes"?><uploadResponse><id>8831</id><md5>a18574a191df0be522cf55d168c04d60</md5><message></message><receivedSize>14252</receivedSize><successful>true</successful></uploadResponse>^C

Full -vv output from autopkg sanitized

autopkg run --recipe-list=/Users/user/Library/AutoPkg/recipe_list.txt -vv
Processing com.github.grahampugh.recipes.download.rosetta...
WARNING: com.github.grahampugh.recipes.download.rosetta is missing trust info and FAIL_RECIPES_WITHOUT_TRUST_INFO is not set. Proceeding...
GitHubReleasesInfoProvider
{'Input': {'asset_regex': '^Rosetta-2-install.*?.signed.pkg$',
           'github_repo': 'grahampugh/Rosetta-2-install',
           'include_prereleases': True}}
GitHubReleasesInfoProvider: No value supplied for CURL_PATH, setting default value of: /usr/bin/curl
GitHubReleasesInfoProvider: No value supplied for GITHUB_URL, setting default value of: https://api.github.com
GitHubReleasesInfoProvider: No value supplied for GITHUB_TOKEN_PATH, setting default value of: ~/.autopkg_gh_token
GitHubReleasesInfoProvider: Matched regex '^Rosetta-2-install.*?.signed.pkg$' among asset(s): Rosetta-2-install-1.0.pkg, Rosetta-2-install-1.0.signed.pkg
GitHubReleasesInfoProvider: Selected asset 'Rosetta-2-install-1.0.signed.pkg' from release '1.0'
{'Output': {'url': 'https://github.com/grahampugh/Rosetta-2-install/releases/download/v1.0/Rosetta-2-install-1.0.signed.pkg',
            'version': '1.0'}}
URLDownloader
{'Input': {'url': 'https://github.com/grahampugh/Rosetta-2-install/releases/download/v1.0/Rosetta-2-install-1.0.signed.pkg'}}
URLDownloader: No value supplied for prefetch_filename, setting default value of: False
URLDownloader: No value supplied for CHECK_FILESIZE_ONLY, setting default value of: False
URLDownloader: Storing new Last-Modified header: Wed, 25 Nov 2020 11:16:21 GMT
URLDownloader: Storing new ETag header: "a18574a191df0be522cf55d168c04d60"
URLDownloader: Downloaded /Users/user/Library/AutoPkg/Cache/com.github.grahampugh.recipes.download.rosetta/downloads/Rosetta-2-install-1.0.signed.pkg
{'Output': {'download_changed': True,
            'etag': '"a18574a191df0be522cf55d168c04d60"',
            'last_modified': 'Wed, 25 Nov 2020 11:16:21 GMT',
            'pathname': '/Users/user/Library/AutoPkg/Cache/com.github.grahampugh.recipes.download.rosetta/downloads/Rosetta-2-install-1.0.signed.pkg',
            'url_downloader_summary_result': {'data': {'download_path': '/Users/user/Library/AutoPkg/Cache/com.github.grahampugh.recipes.download.rosetta/downloads/Rosetta-2-install-1.0.signed.pkg'},
                                              'summary_text': 'The following '
                                                              'new items were '
                                                              'downloaded:'}}}
EndOfCheckPhase
{'Input': {}}
{'Output': {}}
CodeSignatureVerifier
{'Input': {'expected_authority_names': ['Developer ID Installer: ETH Zurich '
                                        '(9Z623UDZ5L)',
                                        'Developer ID Certification Authority',
                                        'Apple Root CA'],
           'input_path': '/Users/user/Library/AutoPkg/Cache/com.github.grahampugh.recipes.download.rosetta/downloads/Rosetta-2-install-1.0.signed.pkg'}}
CodeSignatureVerifier: Verifying installer package signature...
CodeSignatureVerifier: Package "Rosetta-2-install-1.0.signed.pkg":
CodeSignatureVerifier:    Status: signed by a developer certificate issued by Apple for distribution
CodeSignatureVerifier:    Signed with a trusted timestamp on: 2020-11-25 11:08:08 +0000
CodeSignatureVerifier:    Certificate Chain:
CodeSignatureVerifier:     1. Developer ID Installer: ETH Zurich (9Z623UDZ5L)
CodeSignatureVerifier:        Expires: 2023-07-14 11:57:10 +0000
CodeSignatureVerifier:        SHA256 Fingerprint:
CodeSignatureVerifier:            F3 D1 C1 96 3F BD 08 64 92 F4 05 77 57 0B 49 8C 68 47 85 DA 77 AB 
CodeSignatureVerifier:            F2 BE 1C DD BC 0B A6 51 15 F7
CodeSignatureVerifier:        ------------------------------------------------------------------------
CodeSignatureVerifier:     2. Developer ID Certification Authority
CodeSignatureVerifier:        Expires: 2027-02-01 22:12:15 +0000
CodeSignatureVerifier:        SHA256 Fingerprint:
CodeSignatureVerifier:            7A FC 9D 01 A6 2F 03 A2 DE 96 37 93 6D 4A FE 68 09 0D 2D E1 8D 03 
CodeSignatureVerifier:            F2 9C 88 CF B0 B1 BA 63 58 7F
CodeSignatureVerifier:        ------------------------------------------------------------------------
CodeSignatureVerifier:     3. Apple Root CA
CodeSignatureVerifier:        Expires: 2035-02-09 21:40:36 +0000
CodeSignatureVerifier:        SHA256 Fingerprint:
CodeSignatureVerifier:            B0 B1 73 0E CB C7 FF 45 05 14 2C 49 F1 29 5E 6E DA 6B CA ED 7E 2C 
CodeSignatureVerifier:            68 C5 BE 91 B5 A1 10 01 F0 24
CodeSignatureVerifier: 
CodeSignatureVerifier: Signature is valid
CodeSignatureVerifier: Authority name chain is valid
{'Output': {}}
io.github.hjuutilainen.VirusTotalAnalyzer/VirusTotalAnalyzer
{'Input': {'CURL_PATH': '/usr/bin/curl',
           'VIRUSTOTAL_ALWAYS_REPORT': True,
           'VIRUSTOTAL_AUTO_SUBMIT': True,
           'pathname': '/Users/user/Library/AutoPkg/Cache/com.github.grahampugh.recipes.download.rosetta/downloads/Rosetta-2-install-1.0.signed.pkg'}}
VirusTotalAnalyzer: Calculating checksum for /Users/user/Library/AutoPkg/Cache/com.github.grahampugh.recipes.download.rosetta/downloads/Rosetta-2-install-1.0.signed.pkg
VirusTotalAnalyzer: Requesting report...
VirusTotalAnalyzer: Response code: 1
VirusTotalAnalyzer: Message: Scan finished, information embedded
VirusTotalAnalyzer: Scan ID: b6ef466e0503ac824f705abd460c8a8e47b3b48ebaceb57a5d2c7f2415a35992-1617213903
VirusTotalAnalyzer: Detection ratio: 0/56
VirusTotalAnalyzer: Scan date: 2021-03-31 18:05:03
VirusTotalAnalyzer: Permalink: https://www.virustotal.com/gui/file/b6ef466e0503ac824f705abd460c8a8e47b3b48ebaceb57a5d2c7f2415a35992/detection/f-b6ef466e0503ac824f705abd460c8a8e47b3b48ebaceb57a5d2c7f2415a35992-1617213903
{'Output': {'virus_total_analyzer_summary_result': {'data': {'name': 'Rosetta-2-install-1.0.signed.pkg',
                                                             'permalink': 'https://www.virustotal.com/gui/file/b6ef466e0503ac824f705abd460c8a8e47b3b48ebaceb57a5d2c7f2415a35992/detection/f-b6ef466e0503ac824f705abd460c8a8e47b3b48ebaceb57a5d2c7f2415a35992-1617213903',
                                                             'ratio': '0/56'},
                                                    'report_fields': ['name',
                                                                      'ratio',
                                                                      'permalink'],
                                                    'summary_text': 'The '
                                                                    'following '
                                                                    'items '
                                                                    'were '
                                                                    'queried '
                                                                    'from the '
                                                                    'VirusTotal '
                                                                    'database:'}}}
com.github.everetteallen.ega-recipes.postprocessors/PKGPrefixer
{'Input': {'pkg_prefix': 'UNCSystem-AutoPkg-'}}
PKGPrefixer: No value supplied for pkg_name, setting default value of: 
PKGPrefixer: No value supplied for pkg_path, setting default value of: 
{'Output': {'pkg_name': 'UNCSystem-AutoPkg-Rosetta-2-install-1.0.signed.pkg',
            'pkg_path': '/Users/user/Library/AutoPkg/Cache/com.github.grahampugh.recipes.download.rosetta/downloads/UNCSystem-AutoPkg-Rosetta-2-install-1.0.signed.pkg',
            'pkg_prefixer_summary_result': {'data': {'pkg_name': 'UNCSystem-AutoPkg-Rosetta-2-install-1.0.signed.pkg',
                                                     'pkg_path': '/Users/user/Library/AutoPkg/Cache/com.github.grahampugh.recipes.download.rosetta/downloads/Rosetta-2-install-1.0.signed.pkg'},
                                            'report_fields': ['pkg_path',
                                                              'pkg_name'],
                                            'summary_text': 'The following '
                                                            'package was '
                                                            'renamed:'}}}
com.github.grahampugh.jamf-upload.processors/JamfPackageUploader
{'Input': {'API_PASSWORD': '<uwish>',
           'API_USERNAME': 'autopkg',
           'JSS_URL': 'https://nope.jamfcloud.com',
           'pkg_category': 'aaUNC-System',
           'pkg_name': 'UNCSystem-AutoPkg-Rosetta-2-install-1.0.signed.pkg',
           'pkg_path': '/Users/user/Library/AutoPkg/Cache/com.github.grahampugh.recipes.download.rosetta/downloads/UNCSystem-AutoPkg-Rosetta-2-install-1.0.signed.pkg',
           'replace_pkg': 'False',
           'version': '1.0'}}
JamfPackageUploader: No value supplied for pkg_info, setting default value of: 
JamfPackageUploader: No value supplied for pkg_notes, setting default value of: 
JamfPackageUploader: No value supplied for pkg_priority, setting default value of: 10
JamfPackageUploader: No value supplied for reboot_required, setting default value of: 
JamfPackageUploader: No value supplied for os_requirement, setting default value of: 
JamfPackageUploader: No value supplied for required_processor, setting default value of: None
JamfPackageUploader: No value supplied for send_notification, setting default value of: 
JamfPackageUploader: No value supplied for replace_pkg_metadata, setting default value of: False
JamfPackageUploader: No value supplied for SMB_URL, setting default value of: 
JamfPackageUploader: No value supplied for SMB_USERNAME, setting default value of: 
JamfPackageUploader: No value supplied for SMB_PASSWORD, setting default value of: 
JamfPackageUploader: Checking for existing 'UNCSystem-AutoPkg-Rosetta-2-install-1.0.signed.pkg' on https://nc.jamfcloud.com
JamfPackageUploader: No existing cookie found - starting new session
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   439  100   439    0     0   2660      0 --:--:-- --:--:-- --:--:--  2660
JamfPackageUploader: No existing cookie found - starting new session
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 14478    0   226  100 14252      0     56  0:04:14  0:04:14 --:--:--    51
JamfPackageUploader: HTTP response: <property object at 0x7fecd58342f0>
JamfPackageUploader: Package uploaded successfully, ID=8831
JamfPackageUploader: WARNING: Package 'UNCSystem-AutoPkg-Rosetta-2-install-1.0.signed.pkg' upload failed
JamfPackageUploader: b'<?xml version="1.0" encoding="UTF-8" standalone="yes"?><uploadResponse><id>8831</id><md5>a18574a191df0be522cf55d168c04d60</md5><message></message><receivedSize>14252</receivedSize><successful>true</successful></uploadResponse>'
ERROR: Package upload failed.
Failed.
Receipt written to /Users/user/Library/AutoPkg/Cache/com.github.grahampugh.recipes.download.rosetta/receipts/com.github.grahampugh.recipes.download-receipt-20210401-130815.plist

The following recipes failed:
    com.github.grahampugh.recipes.download.rosetta
        Error in com.github.grahampugh.recipes.download.rosetta: Processor: com.github.grahampugh.jamf-upload.processors/JamfPackageUploader: Error: ERROR: Package upload failed.

The following new items were downloaded:
    Download Path                                                                                                               
    -------------                                                                                                               
    /Users/user/Library/AutoPkg/Cache/com.github.grahampugh.recipes.download.rosetta/downloads/Rosetta-2-install-1.0.signed.pkg  

The following items were queried from the VirusTotal database:
    Name                              Ratio  Permalink                                                                                                                                                                                     
    ----                              -----  ---------                                                                                                                                                                                     
    Rosetta-2-install-1.0.signed.pkg  0/56   https://www.virustotal.com/gui/file/b6ef466e0503ac824f705abd460c8a8e47b3b48ebaceb57a5d2c7f2415a35992/detection/f-b6ef466e0503ac824f705abd460c8a8e47b3b48ebaceb57a5d2c7f2415a35992-1617213903  

The following package was renamed:
    Pkg Path                                                                                                                    Pkg Name                                            
    --------                                                                                                                    --------                                            
    /Users/user/Library/AutoPkg/Cache/com.github.grahampugh.recipes.download.rosetta/downloads/Rosetta-2-install-1.0.signed.pkg  UNCSystem-AutoPkg-Rosetta-2-install-1.0.signed.pkg
grahampugh commented 3 years ago

Can you check with the latest push? I've merged in some changes that address HTTP/2 responses.

everetteallen commented 3 years ago

@grahampugh That push seems to have fixed the issue. I now see response codes and correct behavior for uploading or not if existing package and the category is getting set as expected. Thanks!

grahampugh commented 3 years ago

Awesome!