`aws s3 sync` issues warning for files that are excluded

[MLBZ521]

Looks like the sync command's --exclude is still yelling about files that are excluded, which is causing JamfPackageUploader to fail.

com.github.grahampugh.jamf-upload.processors/JamfPackageUploader
{'Input': {'API_PASSWORD': '<redacted>',
           'API_USERNAME': '<redacted>',
           'JSS_URL': 'https://<redacted>:8443',
           'os_requirements': '',
           'pkg_category': 'Creativity',
           'pkg_notes': 'Product of AutoPkg',
           'pkg_path': '/Volumes/Storage/AutoPkg/Cache/local.Snagit/Snagit '
                       '(Universal)-2024.2.4.pkg',
           'sleep': '0',
           'version': '2024.2.4'}}
JamfPackageUploader: No value supplied for CLOUD_DP, setting default value of: False
JamfPackageUploader: No value supplied for SMB_URL, setting default value of: 
JamfPackageUploader: No value supplied for SMB_USERNAME, setting default value of: 
JamfPackageUploader: No value supplied for SMB_PASSWORD, setting default value of: 
JamfPackageUploader: No value supplied for pkg_name, setting default value of: 
JamfPackageUploader: No value supplied for pkg_display_name, setting default value of: 
JamfPackageUploader: No value supplied for pkg_info, setting default value of: 
JamfPackageUploader: No value supplied for pkg_priority, setting default value of: 10
JamfPackageUploader: No value supplied for reboot_required, setting default value of: 
JamfPackageUploader: No value supplied for required_processor, setting default value of: None
JamfPackageUploader: No value supplied for send_notification, setting default value of: 
JamfPackageUploader: No value supplied for replace_pkg, setting default value of: False
JamfPackageUploader: No value supplied for jcds_mode, setting default value of: False
JamfPackageUploader: No value supplied for jcds2_mode, setting default value of: False
JamfPackageUploader: No value supplied for replace_pkg_metadata, setting default value of: False
JamfPackageUploader: No value supplied for skip_metadata_upload, setting default value of: False
JamfPackageUploader: Checking for existing package 'Snagit (Universal)-2024.2.4.pkg' on https://<redacted>:8443
JamfPackageUploader: Checking for existing authentication token
JamfPackageUploader: Checking https://<redacted>:8443 against https://<redacted>:8443
JamfPackageUploader: URL and user for token matches current request
JamfPackageUploader: Existing token is valid
JamfPackageUploader: Output file is:  /tmp/jamf_upload_3oqenou5/jamf_upload_lsop6tzv.txt
JamfPackageUploader: curl command: /usr/bin/curl --dump-header /tmp/jamf_upload_3oqenou5/curl_headers_from_jamf_upload.txt https://<redacted>:8443/JSSResource/packages/name/Snagit%20%28Universal%29-2024.2.4.pkg --request GET --silent --show-error --header authorization: Bearer <redacted> --header Accept: application/json --output /tmp/jamf_upload_3oqenou5/jamf_upload_lsop6tzv.txt --cookie-jar /tmp/jamf_upload_3oqenou5/curl_cookies_from_jamf_upload.txt --cookie /tmp/jamf_upload_3oqenou5/curl_cookies_from_jamf_upload.txt
JamfPackageUploader: ID: -1
JamfPackageUploader: Package 'Snagit (Universal)-2024.2.4.pkg' not found on server
JamfPackageUploader: Number of File Share DPs: 0
JamfPackageUploader: Handling Cloud Distribution Point
Traceback (most recent call last):
  File \"/Volumes/Storage/AutoPkg/RecipeRepos/com.github.autopkg.grahampugh-recipes/JamfUploaderProcessors/JamfUploaderLib/JamfPackageUploaderBase.py\", line 448, in upload_to_aws_s3_bucket
    aws_output = subprocess.check_output(aws_cmd)
  File \"/Library/AutoPkg/Python3/Python.framework/Versions/3.10/lib/python3.10/subprocess.py\", line 420, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File \"/Library/AutoPkg/Python3/Python.framework/Versions/3.10/lib/python3.10/subprocess.py\", line 524, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/usr/local/bin/aws', 's3', 'sync', '/Volumes/Storage/AutoPkg/Cache/local.Snagit/', 's3://<redacted>/', '--exclude', '*', '--include', 'Snagit (Universal)-2024.2.4.pkg', '--output', 'text']' returned non-zero exit status 2.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File \"/Library/AutoPkg/autopkglib/__init__.py\", line 905, in process
    self.env = processor.process()
  File \"/Library/AutoPkg/autopkglib/__init__.py\", line 691, in process
    self.main()
  File \"/Volumes/Storage/AutoPkg/RecipeRepos/com.github.autopkg.grahampugh-recipes/JamfUploaderProcessors/JamfPackageUploader.py\", line 260, in main
    self.execute()
  File \"/Volumes/Storage/AutoPkg/RecipeRepos/com.github.autopkg.grahampugh-recipes/JamfUploaderProcessors/JamfUploaderLib/JamfPackageUploaderBase.py\", line 852, in execute
    self.upload_to_aws_s3_bucket(self.pkg_path, self.pkg_name)
  File \"/Volumes/Storage/AutoPkg/RecipeRepos/com.github.autopkg.grahampugh-recipes/JamfUploaderProcessors/JamfUploaderLib/JamfPackageUploaderBase.py\", line 450, in upload_to_aws_s3_bucket
    raise ProcessorError(f\"Error from aws: {exc}\") from exc
autopkglib.ProcessorError: Error from aws: Command '['/usr/local/bin/aws', 's3', 'sync', '/Volumes/Storage/AutoPkg/Cache/local.Snagit/', 's3://<redacted>/', '--exclude', '*', '--include', 'Snagit (Universal)-2024.2.4.pkg', '--output', 'text']' returned non-zero exit status 2.
  File \"/Library/AutoPkg/autopkglib/__init__.py\", line 905, in process
    self.env = processor.process()
Receipt written to /Volumes/Storage/AutoPkg/Cache/local.Snagit/receipts/local.Snagit-receipt-20240513-200726.plist

The following recipes failed:
    local.Snagit
        Error in local.Snagit: Processor: com.github.grahampugh.jamf-upload.processors/JamfPackageUploader: Error: Error from aws: Command '['/usr/local/bin/aws', 's3', 'sync', '/Volumes/Storage/AutoPkg/Cache/local.Snagit/', 's3://<redacted>/', '--exclude', '*', '--include', 'Snagit (Universal)-2024.2.4.pkg', '--output', 'text']' returned non-zero exit status 2.

Testing outside of JamfUploader produces the same issue:

% /usr/local/bin/aws s3 sync --dryrun '/Volumes/Storage/AutoPkg/Cache/local.Snagit/' 's3://<redacted>/' --exclude '*' --exclude '*/*' --include 'Snagit (Universal)-2024.2.4.pkg' --output 'text'
(dryrun) upload: ../../Volumes/Storage/AutoPkg/Cache/local.Snagit/Snagit (Universal)-2024.2.4.pkg to s3://<redacted>/Snagit (Universal)-2024.2.4.pkg
warning: Skipping file /Volumes/Storage/AutoPkg/Cache/local.Snagit/Snagit/Applications/Snagit 2024.app/Contents/Frameworks/Sparkle.framework/XPCServices. File does not exist.
% echo $?
2

So, obviously not a JamfUploader issue, but.....not sure if it's something that should be accounted for in JamfUploader or hope we can get parent recipe maintainers to update their recipes. (I have a PR in for this recipe that would prevent this, but...that's been open since October 2023...)

[grahampugh]

I did encounter this error under certain circumstances when building AWS mode (see the note in my blog post about that).

https://grahamrpugh.com/2024/03/06/uploading-packages-to-aws-cdp.html

Note that AWS mode won't be needed once upgrading to 11.5 (more on that once it is released).

[MLBZ521]

Ah, ok. I read the blog post a while ago, just forgot about that part. Will look forward to Jamf Pro 11.5 then....as well as will try to get that PR merged.

Thanks Graham