search

grahampugh / jamf-upload

Scripts for uploading packages to Jamf Cloud
Apache License 2.0
149 stars 37 forks source link

JCDS mode: upload working, but policy can't find package #100

Closed inglenny closed 1 year ago

inglenny commented 1 year ago

I updated to the latest version of jamfuploader and corrected local trust. However I'm having issues with jcds upload. API access rights are set as described with SSO read/update.

Package was removed from server as well as policy.

Here is the log of what happens:

Processing local.jamf.MicrosoftWord...
...
{'Output': {}}
com.github.grahampugh.jamf-upload.processors/JamfPackageUploader
{'Input': {'API_PASSWORD': 'xxxx',
           'API_USERNAME': 'xxx',
           'JSS_URL': 'https://xxx.jamfcloud.com',
           'jcds_mode': 'True',
           'pkg_category': '#SW - Basics',
           'pkg_path': '/Users/autopkg/Library/AutoPkg/Cache/local.jamf.MicrosoftWord/MicrosoftWord-UNI-16.72.pkg',
           'version': '16.72'}}
JamfPackageUploader: No value supplied for pkg_name, setting default value of: 
JamfPackageUploader: No value supplied for pkg_info, setting default value of: 
JamfPackageUploader: No value supplied for pkg_notes, setting default value of: 
JamfPackageUploader: No value supplied for pkg_priority, setting default value of: 10
JamfPackageUploader: No value supplied for reboot_required, setting default value of: 
JamfPackageUploader: No value supplied for os_requirements, setting default value of: 
JamfPackageUploader: No value supplied for required_processor, setting default value of: None
JamfPackageUploader: No value supplied for send_notification, setting default value of: 
JamfPackageUploader: No value supplied for replace_pkg, setting default value of: False
JamfPackageUploader: No value supplied for replace_pkg_metadata, setting default value of: False
JamfPackageUploader: No value supplied for skip_metadata_upload, setting default value of: False
JamfPackageUploader: No value supplied for CLOUD_DP, setting default value of: False
JamfPackageUploader: No value supplied for SMB_URL, setting default value of: 
JamfPackageUploader: No value supplied for SMB_USERNAME, setting default value of: 
JamfPackageUploader: No value supplied for SMB_PASSWORD, setting default value of: 
JamfPackageUploader: No value supplied for sleep, setting default value of: 0
JamfPackageUploader: Checking for existing package 'MicrosoftWord-UNI-16.72.pkg' on https://ing.jamfcloud.com
JamfPackageUploader: Checking for existing authentication token
JamfPackageUploader: No existing valid token found
JamfPackageUploader: Getting an authentication token
JamfPackageUploader: No existing cookie found - starting new session
JamfPackageUploader: Session token received
JamfPackageUploader: Token: eyJhbGciOiJIUzI1NiJ9.eyJhdXRoZW50aWNhdGVkLWFwcCI6IkdFTkVSSUMiLCJhdXRoZW50aWNhdGlvbi10eXBlIjoiSlNTIiwiZ3JvdXBzIjpbXSwic3ViamVjdC10eXBlIjoiSlNTX1VTRVJfSUQiLCJ0b2tlbi11dWlkIjoiMWIwOGYwNDQtM2FhNS00ZTM0LWI2ZmMtMDY2N2RhN2RlYmFmIiwibGRhcC1zZXJ2ZXItaWQiOi0xLCJzdWIiOiIyMiIsImV4cCI6MTY4MjUyMjE2M30.8EzGO9on7oPA8islt9W25PULoIRxOcUSxwE29a9BOY4
JamfPackageUploader: Expires: 2023-04-26T15:16:03.439Z
JamfPackageUploader: Jamf Pro Version: 10.45.0-t1678116779
JamfPackageUploader: Token auth will be used
JamfPackageUploader: Package 'MicrosoftWord-UNI-16.72.pkg' not found on server
JamfPackageUploader: Number of File Share DPs: 0
JamfPackageUploader: Handling Cloud Distribution Point
JamfPackageUploader: Uploading package using experimental JCDS mode
JamfPackageUploader: Checking for existing '#SW - Basics' on https://xxx.jamfcloud.com
JamfPackageUploader: ID for category #SW - Basics: 8
JamfPackageUploader: Failover URL obtained: https://xxx.jamfcloud.com/?failover=XXXXXXXXXXXXX
JamfPackageUploader: HTTP response: 200
JamfPackageUploader: Headers: ['HTTP/2 302', 'date: Wed, 26 Apr 2023 14:46:04 GMT', 'content-type: text/html;charset=utf-8', 'content-length: 0', 'set-cookie: AWSALB=jWIOn8t3R2e6tr7V1isYwLwJ97ixoLEgzQROtWT75Oqca9b+iycOOVDUit6VMtofWmek1FEZAPy0ap1E29UBo4t0dqS+vSNeTA+SHXbT/EJlWlUuPBLgkpe5T2hv; Expires=Wed, 03 May 2023 14:46:03 GMT; Path=/', 'set-cookie: AWSALBCORS=jWIOn8t3R2e6tr7V1isYwLwJ97ixoLEgzQROtWT75Oqca9b+iycOOVDUit6VMtofWmek1FEZAPy0ap1E29UBo4t0dqS+vSNeTA+SHXbT/EJlWlUuPBLgkpe5T2hv; Expires=Wed, 03 May 2023 14:46:03 GMT; Path=/; SameSite=None; Secure', 'x-frame-options: SAMEORIGIN', 'cache-control: no-store, no-cache, must-revalidate, max-age=0, post-check=0, pre-check=0', 'strict-transport-security: max-age=31536000;includeSubDomains', 'x-xss-protection: 1; mode=block', 'cache-control: no-store, no-cache, must-revalidate, max-age=0, post-check=0, pre-check=0', 'set-cookie: JSESSIONID=ODllYWE3YzktNzkyMi00OWE1LWI0ZDgtZDZmNDBjYmJjY2Nl; Path=/; Secure; HttpOnly; SameSite=Lax', 'location: /?failover=Xw30Mw6iQACWhIk9', 'server: Jamf Cloud Node', '', 'HTTP/2 200', 'date: Wed, 26 Apr 2023 14:46:04 GMT', 'content-type: text/html;charset=utf-8', 'set-cookie: AWSALB=whBtDVupgABwlUuxuXdVOCchnbx7WYTyQqzO+Lm/42v6qzsQuic5A7MoLsfBdx7LC/c8rPkNZc4wnfSI9P5puehBA6MbfIxEGKlnv7kgGKUw2ASOjOh2pDIPKeGK; Expires=Wed, 03 May 2023 14:46:04 GMT; Path=/', 'set-cookie: AWSALBCORS=whBtDVupgABwlUuxuXdVOCchnbx7WYTyQqzO+Lm/42v6qzsQuic5A7MoLsfBdx7LC/c8rPkNZc4wnfSI9P5puehBA6MbfIxEGKlnv7kgGKUw2ASOjOh2pDIPKeGK; Expires=Wed, 03 May 2023 14:46:04 GMT; Path=/; SameSite=None; Secure', 'x-frame-options: SAMEORIGIN', 'cache-control: no-store, no-cache, must-revalidate, max-age=0, post-check=0, pre-check=0', 'strict-transport-security: max-age=31536000;includeSubDomains', 'x-xss-protection: 1; mode=block', 'cache-control: no-store, no-cache, must-revalidate, max-age=0, post-check=0, pre-check=0', 'server: Jamf Cloud Node', '']
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 72525    0 72525    0     0   367k      0 --:--:-- --:--:-- --:--:--  389k
JamfPackageUploader: HTTP response: 200
JamfPackageUploader: Session Token: sd9ZGQQuPZowzXH9Zb093E1QQCCGUsyD
JamfPackageUploader: X-Auth Token: d9379492de864fbfa412f2c8f2a73ec6tf25s7jr3m68ddc46xtdmoufcg1n7gdu
JamfPackageUploader: Pkg Upload URL: https://euc1-jcds.services.jamfcloud.com/api/file/v2/d67c42d9f2d04ded8370873742cc700f/
JamfPackageUploader: Pkg Upload URL: https://euc1-jcds.services.jamfcloud.com/api/file/v2/d67c42d9f2d04ded8370873742cc700f//MicrosoftWord-UNI-16.72.pkg
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 1154M    0   389  100 1154M      8  25.8M  0:00:44  0:00:44 --:--:-- 22.9M
JamfPackageUploader: HTTP response: 201
JamfPackageUploader: Pkg Object URL: https://xxx.jamfcloud.com/?failover=Xw30Mw6iQACWhIk9/legacy/packages.html?id=-1&o=c
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  4635  100  4287  100   348  20981   1703 --:--:-- --:--:-- --:--:-- 24523
JamfPackageUploader: HTTP response: 200
{'Output': {'jamfpackageuploader_summary_result': {'data': {'category': '#SW - '
                                                                        'Basics',
                                                            'name': 'MicrosoftWord',
                                                            'pkg_name': 'MicrosoftWord-UNI-16.72.pkg',
                                                            'pkg_path': '/Users/autopkg/Library/AutoPkg/Cache/local.jamf.MicrosoftWord/MicrosoftWord-UNI-16.72.pkg',
                                                            'version': '16.72'},
                                                   'report_fields': ['category',
                                                                     'name',
                                                                     'pkg_name',
                                                                     'pkg_path',
                                                                     'version'],
                                                   'summary_text': 'The '
                                                                   'following '
                                                                   'packages '
                                                                   'were '
                                                                   'uploaded '
                                                                   'to or '
                                                                   'updated in '
                                                                   'Jamf Pro:'},
            'pkg_name': 'MicrosoftWord-UNI-16.72.pkg',
            'pkg_path': '/Users/autopkg/Library/AutoPkg/Cache/local.jamf.MicrosoftWord/MicrosoftWord-UNI-16.72.pkg',
            'pkg_uploaded': True}}
com.github.grahampugh.jamf-upload.processors/JamfPolicyUploader
{'Input': {'API_PASSWORD': 'xxx',
           'API_USERNAME': 'xxxx',
           'JSS_URL': 'https://xxx.jamfcloud.com',
           'icon': 'MicrosoftWord.png',
           'policy_name': 'SelfService-GL-InstallMicrosoftWord-UNI',
           'policy_template': 'JamfSelfServicePolicyTemplateAdditonalPKG.xml',
           'replace_policy': 'True',
           'sleep': '0'}}
JamfPolicyUploader: No value supplied for replace_icon, setting default value of: False
JamfPolicyUploader: Parent Recipe: /Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.ing-bank.wps-osx-autopkg/Microsoft/MicrosoftWord.jamf.recipe
JamfPolicyUploader: File found at: /Users/autopkg/Library/AutoPkg/RecipeRepos/com.github.ing-bank.wps-osx-autopkg/JamfSelfServicePolicyTemplateAdditonalPKG.xml
JamfPolicyUploader: ("Replacing any instances of 'POLICY_NAME' with", "'SelfService-GL-InstallMicrosoftWord-UNI'")
JamfPolicyUploader: ("Replacing any instances of 'ALL_COMPUTERS' with", "'true'")
JamfPolicyUploader: ("Replacing any instances of 'GROUP_NAME' with", "'All Managed Clients'")
JamfPolicyUploader: ("Replacing any instances of 'pkg_name' with", "'MicrosoftWord-UNI-16.72.pkg'")
JamfPolicyUploader: ("Replacing any instances of 'SOFTWAREPKG2' with", "'OfficeTemplates-3.0.pkg'")
JamfPolicyUploader: ("Replacing any instances of 'SELF_SERVICE_DISPLAY_NAME' with", "'%FRIENDLYNAME% - %version%'")
JamfPolicyUploader: ("Replacing any instances of 'SELF_SERVICE_DESCRIPTION' with", "'Microsoft Word is a widely used commercial word processor designed by Microsoft and a component of the Microsoft Office suite of productivity software.'")
JamfPolicyUploader: ("Replacing any instances of 'SELF_SERVICE_FEATURE_MAINPAGE' with", "'false'")
JamfPolicyUploader: ("Replacing any instances of 'SELFSERVICE_CATEGORY' with", "'Office'")
JamfPolicyUploader: ("Replacing any instances of 'FRIENDLYNAME' with", "'Microsoft Word'")
JamfPolicyUploader: ("Replacing any instances of 'version' with", "'16.72'")
JamfPolicyUploader: Policy data:
JamfPolicyUploader: <policy>
     <!-- General - Name, Enabled/Disabled, Triggers, Frequency, Categories -->
     <general>
          <name>SelfService-GL-InstallMicrosoftWord-UNI</name>
          <enabled>true</enabled>
          <frequency>Ongoing</frequency>
     </general>
     <!-- Scope -->
     <scope>
          <all_computers>true</all_computers>
          <computer_groups>
               <computer_group>
                    <name>All Managed Clients</name>
               </computer_group>
          </computer_groups>
          <exclusions/>
     </scope>
     <!-- Packages -->
     <package_configuration>
          <packages>
               <size>2</size>
               <package>
                    <name>MicrosoftWord-UNI-16.72.pkg</name>
                    <action>Install</action>
               </package>
               <package>
                    <name>OfficeTemplates-3.0.pkg</name>
                    <action>Install</action>
               </package>
          </packages>
     </package_configuration>
     <!-- Scripts -->
     <scripts>
          <size>0</size>
     </scripts>
     <!-- Self Service details -->
     <self_service>
          <use_for_self_service>true</use_for_self_service>
          <self_service_display_name>Microsoft Word - 16.72</self_service_display_name>
          <install_button_text>Install</install_button_text>
          <reinstall_button_text>Reinstall</reinstall_button_text>
          <self_service_description>Microsoft Word is a widely used commercial word processor designed by Microsoft and a component of the Microsoft Office suite of productivity software.</self_service_description>
          <feature_on_main_page>false</feature_on_main_page>
          <self_service_categories>
               <category>
                    <name>Office</name>
                    <display_in>true</display_in>
                    <feature_in>false</feature_in>
               </category>
          </self_service_categories>
     </self_service>
     <!-- Maintenance e.g. recon -->
     <maintenance>
          <recon>true</recon>
     </maintenance>
     <!-- Files and processes e.g. run command -->
     <files_processes/>
     <!-- User interaction e.g. start and finish messages -->
     <user_interaction>
          <message_start/>
          <message_finish/>
     </user_interaction>
</policy>

JamfPolicyUploader: Checking for existing 'SelfService-GL-InstallMicrosoftWord-UNI' on https://xxx.jamfcloud.com
JamfPolicyUploader: Checking for existing authentication token
JamfPolicyUploader: No existing valid token found
JamfPolicyUploader: Getting an authentication token
JamfPolicyUploader: Session token received
JamfPolicyUploader: Token: eyJhbGciOiJIUzI1NiJ9.eyJhdXRoZW50aWNhdGVkLWFwcCI6IkdFTkVSSUMiLCJhdXRoZW50aWNhdGlvbi10eXBlIjoiSlNTIiwiZ3JvdXBzIjpbXSwic3ViamVjdC10eXBlIjoiSlNTX1VTRVJfSUQiLCJ0b2tlbi11dWlkIjoiNTM2Zjc4NzAtZGNjNC00ZWRiLWI0MzktYTZlY2E2N2E0MWI3IiwibGRhcC1zZXJ2ZXItaWQiOi0xLCJzdWIiOiIyMiIsImV4cCI6MTY4MjUyMjIwOX0.OAosUH4dLwwK3w7qMADTEMTbCNGBg_t6YmNgjkmEgag
JamfPolicyUploader: Expires: 2023-04-26T15:16:49.266Z
JamfPolicyUploader: Jamf Pro Version: 10.45.0-t1678116779
JamfPolicyUploader: Token auth will be used
JamfPolicyUploader: Uploading Policy...
JamfPolicyUploader: Policy upload attempt 1
JamfPolicyUploader: API Error: Problem with package
WARNING: Policy 'SelfService-GL-InstallMicrosoftWord-UNI' upload failed due to the following conflict: Problem with package
Failed.
Receipt written to /Users/autopkg/Library/AutoPkg/Cache/local.jamf.MicrosoftWord/receipts/local.jamf-receipt-20230426-164649.plist

The following recipes failed:
    local.jamf.MicrosoftWord
        Error in local.jamf.MicrosoftWord: Processor: com.github.grahampugh.jamf-upload.processors/JamfPolicyUploader: Error: WARNING: Policy 'SelfService-GL-InstallMicrosoftWord-UNI' upload failed due to the following conflict: Problem with package

The following new items were downloaded:
    Download Path                                                                              
    -------------                                                                              
    /Users/autopkg/Library/AutoPkg/Cache/local.jamf.MicrosoftWord/downloads/MicrosoftWord.pkg  

The following packages were copied:
    Pkg Path                                                                                   
    --------                                                                                   
    /Users/autopkg/Library/AutoPkg/Cache/local.jamf.MicrosoftWord/MicrosoftWord-UNI-16.72.pkg  

The following packages were uploaded to or updated in Jamf Pro:
    Category      Name           Pkg Name                     Pkg Path                                                                                   Version  
    --------      ----           --------                     --------                                                                                   -------  
    #SW - Basics  MicrosoftWord  MicrosoftWord-UNI-16.72.pkg  /Users/autopkg/Library/AutoPkg/Cache/local.jamf.MicrosoftWord/MicrosoftWord-UNI-16.72.pkg  16.72

I replaced Username, Password and URL and failover URL with XXX

grahampugh commented 1 year ago

Does OfficeTemplates-3.0.pkg exist on the server?

inglenny commented 1 year ago

Yes it is. However the new package (MicrosoftWord-UNI-16.72.pkg) cannot be found on the server. Even though it seems like it uploads just fine.

grahampugh commented 1 year ago

That explains the policy failing to upload, at least. Is this affecting all new packages or just this one?

inglenny commented 1 year ago

I just tested this with this package after configuring the failover URL in jamf. All other recipes using the jcds mode haven't been tested yet.

inglenny commented 1 year ago

Just tested it with another Package. Also not working. Any idea what we can do?

grahampugh commented 1 year ago

Is that package also not actually present on the server after the recipe run?

inglenny commented 1 year ago

Correct. The Autopkg logs indicate that it was uploaded correctly, but it cannot be found in Jamf Cloud, if I search for it in Settings -> Packages

inglenny commented 1 year ago

Any idea what we could do?

grahampugh commented 1 year ago

Does it work manually adding a package via the Jamf Admin GUI on the same machine that AutoPkg is running?

inglenny commented 1 year ago

Yes manually uploading the package via Jamf GUI Packages uploads the package just fine.

grahampugh commented 1 year ago

Not sure what to suggest. The output from curl is that the package was successfully uploaded. Does it never appear or is it just taking a long time? You could add in a Sleep processor if the latter.

I would also try to eliminate any doubt by removing the separate Office templates package from the policy and seeing if that works.

inglenny commented 1 year ago

It does never appear. Even after a week. Since randomising the failover URL JCDS mode is not working anymore. However, we used JCDS only for packages which are 1GB and larger, because the upload was not working properly with standard mode. Just for the fun I tried it again with the standard upload mode and now also packages over 1 GB get uploaded just fine.

So we deactivated JCDS mode completely and have no issues anymore. No idea why it suddenly works.

grahampugh commented 1 year ago

Labelling as "feature of experimental jcds_mode" :-) Glad you got something working.