grahampugh / jamf-upload

Scripts for uploading packages to Jamf Cloud
Apache License 2.0
150 stars 38 forks source link

Cookie issues against Jamf Pro 10.26.0 cloud using jamf_api_tool.py.... #19

Closed everetteallen closed 3 years ago

everetteallen commented 3 years ago

Not seeing an error but not connecting:

jamf_api_tool.py --prefs ~/Library/Preferences/com.github.autopkg.plist --packages --unused --verbose

** Jamf API Tool for Jamf Pro.

No existing cookie found - starting new session

curl command: /usr/bin/curl -X POST -D /tmp/jamf_upload/curl_headers_from_jamf_upload.txt --output /tmp/jamf_upload/curl_output_from_jamf_upload.txt https://nc.jamfcloud.com:443//uapi/auth/tokens --header authorization: Basic xxxxx= --header Content-type: application/json % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 379 0 379 0 0 945 0 --:--:-- --:--:-- --:--:-- 945 ERROR: No token received No existing cookie found - starting new session

curl command: /usr/bin/curl -X GET -D /tmp/jamf_upload/curl_headers_from_jamf_upload.txt --output /tmp/jamf_upload/curl_output_from_jamf_upload.txt https://nc.jamfcloud.com:443//uapi/v2/computer-prestages?page=0&page-size=1000&sort=id%3Adesc --header authorization: Bearer None --header Accept: application/json % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 42 0 42 0 0 229 0 --:--:-- --:--:-- --:--:-- 229 No existing cookie found - starting new session

curl command: /usr/bin/curl -X GET -D /tmp/jamf_upload/curl_headers_from_jamf_upload.txt --output /tmp/jamf_upload/curl_output_from_jamf_upload.txt https://nc.jamfcloud.com:443//JSSResource/patchsoftwaretitles --header authorization: Basic xxxxx= --header Accept: application/json % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 796 100 796 0 0 2680 0 --:--:-- --:--:-- --:--:-- 2671 No existing cookie found - starting new session

curl command: /usr/bin/curl -X GET -D /tmp/jamf_upload/curl_headers_from_jamf_upload.txt --output /tmp/jamf_upload/curl_output_from_jamf_upload.txt https://nc.jamfcloud.com:443//JSSResource/policies --header authorization: Basic xxxxxx= --header Accept: application/json % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 796 100 796 0 0 2981 0 --:--:-- --:--:-- --:--:-- 2981

grahampugh commented 3 years ago
ERROR: No token received

Can you check on https://your.jamfcloud.com:443/api that your creds can get an authorization token?

everetteallen commented 3 years ago

Works as expected with same name and password. Screen Shot 2021-01-21 at 9 46 12 AM

grahampugh commented 3 years ago

Hmm. And what about if instead of using --prefs, you use --url https://nc.jamfcloud.com --user autopkgapi (optionally --pass XXXX but it will prompt you if you don't provide it in the command).

Note also I dropped the :443 as that should be default anyway, and could maybe cause some issue (not sure why, but worth a try).

everetteallen commented 3 years ago

jamf_api_tool.py --url https://nc.jamfcloud.com --user autopkgapi --packages --unused --verbose

** Jamf API Tool for Jamf Pro.

Enter the password for 'autopkgapi' : No existing cookie found - starting new session

curl command: /usr/bin/curl -X POST -D /tmp/jamf_upload/curl_headers_from_jamf_upload.txt --output /tmp/jamf_upload/curl_output_from_jamf_upload.txt https://nc.jamfcloud.com/uapi/auth/tokens --header authorization: Basic xxx= --header Content-type: application/json % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 379 0 379 0 0 915 0 --:--:-- --:--:-- --:--:-- 915 ERROR: No token received No existing cookie found - starting new session

curl command: /usr/bin/curl -X GET -D /tmp/jamf_upload/curl_headers_from_jamf_upload.txt --output /tmp/jamf_upload/curl_output_from_jamf_upload.txt https://nc.jamfcloud.com/uapi/v2/computer-prestages?page=0&page-size=1000&sort=id%3Adesc --header authorization: Bearer None --header Accept: application/json % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 42 0 42 0 0 184 0 --:--:-- --:--:-- --:--:-- 184 No existing cookie found - starting new session

curl command: /usr/bin/curl -X GET -D /tmp/jamf_upload/curl_headers_from_jamf_upload.txt --output /tmp/jamf_upload/curl_output_from_jamf_upload.txt https://nc.jamfcloud.com/JSSResource/patchsoftwaretitles --header authorization: Basic xxxx= --header Accept: application/json % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 42520 100 42520 0 0 1315 0 0:00:32 0:00:32 --:--:-- 10648 No existing cookie found - starting new session

curl command: /usr/bin/curl -X GET -D /tmp/jamf_upload/curl_headers_from_jamf_upload.txt --output /tmp/jamf_upload/curl_output_from_jamf_upload.txt https://nc.jamfcloud.com/JSSResource/policies --header authorization: Basic xxxxx= --header Accept: application/json % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 313k 100 313k 0 0 256k 0 0:00:01 0:00:01 --:--:-- 256k

grahampugh commented 3 years ago

OK, weird. I'm not sure what's going on there. It works fine for me on my cloud beta, I'll try and check it on some other servers to see if I can reproduce your problem.

everetteallen commented 3 years ago

Thanks!

grahampugh commented 3 years ago

Actually since your downloads from the classic API are also failing, it must be a failure with the Basic Authentication. What if you run the curl command directly?

/usr/bin/curl -X GET https://nc.jamfcloud.com/uapi/auth/tokens --header "authorization: Basic xxxxx=" --header "Content-type: application/json"

If that fails, what if you do this?:

/usr/bin/curl -X GET https://nc.jamfcloud.com/uapi/auth/tokens --user autopkgapi:XXXYYYZZZ --header "Content-type: application/json"
everetteallen commented 3 years ago

% /usr/bin/curl -X GET https://nc.jamfcloud.com/uapi/auth/tokens --header authorization: Basic xxxxx= --header Content-type: application/json { "httpStatus" : 401, "errors" : [ ] }curl: (6) Could not resolve host: Basic curl: (6) Could not resolve host: xxxxxx= curl: (6) Could not resolve host: application ega@ega-yyyyyyyy ~ % /usr/bin/curl -X GET https://nc.jamfcloud.com/uapi/auth/tokens --user autopkgapi:uwish --header Content-type: application/json

{ "httpStatus" : 405, "errors" : [ ] }curl: (6) Could not resolve host: application

grahampugh commented 3 years ago

Sorry, see edit above (missing quotes)

everetteallen commented 3 years ago

So still 405's % /usr/bin/curl -X GET https://nc.jamfcloud.com/uapi/auth/tokens --header "authorization: Basic xxxxx=" --header "Content-type: application/json" { "httpStatus" : 405, "errors" : [ ] }%
% /usr/bin/curl -X GET https://nc.jamfcloud.com/uapi/auth/tokens --user autopkgapi:uwish --header "Content-type: application/json"

{ "httpStatus" : 405, "errors" : [ ] }%

grahampugh commented 3 years ago

Oh, wait! GET is wrong. Which means the script is wrong (so not sure why it works for me). Can you try:

/usr/bin/curl -X POST https://nc.jamfcloud.com/uapi/auth/tokens --header "authorization: Basic xxxxxx=" --header "Content-type: application/json"

If that fails, what if you do this?:

/usr/bin/curl -X POST https://nc.jamfcloud.com/uapi/auth/tokens --user autopkgapi:XXXYYYZZZ --header "Content-type: application/json"
everetteallen commented 3 years ago

That works in both cases and returns token:

% /usr/bin/curl -X POST https://nc.jamfcloud.com/uapi/auth/tokens --header "authorization: Basic xxxxxx=" --header "Content-type: application/json"

{ "token" : "eyJhbGciOiJIUzI1NiJ9.yyyyyyy.DD10SqOxnfUV3XGCZpak3edaziWfUEkh3gfHge-Y-MY", "expires" : 1611253228066 }%
% /usr/bin/curl -X POST https://nc.jamfcloud.com/uapi/auth/tokens --user autopkgapi:uwish --header "Content-type: application/json" { "token" : "eyJhbGciOiJIUzI1NiJ9.yyyyyyyy-XKSX1SpBQJUIuU", "expires" : 1611253269663 }%

grahampugh commented 3 years ago

Great! I'll amend the script later and let you know.

everetteallen commented 3 years ago

Thx!

grahampugh commented 3 years ago

Hmm. Actually was getting confused. In the script, POST is already used to get the token, so it is correct. Does this work?:

/usr/bin/curl -X POST -D /tmp/jamf_upload/curl_headers_from_jamf_upload.txt --output /tmp/jamf_upload/curl_output_from_jamf_upload.txt https://nc.jamfcloud.com/uapi/auth/tokens --header "authorization: Basic xxxxx=" --header "Content-type: application/json"

If not, then maybe it's something to do with writing out the headers to a file on our computer?

Does /tmp/jamf_upload/ exist? Does /tmp/jamf_upload/curl_output_from_jamf_upload.txt exist? Is there a "token" key in that file?

I probably should exit if the token is not received - at the moment it does not.

everetteallen commented 3 years ago

Short answer is yes:

% date Thu Jan 21 15:22:50 EST 2021 % /usr/bin/curl -X POST -D /tmp/jamf_upload/curl_headers_from_jamf_upload.txt --output /tmp/jamf_upload/curl_output_from_jamf_upload.txt https://nc.jamfcloud.com/uapi/auth/tokens --header "authorization: Basic xxxxx=" --header "Content-type: application/json" % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 379 0 379 0 0 1024 0 --:--:-- --:--:-- --:--:-- 1024 % ls -hal /tmp/jamf_upload/curl_output_from_jamf_upload.txt -rw-r--r-- 1 testuser wheel 379B Jan 21 15:22 /tmp/jamf_upload/curl_output_from_jamf_upload.txt % cat /tmp/jamf_upload/curl_output_from_jamf_upload.txt
{ "token" : "eyJhbGciOiJIUzI1NiJ9.yyyyyy.fEJ-UfGHb_fo8I1fSXE8rP8t_SNnZqKSltP8KTc-ABg", "expires" : 1611262377663 }%
% date Thu Jan 21 15:23:28 EST 2021

grahampugh commented 3 years ago

Well I'm beaten by this. As far as I can tell the script is doing exactly the same as the command. I'll keep looking though.

PS it occurs to me that you shouldn't be posting the basic auth string here as anyone could use it. I recommend changing your API password.

everetteallen commented 3 years ago

Thanks. The basic auth string has been altered and neither string is actually correct.

everetteallen commented 3 years ago

so today 01/22/2021 In a rather bazaar turn of events I am actually getting Information back and no error! However issuing: jamf_api_tool.py --url https://nc.jamfcloud.com --user autopkgapi --packages --unused --verbose

Returns a list of policies with JSON starting like {"policies":[{"id":19102,"name":"

hundreds of policies removed "name":"zzWCU Icon"}]} From /private/tmp/jamf_upload/curl_output_from_jamf_upload.txt So if I remove both /private/tmp/jamf_upload/curl_output_from_jamf_upload.txt and /private/tmp/jamf_upload/curl_headers_from_jamf_upload.txt I get /usr/bin/curl -X POST -D /tmp/jamf_upload/curl_headers_from_jamf_upload.txt --output /tmp/jamf_upload/curl_output_from_jamf_upload.txt https://nc.jamfcloud.com/uapi/auth/tokens --header authorization: Basic Yz2c= --header Content-type: application/json % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 379 0 379 0 0 440 0 --:--:-- --:--:-- --:--:-- 440 ERROR: No token received No existing cookie found - starting new session curl command: /usr/bin/curl -X GET -D /tmp/jamf_upload/curl_headers_from_jamf_upload.txt --output /tmp/jamf_upload/curl_output_from_jamf_upload.txt https://nc.jamfcloud.com/uapi/v2/computer-prestages?page=0&page-size=1000&sort=id%3Adesc --header authorization: Bearer None --header Accept: application/json % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 42 0 42 0 0 80 0 --:--:-- --:--:-- --:--:-- 80 No existing cookie found - starting new session curl command: /usr/bin/curl -X GET -D /tmp/jamf_upload/curl_headers_from_jamf_upload.txt --output /tmp/jamf_upload/curl_output_from_jamf_upload.txt https://nc.jamfcloud.com/JSSResource/patchsoftwaretitles --header authorization: Basic Ywa2c= --header Accept: application/json % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 42787 100 42787 0 0 1200 0 0:00:35 0:00:35 --:--:-- 10619 No existing cookie found - starting new session curl command: /usr/bin/curl -X GET -D /tmp/jamf_upload/curl_headers_from_jamf_upload.txt --output /tmp/jamf_upload/curl_output_from_jamf_upload.txt https://nc.jamfcloud.com/JSSResource/policies --header authorization: Basic YX2c= --header Accept: application/json % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 313k 100 313k 0 0 689k 0 --:--:-- --:--:-- --:--:-- 688k Resulting in the JSON list of policies. I spot check the id numbers and names to confirm they were polices not packages as well...
primalcurve commented 3 years ago

I think I may have been able to trace this issue as it has been plaguing me all day.

In this line, you check for HTTP/1.1 in order to set the status code. When I ran the script, I was getting back HTTP/2. Since this returns False, the status_code in the namedtuple remains unset.

I have some recent experience with the quirkiness of namedtuples. If you need a one-time read-only object with named attributes they're nice, but they can get weird when things go wrong.

If you never close a namedtuple, unset fields are not NoneType, they're the _tuplegetter object. The best way to avoid this is to set default values, and close the namedtuple in the return statement. I would recommend expanding this line to include the defaults (simple example, they could be other Python primitives):

r = namedtuple("r", ["headers", "status_code", "output"], defaults=(None, None, None))

And change this line to

 return r()

You should do this so that the object is read-only (it is a tuple after all) and any unset values are now the default of None. Relying on an unclosed namedtuple for any sort of logic will create all kinds of weirdness like this.

If you like I can do a PR with these changes after some testing.

primalcurve commented 3 years ago

I should also mention that I was using api_get.get_policies_in_category in my testing, so this was a classic API call. It was returning 200 and the correct data was in curl_output_from_jamf_upload.txt. The logic was just failing when it go to checking the status code from curl.