cURL adapter attempts to use a File object in a dict comprehension

[mosen]

subprocess seems to bail when it encounters a list that contains a File object, example print debug:

['curl', '-u', 'admin:password', '--silent', '--write-out', '|%{response_code}', 
'--insecure', '--tlsv1', '--header', 'Accept: application/xml', '--header', 'Content-Type: text/xml', 
'--header', 'FILE_TYPE', '--header', 'FILE_NAME', '--header', 'DESTINATION',
 '--header', 'OBJECT_ID', '--data', <open file u'/Users/admin/Library/AutoPkg/Cache/com.github.jss-recipes.jss.iTerm2/iTerm2-3.1.5.pkg', mode 'rb' at 0x10d0d1f60>, 
'--request', 'POST', u'https://localhost:8444/dbfileupload']

This may or may not be the issue affecting CDP.

[mosen]

Added another commit which resolves request headers not being properly supplied to curl eg. OBJECT_ID instead of OBJECT_ID: 1 due to a statement being array += dict.

[sheagcraig]

OMG. I've been in holiday ignoring-Github land. Seems as if you've been busy. Can you give me a summary of what all is going on here, and where you are? Thanks!

[mosen]

Sorry this went a bit insane, and I didn't end up making feature branches for every added feature. The summary:

• CurlAdapter was not handling uploads to CDP correctly because it attempted to reference a file object as a string value, this was fixed.
• Seeing the issue with CurlAdapter, i picked up gurl.py from the munki project and created GurlAdapter to use this as the main transport.
• gurl.py fell short in several areas: It was never written to handle uploads, and it was never written to download contents into a buffer. It also specifically keeps information about resumed downloads in the xattrs of the incomplete file, so this wasn't a match for me, so:
• I started NSURLSessionAdapter which uses all the same method signatures as a custom transport adapter for the popular "requests" package. meaning that we could use requests and just change adapter based on the host operating system. Linux would use regular urllib3 transport and mac will detect and use NSURLSessionAdapter.
• I thought the nose unit tests might be worth ditching as I created 30-40 unit tests under pytest specifically for NSURLSessionAdapter and the JSS object. I kept the discarded nose tests under _nose. I also added distribution point tests which spin up a samba docker container and test against that.
• I started on UAPI by making a parallel implementation of UAPI objects with the same API as JSSObjects. All of the search methods are added dynamically to the JSS object and each UAPIObject contains attributes which determine its behaviour in the same way that JSSObjects/Container objects behave.
• There is an outstanding bug on UAPI because it requires basic auth to get a token but NSURLSession requires an authentication header before it will attempt to auth using the stored credentials. a workaround might be to construct our own header.
• I added a scrape() method to jss which will be a dumb get() method but which should always create or cache the session ID of the current login before scraping a page.

Thats about it!

[mosen]

Worth mentioning that the NSURLSessionAdapter would break uploads in its current state so hold off :)

[sheagcraig]

I'm down for everything that's going on here. It's such a huge change that I'm wondering how to merge it though.

Also, about the tests: They are definitely tied to a JSS I used to have, so they won't even run now. My understanding is that they should still run under pytest (and that's what I would use if I get around to updating them). They just need to be updated to mock out the responses that I have hardcoded in (and had an actual server to respond to).

[mosen]

This is 342 commits ahead of your fork haha. No chance of merging anything. I need to examine how I can divide the spaghetti using surgical precision git cherry-pick or maybe a file by file basis

[mosen]

ok im assessing how to split this.

probably:

• test improvements
• URLSession adapter
• UAPI support

Rather than cherry pick the commits i feel like starting new ones on a new branch in case there are commits that run across those functions.

[mosen]

Split into multiple PRs