Error Adding Repo

[jmb03012]

Just installed AutoPkgr for the first time to begin testing in our development environment. Install went fine, installed Git, installed AutoPkg, all without issue.

Went to add a repo for the first time though and I consistently get errors that there was a a SSL certificate problem: Invalid Certificate Chain.

I can confirm that when accessing git via https that the github.com certificate is showing as valid via our internal sub ca (we go through a proxy) so not quite sure what is wrong. I've tried manually adding the github.com certificate to my macs keychain on its own and setting it to always trust but the issue still persists. All my web searches and their solutions have not worked so far as we aren't having the digicert issue that cause issues for some 10.9 users, and disabling ssl for git is just not an option.

Any advice would be appreciated. Thanks in advance. - Jordan

Also a screenshot of the error has been attached.

[jmb03012]

Where is python-jss getting its info from with regard to AutoPkgr? If we are getting 403 errors when it runs, but when we access the API directly using the same values we provide AutoPkgr we get not issues, it makes me think that python isn't getting the right values, or its getting the right ones but interpreting and.or sending them incorrectly.

Obviously this setup works for other people (hell I even watched it at JNUC) so the bigger question is do I have something misconfigured, or is there something unique about our environment that hasn't been stumbled across yet by someone else yet.

Should the add-on thats integrated being pulling creds, url, etc or does a plist specifying those values need to be created as you would do when using AutoPKG with python-jss installed but without AutoPkgr (referenced around line 99 in jss.py)

Just pulling at straws.

[eahrold]

try that curl command and make sure you also do it for both http and https, it's possibly an issue with the exceptions list

export HTTP_PROXY=proxy.sample.company.com:81
export HTTPS_PROXY=proxy.sample.company.com:81

To answer your question it autopkg -> jss-autopkg-addon -> python-jss get the API_USERNAME, API_PASSWORD, JSS_URL and JSS_REPOS from here ~/Librariy/Preferences/com.github.autopkg.plist

defaults read com.github.autopkg API_USERNAME
defaults read com.github.autopkg API_PASSWORD
defaults read com.github.autopkg JSS_URL

# The password for the DP and the Share Name for the DP is set here
# python-jss actually pulls the username from the JSS during the run.
defaults read com.github.autopkg JSS_REPOS

All AutoPkgr does is set those values once it has verified that the JSS URL and Credentials are valid. The fact that the python-jss isn't even hitting the server makes me still suspect the proxy settings. AutoPkgr is written in Objective-c and uses NSURLRequest to GET the info from the API which automatically obeys all system proxy settings . But with python-iss all you can do is export those values to the shell env.

The other thing that may be occurring is you don't need a proxy to connect to the local JSS and would need to include these in the env export when running autopkg

export no_proxy=localhost,127.0.0.0/8,*.local,*youdomain.com

I know soooo many moving parts...

[jmb03012]

In the case of our lab environment, we do not need our proxy to hit the JSS because its internal. I just tested connecting to the JSS via a web browser with our wpad disabled on an OS X client (I've since re-enabled it though because the setting doesn't carry over to AutoPkgr's plist anyway).

With that said, if I'm understanding the above correct should I run a defaults write command to the AutoPkgr plist for a noproxy=localhost,127.0.0.0/8,.local,_yourdomain.com?

Just want to make sure because there was a lot of data on a few different topics in that last post =)

CORRECTION:

If python-jss is gettings its values from com.github.autopkg.plist and python-jss needs a proxy (or to be told no proxy), can't a proxy value be specified in the autopkg plist for python-jss to use the same one supplying one in com.lindegroup.AutoPkgr.plist provides one for AutoPKG to use? If AutoPkgr is just providing the GUI front end for AutoPKG and all its values entered end up getting passed to the AutoPKG plist, how come the proxy values don't get passed and instead have to get written to AutoPkgr's plist, not AutoPkgs?

[eahrold]

No, currently AutoPkgr is not doing anything at all for an exceptions list, it's just setting the proxy, which may be causing this condition. (I'm currently working on a framework to remedy this as all as using the system settings to populate the proxy values)

Next what I'd like for you to try is running autopkg directly

export HTTP_PROXY=http://your.proxy.server:81
export HTTPS_PROXY=http://your.proxy.server:81

/usr/bin/python /usr/local/bin/autopkg run -v Firefox.jss

If this fails, then try with the NO_PROXY key set in addition (don't close the shell)

export NO_PROXY=localhost,127.0.0.0/8,*.local,*.yourdomain.com
/usr/bin/python /usr/local/bin/autopkg run -v Firefox.jss

[jmb03012]

Both configurations fail. I did close the shell just to separate the test instances but on the second try, I did re-enter the values for the HTTP and HTTPS proxy in addition to No Proxy. I believe I typed everything in the way you asked but please verify if wouldn't mined. Please See below:

AutoPkgTestServerName:~ computerusername$ export HTTP_PROXY=http://proxy.sample.company.com:81 AutoPkgTestServerName:~ computerusername$ export HTTPS_PROXY=https://proxy.sample.company.com:81 AutoPkgTestServerName:~ computerusername$ /usr/bin/python /usr/local/bin/autopkg run -v Firefox.jss Processing Firefox.jss... MozillaURLProvider MozillaURLProvider: Found URL http://ftp.mozilla.org/pub/mozilla.org//firefox/releases/latest/mac/en-US/Firefox%2033.1.dmg URLDownloader URLDownloader: Item at URL is unchanged. URLDownloader: Using existing /Users/computerusername/Library/AutoPkg/Cache/com.github.autopkg.jss.Firefox_EN/downloads/Firefox.dmg EndOfCheckPhase AppDmgVersioner AppDmgVersioner: Mounted disk image /Users/computerusername/Library/AutoPkg/Cache/com.github.autopkg.jss.Firefox_EN/downloads/Firefox.dmg AppDmgVersioner: BundleID: org.mozilla.firefox AppDmgVersioner: Version: 33.1 PkgRootCreator PkgRootCreator: Created /Users/computerusername/Library/AutoPkg/Cache/com.github.autopkg.jss.Firefox_EN/Firefox PkgRootCreator: Created /Users/computerusername/Library/AutoPkg/Cache/com.github.autopkg.jss.Firefox_EN/Firefox/Applications Copier Copier: Mounted disk image /Users/computerusername/Library/AutoPkg/Cache/com.github.autopkg.jss.Firefox_EN/downloads/Firefox.dmg Copier: Copied /private/tmp/dmg.jGFhYO/Firefox.app to /Users/computerusername/Library/AutoPkg/Cache/com.github.autopkg.jss.Firefox_EN/Firefox/Applications/Firefox.app PkgCreator PkgCreator: Package already exists at path /Users/computerusername/Library/AutoPkg/Cache/com.github.autopkg.jss.Firefox_EN/Firefox-33.1.pkg. PkgCreator: Existing package matches version and identifier, not building. JSSImporter Traceback (most recent call last): File "/usr/local/bin/autopkg", line 1334, in sys.exit(main(sys.argv)) File "/usr/local/bin/autopkg", line 1328, in main exit(subcommands[verb]'function') File "/usr/local/bin/autopkg", line 1152, in run_recipes autopackager.process(recipe) File "/Library/AutoPkg/autopkglib/init.py", line 466, in process self.env = processor.process() File "/Library/AutoPkg/autopkglib/init.py", line 295, in process self.main() File "/Library/AutoPkg/autopkglib/JSSImporter.py", line 571, in main ssl_verify=sslVerify, repo_prefs=repos) File "/Library/Python/2.7/site-packages/jss/jss.py", line 166, in init self.distribution_points = distribution_points.DistributionPoints(self) File "/Library/Python/2.7/site-packages/jss/distribution_points.py", line 58, in init self.response = j.DistributionPoint().retrieve_all() File "/Library/Python/2.7/site-packages/jss/jss.py", line 314, in DistributionPoint return self.factory.get_object(DistributionPoint, data) File "/Library/Python/2.7/site-packages/jss/jss.py", line 451, in get_object result = self.jss.get(url) File "/Library/Python/2.7/site-packages/jss/jss.py", line 199, in get self._error_handler(JSSGetError, response) File "/Library/Python/2.7/site-packages/jss/jss.py", line 186, in _error_handler raise exception jss.jss.JSSGetError: JSS Error. Response Code: 403 Response"

AutoPkgTestServerName:~ computerusername$ export HTTP_PROXY=http://proxy.sample.company.com:81 AutoPkgTestServerName:~ computerusername$ export HTTPS_PROXY=https://proxy.sample.company.com:81 AutoPkgTestServerName:~ computerusername$ export NOPROXY=localhost,127.0.0.0/8,.local,_.company.com AutoPkgTestServerName:~ computerusername$ /usr/bin/python /usr/local/bin/autopkg run -v Firefox.jss Processing Firefox.jss... MozillaURLProvider MozillaURLProvider: Found URL http://ftp.mozilla.org/pub/mozilla.org//firefox/releases/latest/mac/en-US/Firefox%2033.1.dmg URLDownloader URLDownloader: Item at URL is unchanged. URLDownloader: Using existing /Users/computerusername/Library/AutoPkg/Cache/com.github.autopkg.jss.Firefox_EN/downloads/Firefox.dmg EndOfCheckPhase AppDmgVersioner AppDmgVersioner: Mounted disk image /Users/computerusername/Library/AutoPkg/Cache/com.github.autopkg.jss.Firefox_EN/downloads/Firefox.dmg AppDmgVersioner: BundleID: org.mozilla.firefox AppDmgVersioner: Version: 33.1 PkgRootCreator PkgRootCreator: Created /Users/computerusername/Library/AutoPkg/Cache/com.github.autopkg.jss.Firefox_EN/Firefox PkgRootCreator: Created /Users/computerusername/Library/AutoPkg/Cache/com.github.autopkg.jss.Firefox_EN/Firefox/Applications Copier Copier: Mounted disk image /Users/computerusername/Library/AutoPkg/Cache/com.github.autopkg.jss.Firefox_EN/downloads/Firefox.dmg Copier: Copied /private/tmp/dmg.VJJ7zr/Firefox.app to /Users/computerusername/Library/AutoPkg/Cache/com.github.autopkg.jss.Firefox_EN/Firefox/Applications/Firefox.app PkgCreator PkgCreator: Package already exists at path /Users/computerusername/Library/AutoPkg/Cache/com.github.autopkg.jss.Firefox_EN/Firefox-33.1.pkg. PkgCreator: Existing package matches version and identifier, not building. JSSImporter Traceback (most recent call last): File "/usr/local/bin/autopkg", line 1334, in sys.exit(main(sys.argv)) File "/usr/local/bin/autopkg", line 1328, in main exit(subcommands[verb]'function') File "/usr/local/bin/autopkg", line 1152, in run_recipes autopackager.process(recipe) File "/Library/AutoPkg/autopkglib/init.py", line 466, in process self.env = processor.process() File "/Library/AutoPkg/autopkglib/init.py", line 295, in process self.main() File "/Library/AutoPkg/autopkglib/JSSImporter.py", line 571, in main ssl_verify=sslVerify, repo_prefs=repos) File "/Library/Python/2.7/site-packages/jss/jss.py", line 166, in init self.distribution_points = distribution_points.DistributionPoints(self) File "/Library/Python/2.7/site-packages/jss/distribution_points.py", line 58, in init self.response = j.DistributionPoint().retrieve_all() File "/Library/Python/2.7/site-packages/jss/jss.py", line 314, in DistributionPoint return self.factory.get_object(DistributionPoint, data) File "/Library/Python/2.7/site-packages/jss/jss.py", line 451, in get_object result = self.jss.get(url) File "/Library/Python/2.7/site-packages/jss/jss.py", line 199, in get self._error_handler(JSSGetError, response) File "/Library/Python/2.7/site-packages/jss/jss.py", line 186, in _error_handler raise exception jss.jss.JSSGetError: JSS Error. Response Code: 403 Response"

[eahrold]

I think you need a wild card in front of the .local and .company.com, eg ".local,.company.com"

[eahrold]

Actually github may be stripping that out of the comment.

[homebysix]

The backtick (`) works to preserve asterisks:

*.local,*.company.com

[jmb03012]

it was stripped out. I had an asterisk for a wildcard before .local and my company domain.

----- Original Message ----- From: reply@reply.github.com To: autopkgr@noreply.github.com Cc: JORDAN BENDER (BLOOMBERG/ 731 LEX -) At: Nov 12 2014 17:31:11

Actually github may be stripping that out of the comment. — Reply to this email directly or view it on GitHub.

[eahrold]

Oddly enough, I just found some more information about that key. Looks like I was wrong about using wild cards in that way...

no_proxy This variable should contain a comma-separated list of domain extensions proxy should not be used for. For instance, if the value of "no_proxy" is ".mit.edu", proxy will not be used to retrieve documents from MIT.

So it looks like github stripping it out was a happy accident.

When you get a chance try setting up the export w/o them.

export NO_PROXY=localhost,127.0.0.0/8,.local,.company.com

[eahrold]

@jmb03012 did you have a chance to test it out w/o wild cards?

If that works, you should give this build a try. I will use the system proxies.
https://github.com/eahrold/autopkgr/releases/tag/v1.1.1-beta
All you should need to do is

defaults write com.lindegroup.AutoPkgr useSystemProxies -bool true

[jmb03012]

Just got a chance to try without the wildcard, basically just typed the same thing I had before but without the asterisks, still failed with a 403 error though.

[eahrold]

That's too bad. That was my really my last hope solving it on our end. Like I said earlier, start marking up you jss.py with print() statements and see if you can squeeze more information out of it regarding where it's failing.

Here's what I would put for Around L195-L198 (right after response = self.session.get(url) )

        print "URL REQUESTED: %s\n" % url
        print "REQUEST HEADERS: %s\n" % self.session.headers
        print "RESPONSE HEADERS: %s\n" % response.headers
        print "RESPONSE FROM JSS %s\n" % response.text.encode('utf-8')

Also if you have access, take a look at your proxy server logs and see if it is getting requests for you JSS url that it should not be getting. Especially since you said your jss log isn't showing any requests during autopkg run.

And for S&G's you may want to try that beta build that uses the proxy settings from system preferences. Just make sure to include your JSS server url in the "Bypass proxy settings..." area

[jmb03012]

Where is the proxy bypass settings area you are referring to? is that in the autopkgr plist?

UPDATE: NM, after reading that again obviously its the OS since you are using system preferences......today has not been a good day.

[jmb03012]

OK so I spun up a new VM, installed the 1.1.1 beta, added the key for useSystemProxies true, confirmed it was there, tried adding recipes, they add so autopkgr is grabbing the os proxies correctly.

Added the server IP for our dev JSS to the bypass proxy settings area, attempted to run the same Firefox jss recipe, not only does it fail, but now instead of a 403 error, we get a SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

Heres the traceback: 11/13/14 5:34:00.128 PM AutoPkgr[439]: (FULL AUTOPKG TRACEBACK) File "/usr/local/bin/autopkg", line 1334, in sys.exit(main(sys.argv)) File "/usr/local/bin/autopkg", line 1328, in main exit(subcommands[verb]'function') File "/usr/local/bin/autopkg", line 1152, in run_recipes autopackager.process(recipe) File "/Library/AutoPkg/autopkglib/init.py", line 466, in process self.env = processor.process() File "/Library/AutoPkg/autopkglib/init.py", line 295, in process self.main() File "/Library/AutoPkg/autopkglib/JSSImporter.py", line 571, in main ssl_verify=sslVerify, repo_prefs=repos) File "/Library/Python/2.7/site-packages/jss/jss.py", line 166, in init self.distribution_points = distribution_points.DistributionPoints(self) File "/Library/Python/2.7/site-packages/jss/distribution_points.py", line 58, in init self.response = j.DistributionPoint().retrieve_all() File "/Library/Python/2.7/site-packages/jss/jss.py", line 314, in DistributionPoint return self.factory.get_object(DistributionPoint, data) File "/Library/Python/2.7/site-packages/jss/jss.py", line 451, in get_object result = self.jss.get(url) File "/Library/Python/2.7/site-packages/jss/jss.py", line 193, in get response = self.session.get(url) File "/Library/Python/2.7/site-packages/jss/contrib/requests/sessions.py", line 460, in get return self.request('GET', url, _kwargs) File "/Library/Python/2.7/site-packages/jss/contrib/requests/sessions.py", line 448, in request resp = self.send(prep, _send_kwargs) File "/Library/Python/2.7/site-packages/jss/contrib/requests/sessions.py", line 554, in send r = adapter.send(request, **kwargs) File "/Library/Python/2.7/site-packages/jss/contrib/requests/adapters.py", line 417, in send raise SSLError(e, request=request) jss.contrib.requests.exceptions.SSLError: [Errno 1] _ssl.c:504: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

[eahrold]

A new error is good. we may be getting somewhere. try this and run again

 defaults write com.github.autopkg JSS_VERIFY_SSL -bool False

[jmb03012]

I was just about to update my most recent post that I tried that shortly after the failure but it still results in the same error.

Update: Interestingly though, I ran it again and actually got an error but a different one then the SSL but its the one I expected to get from Python, that an unverified HTTPS request is being made and that I should add certificate verification if I know whats good for me.

HOWEVER....at the very end of the error is a reference to a local variable, 'dp' referenced before assignment as causing an UnboundLocalError.

Notice in the traceback below there is no 403 error, and no certificate verification failure.

11/13/14 5:45:26.554 PM AutoPkgr[804]: (FULL AUTOPKG TRACEBACK) File "/usr/local/bin/autopkg", line 1334, in sys.exit(main(sys.argv)) File "/usr/local/bin/autopkg", line 1328, in main exit(subcommands[verb]'function') File "/usr/local/bin/autopkg", line 1152, in run_recipes autopackager.process(recipe) File "/Library/AutoPkg/autopkglib/init.py", line 466, in process self.env = processor.process() File "/Library/AutoPkg/autopkglib/init.py", line 295, in process self.main() File "/Library/AutoPkg/autopkglib/JSSImporter.py", line 571, in main ssl_verify=sslVerify, repo_prefs=repos) File "/Library/Python/2.7/site-packages/jss/jss.py", line 166, in init self.distribution_points = distribution_points.DistributionPoints(self) File "/Library/Python/2.7/site-packages/jss/distribution_points.py", line 79, in init self._children.append(dp) UnboundLocalError: local variable 'dp' referenced before assignment

[eahrold]

Is the request actually hitting the JSS server now?

[jmb03012]

If I'm interpreting that traceback correctly I think it is, but this is really my first experience with Python so I could be wrong.

It looks like additional python scripts are now being called up, not just JSSImporter.py and jss.py which were the only two we were seeing before. Now distribution_points.py is also getting called up and that appears to be the one that is getting stuck.

That part of the script (line 79) looks like its where the distribution point dict gets populated from the JSS server's distribution points.

[eahrold]

You are correct. The good news is that we've got the proxy issue worked out, and thanks for staying with it for so long, it should really help a number of people.

The issue with the dp getting referenced before assignment is strange too. What type of DP are you running? AFP, SMB, or something else.

[jmb03012]

For the purposes of this dev environment, the windows 2008 r2 vm that is hosting the JSS is also hosting an SMB DP just from a shared folder at the root of C. Was the simplest way to go for this type of testing.

I don't know if its against the 'unspoken rules' on here but would it be possible for us to actually have a phone call?

[eahrold]

As long as it's one of the two it shouldn't matter the logic is such

if connection_type == 'AFP':
    dp = AFPDistributionPoint(URL=URL, port=port, share_name=share_name, mount_point=mount_point, username=username, password=password)
elif connection_type == 'SMB':
    dp = SMBDistributionPoint(URL=URL, port=port, share_name=share_name, mount_point=mount_point, domain=domain, username=username, password=password)

So if it was something other, the unbound reference would be expected. For some reason the values it's getting from the API for your share's name are not getting parsed into one of those two.

cURL from the api and see what the xml returns for the <connection_type> key

https://yourserver.com/JSSResource/distributionpoints/name/dpname

[jmb03012]

Will give it a go tomorrow. Had to call it quits for today to get back to jersey. The joys of mass transit await.

Sent from Bloomberg Professional for iPhone

----- Original Message ----- From: Eldon Ahrold reply@reply.github.com To: autopkgr@noreply.github.com CC: JORDAN BENDER At: Nov 13, 2014 18:47:06

As long as it's one of the two it shouldn't matter the logic is such if connection_type == 'AFP': dp = AFPDistributionPoint(URL=URL, port=port, share_name=share_name, mount_point=mount_point, username=username, password=password) elif connection_type == 'SMB': dp = SMBDistributionPoint(URL=URL, port=port, share_name=share_name, mount_point=mount_point, domain=domain, username=username, password=password) So if it was something other, the unbound reference would be expected. For some reason the values it's getting from the API for your share's name are not getting parsed into one of those two. cURL from the api and see what the xml returns for the keyhttps://yourserver.com/JSSResource/distributionpoints/name/dpname — Reply to this email directly or view it on GitHub.

[eahrold]

@jmb03012 we just released version 1.1.1 which addresses at least the proxy component of your issue. Keep us posted as to whether you make any progress solving your issue.

[eahrold]

@jmb03012 I'm going to close this issue since we're in to new territory, feel free to open a new issue to address the current conditions.

--Eldon