search

lindegroup / autopkgr

AutoPkgr is a free Mac app that makes it easy to install and configure AutoPkg.
http://www.lindegroup.com/autopkgr
Apache License 2.0
535 stars 51 forks source link

Error Adding Repo #187

Closed jmb03012 closed 9 years ago

jmb03012 commented 9 years ago

Just installed AutoPkgr for the first time to begin testing in our development environment. Install went fine, installed Git, installed AutoPkg, all without issue.

Went to add a repo for the first time though and I consistently get errors that there was a a SSL certificate problem: Invalid Certificate Chain.

I can confirm that when accessing git via https that the github.com certificate is showing as valid via our internal sub ca (we go through a proxy) so not quite sure what is wrong. I've tried manually adding the github.com certificate to my macs keychain on its own and setting it to always trust but the issue still persists. All my web searches and their solutions have not worked so far as we aren't having the digicert issue that cause issues for some 10.9 users, and disabling ssl for git is just not an option.

Any advice would be appreciated. Thanks in advance. - Jordan

Also a screenshot of the error has been attached. screen shot 2014-11-07 at 9 48 35 am

eahrold commented 9 years ago

@jmb03012 do you have AutoPkgr set to use the proxy?

jmb03012 commented 9 years ago

@eahrold I was not aware that you could. Right now I have the Mac using our PAC through the OS settings. Do I need to apply that same setting within AutoPkgr and if so where do I make that change?

eahrold commented 9 years ago

You may, it's worth a shot. There's info on what to do here. https://github.com/lindegroup/autopkgr/releases/tag/v1.1

Let us know if it helps.

@hombysix can you update the README.md with these instructions too?

homebysix commented 9 years ago

@eahrold Does this look accurate? If so, you can merge into 1.1.1-update.

jmb03012 commented 9 years ago

i added our internet proxy from the value in the dat file that we use using the commands in the post you pointed me at but the issue persists.

----- Original Message ----- From: reply@reply.github.com To: autopkgr@noreply.github.com Cc: JORDAN BENDER (BLOOMBERG/ 731 LEX -) At: Nov 7 2014 12:22:07

You may, it's worth a shot. There's info on what to do here. https://github.com/lindegroup/autopkgr/releases/tag/v1.1

Let us know if it helps.

@hombysix can you update the README.md with these instructions too?

Reply to this email directly or view it on GitHub: https://github.com/lindegroup/autopkgr/issues/187#issuecomment-62178712

eahrold commented 9 years ago

can you get autopkg itself running from the command line?

jmb03012 commented 9 years ago

I can try but can you be a little more specific as to what you like me to do.

I apologize but I've never actually used AutoPKG prior to now. Had planned to but kept getting pushed off and then AutoPKGR came out so figured I'd take the easier route =).

----- Original Message ----- From: reply@reply.github.com To: autopkgr@noreply.github.com Cc: JORDAN BENDER (BLOOMBERG/ 731 LEX -) At: Nov 7 2014 13:58:20

can you get autopkg itself running from the command line?

Reply to this email directly or view it on GitHub: https://github.com/lindegroup/autopkgr/issues/187#issuecomment-62194173

eahrold commented 9 years ago

sure the command would be 

/usr/bin/python /usr/local/bin/autopkg repo-add recipes
eahrold commented 9 years ago

@jmb03012 take a look at this too

http://superuser.com/questions/721778/can-not-clone-any-git-repository-from-github

jmb03012 commented 9 years ago

Tried just now, same error. SSL certificate problem: Invalid Certificate Chain

----- Original Message ----- From: reply@reply.github.com To: autopkgr@noreply.github.com Cc: JORDAN BENDER (BLOOMBERG/ 731 LEX -) At: Nov 7 2014 14:33:58

sure the command would be 

/usr/bin/python /usr/local/bin/autopkg repo-add recipes

Reply to this email directly or view it on GitHub: https://github.com/lindegroup/autopkgr/issues/187#issuecomment-62199468

eahrold commented 9 years ago

did http://superuser.com/questions/721778/can-not-clone-any-git-repository-from-github get you anywhere?

jmb03012 commented 9 years ago

Yeah I had seen this earlier in my searches as well.

I looked to see if we were ahving the DigiCert High Assurance EV Root CA issue but we are not. We do have their root CA but its not the expired one and none are present in the login keychain, just the valid one in System.

When I saw this earlier too I figured maybe I have a valid certificate, but my browser isnt considering it trusted and then the SSL certificate is invalid but when I navigate to github.com via Safari and click on HTTPS, I can see the github certificate listed as under our SubCA as valid and our SubCA as Always Trust.

----- Original Message ----- From: reply@reply.github.com To: autopkgr@noreply.github.com Cc: JORDAN BENDER (BLOOMBERG/ 731 LEX -) At: Nov 7 2014 14:36:31

@jmb03012 take a look at this too http://superuser.com/questions/721778/can-not-clone-any-git-repository-from-github — Reply to this email directly or view it on GitHub.

jmb03012 commented 9 years ago

There is a link from that page to another post on stackoverflow but I cant exactly follow it as clicking on HTTPS on github.com does yield a box where I can press show certificate but there are no boxes to check always trust (not that it matters though because it shows its trusted already). Alternatively temporarily disabling my proxy wont help even to test because I already cant hit the url except from the browser but thats only because the proxy is there, without it there wont be any internet from anywhere.

I also tried bringing up the window after clicking HTTP, dragging the github.com certificate to my desktop, and then importing it into my login keychain and marking always trust but the issue still persists.

I know I can hit the sight through our proxy because I can get to it via a web browser, question is just why via autopkg(r) its not working....

----- Original Message ----- From: reply@reply.github.com To: autopkgr@noreply.github.com Cc: JORDAN BENDER (BLOOMBERG/ 731 LEX -) At: Nov 7 2014 15:09:19

did http://superuser.com/questions/721778/can-not-clone-any-git-repository-from-github get you anywhere? — Reply to this email directly or view it on GitHub.

eahrold commented 9 years ago

Most likely nothing in your shell will work ( not just autopkg(r) ) To test just do 

curl https://github.com

and I'm guessing you'll get the same message.

eahrold commented 9 years ago

If you can get this working in the shell and provide us with some insight on how you did it we may be able to roll it in to AutoPkgr

jmb03012 commented 9 years ago

Correct. Tried this and get the same error and an explanation about maybe missing a cert bundle.

----- Original Message ----- From: reply@reply.github.com To: autopkgr@noreply.github.com Cc: JORDAN BENDER (BLOOMBERG/ 731 LEX -) At: Nov 7 2014 15:25:23

Most likely nothing in your shell will work ( not just autopkg(r) ) To test just do curl https://github.com and I'm guessing you'll get the same message. — Reply to this email directly or view it on GitHub.

jmb03012 commented 9 years ago

Would be more then happy too, I just have no idea why its not working though. I mean I know its related to the certificates, but whats actually wrong I cant seem to figure out. Will continue to dig.

----- Original Message ----- From: reply@reply.github.com To: autopkgr@noreply.github.com Cc: JORDAN BENDER (BLOOMBERG/ 731 LEX -) At: Nov 7 2014 15:26:29

If you can get this working in the shell and provide us with some insight on how you did it we may be able to roll it in to AutoPkgr — Reply to this email directly or view it on GitHub.

eahrold commented 9 years ago

In your shell see if you can get it working by exporting the HTTP_PROXY / HTTPS_PROXY settings if you can figure out which do work that's what you'll want to put into the defaults command above

https://wiki.archlinux.org/index.php/proxy_settings

You'll probably have to do a bunch of fiddling but you may stumble on the secret sauce.

eahrold commented 9 years ago

@jmb03012 would you mind running this command and posting the output

scutil --proxy
jmb03012 commented 9 years ago

Unfortunately without an NDA in place and because this is a public forum I cant post the output of that command. Can you tell me if there is something in particular you are looking for?

----- Original Message ----- From: reply@reply.github.com To: autopkgr@noreply.github.com Cc: JORDAN BENDER (BLOOMBERG/ 731 LEX -) At: Nov 7 2014 16:21:00

@jmb03012 would you mind running this command and posting the outputscutil --proxy — Reply to this email directly or view it on GitHub.

jmb03012 commented 9 years ago

Will give this a try a little later or on Monday.

----- Original Message ----- From: reply@reply.github.com To: autopkgr@noreply.github.com Cc: JORDAN BENDER (BLOOMBERG/ 731 LEX -) At: Nov 7 2014 15:34:33

In your shell see if you can get it working by exporting the HTTP_PROXY / HTTPS_PROXY settings if you can figure out which do work that's what you'll want to put into the defaults command above https://wiki.archlinux.org/index.php/proxy_settings You'll probably have to do a bunch of fiddling but you may stumble on the secret sauce. — Reply to this email directly or view it on GitHub.

eahrold commented 9 years ago

That's fine. I was just going to put a second set of eyes on it for you.

Basically if it's returning anything at all it will most likely be a combo of those values you'll want to export into your environment.

And for us, I'm trying to figure out if we can use the Objective-c equivelant to auto-populate the proxy values in autopkgr rather than needing people to do the defaults command to. There are so many different ways to implement proxies, some people set them direct, some use a pac, you said you were using a .dat file. I really don't care about the specifics just wether to you that command looks to be returning the actual values for your proxy configuration.

jmb03012 commented 9 years ago

Here is a redacted version of the results of scutil --proxy on the box I am trying test out autopkg(r) on:

{ ExceptionsList : { 0 : *.local 1 : 169.254/16 } FTPPassive : 1 ProxyAutoConfigEnable : 1 ProxyAutoConfigURLString : http://wpad.company.com/wpadname.dat }
jmb03012 commented 9 years ago

I got it working =)

I had to just specify the proxy url from inside our .dat file for both HTTP and HTTPS proxy settings with the defaults write command and now the repos come down without any issue.

Thanks for all your help!

eahrold commented 9 years ago

@jmb03012 that's great news.
I've started actually working on making this even more stream-lined and auto populating the values using the system settings.

Out of curiosity once you figured out the export values, did you

1) include the url scheme in the proxy address 

export HTTP_PROXY=http://10.0.0.1:8080
export HTTPS_PROXY=https://10.0.0.1:8080

2) omit the url scheme

export HTTP_PROXY=10.0.0.1:8080
export HTTPS_PROXY=10.0.0.1:8080

Many of the docs I've found seem to indicate 1, but we have a few other issues here which seem to indicate 2.

jmb03012 commented 9 years ago

Originally I included the URL scheme in the proxy address but I just tested without it and everything still appears to work.

jmb03012 commented 9 years ago

So the saga continues. I've got a few JSS recipes working as far as downloading, but I'm noticing nothing is going up into the JSS. I did a traceback and I can see in the debug logs that I am getting 403 errors.

I checked the API Username, Password, JSS URL, etc in the autopkg plist and they are all correct. I also verified that the user being used as sufficient privileges (at this point it has full user and API rights).

Any thoughts regarding the 403?

eahrold commented 9 years ago

@homebysix have you seen this before?

jmb03012 commented 9 years ago

It shouldn't matter that I am running Casper 8 vs 9 correct? I don't believe I saw anything on any of the sites to say otherwise.

Also as an aside, if I use Safari just to access resources via the API, when prompted for creeds I tested supplying the same ones that I input into AutoPkgr and I don't get rejected so the account definitely has sufficient rights to communicate with the API.

Lastly, I tried running a Curl command over HTTPS with the same set of creds and had no issue either. Only seems to be getting rejected through AutoPkgr.

eahrold commented 9 years ago

if you can post the traceback and the response message from the debug log that would be helpful.

Retract any private info such as IP addresses, but make sure to keep the path components to the url's intact.

jmb03012 commented 9 years ago

screen shot 2014-11-10 at 4 53 23 pm

jmb03012 commented 9 years ago

As a test I also just tried disabling SSL verification in case it was a certificate issue but I still get a 403 error as well.

eahrold commented 9 years ago

Does your server have an subpath in it's url? Something like https://server.com/myjss

jmb03012 commented 9 years ago

Nope. This is actually a development box so it doesn't even have a real name.

Formatted as follows: HTTPS://0.0.0.0:8443

Sent from Bloomberg Professional for iPhone

----- Original Message ----- From: Eldon Ahrold reply@reply.github.com To: autopkgr@noreply.github.com CC: JORDAN BENDER At: Nov 10, 2014 17:01:17

Dose your server have an subpath in it's url? Something like https://server.com/myjss — Reply to this email directly or view it on GitHub.

eahrold commented 9 years ago

OK, we saw 403 error in the case of cloud instances, but that was only due to it stripping off the path component.

Does your password have any special characters in it. If so see https://github.com/lindegroup/autopkgr/issues/177 It's slightly different and refers to the mounting of DP's but could relate.

jmb03012 commented 9 years ago

I have separate accounts for the DP and for the API access. I changed the password just now for the API access account to one with no special characters (the previous one had them) and although AutoPkgr accepts the password, I still get a 403 error in the log when AutoPkgr attempts to run a Get....

eahrold commented 9 years ago

Do this and check the values were successfully updated

defaults read com.github.autopkg API_USERNAME
defaults read com.github.autopkg API_PASSWORD
eahrold commented 9 years ago

When you go here on a browser what does the resulting XML look like? https://your.jss.server/JSSResource/distributionpoints

jmb03012 commented 9 years ago

defaults read show the correct new username and password for the API user.

shows the following with number and letter formatting kept as is but with values changed for security:

00xx00-xxxxxx00

The actual value listed with the above format is the name of the DP for this JSS and the one that shows as connected in autopkgr. The only thing that is odd though is the first two digits before the first letters are not part of the name and do not show up with the name in the JSS or in autopkgr. May not be related though because the entire GET appears to fail, not just the distro stuff.

I also used the same creeds to access the above data via the API that I have plugged into AutoPkgr.

eahrold commented 9 years ago

I'm really stumped,

on casper 8 what is the full api url to GET the list DPs?

eahrold commented 9 years ago

digging around here seems like it's not authentication, it's the user itself...

http://bryson3gps.wordpress.com/2014/03/30/the-jss-rest-api-for-everyone/

403: You have made a valid request, but you lack permissions to the object you are trying to interract with. You will need to check the permissions of the account being used in the JSS interface under “System Settings > JSS User Accounts & Groups”.

Does the username have any special characters in it per chance?

jmb03012 commented 9 years ago

No special characters in the API name. I have even tried setting up AutoPkgr with the username and password for the initial jss administrative account which has full privs for the JSS and its API but still no dice.

Keep in mind two those accounts that don't work in AutoPkgr work fine if used outside of it. For example if I access the API via terminal or via a web browser and authenticate with those accounts, I can access the API with no issues.

eahrold commented 9 years ago

Is the globe next to JSS URL text field green or yellow? If green (click connect again to double check) then AutoPkgr was actually able to access the API too, and It's something with the python-jss lib itself and we may be better off moving the issue over there since Shea's truly the expert on JSS.

jmb03012 commented 9 years ago

Yup its green, and I've tried hitting connect time over time before but no dice. Good to know though that if its green there it means that AutoPkgr was able to successfully access the API too.

That said, it would point to the issue being with python-jss since the issue only appears to happen once GET is triggered against the JSS. I will see about opening a case with him and linking it to this thread.

eahrold commented 9 years ago

are your proxy defaults setup with the http(s):// schemas included. I just did some cursory research and the requests module that python-jss uses may actually need it. Worth a shot on that.

jmb03012 commented 9 years ago

Right now they aren't as I left them without the schemas from when we were testing earlier in the thread since for autopkg it worked both with and without them.

I just deleted the values for HTTP_PROXY and HTTPS_PROXY in the AutoPkgr plist and re-added them with the http(s):// schemas included and tested running one of the .jss recipes again.

Still getting a jss.jss.JSSGetError: JSS Error. Response Code: 403 Response in the trace pack. Its happening right after jss.py gets called up so something being fed into that is not right.

I tried to look at lines 186 and 199 in jss.py (python is not my strong point) and it looks like it has something to do with getting a url, and the jss returning JSON and not XML but I could be wrong.

eahrold commented 9 years ago

That may be the issue... At line 160 of jss.py

headers = {"content-type": 'text/xml', 'Accept': 'application/xml'}

you can try changing it to 

headers = {"content-type": 'text/json', 'Accept': 'application/json'}

You can also start marking up the jss.py with print() statements for example around L195 you could put in this

        print "URL REQUESTED: %s\n" % url
        print "RESPONSE FROM JSS %s\n" % response.text

Although none of this really gets to the heart of a 403 error.

Have you looked at the logs on your JSS Server to see if there is any indication of what's happening when those requests are being made?

jmb03012 commented 9 years ago

Changing xml to json resulted in the same errors.

The JAMFSoftwareServer log shows nothing about API calls.

In the Tomcat logs folder though, the catalina log does show AutoPkgr making GET commands to /JSSResource/distributionpoints and they appear to be working properly. Output as follows (IPs modified for security; first IP matches the OS X Client with AutoPkgr installed. Second and third IPs match the JSS):

Nov 12, 2014 8:37:35 AM org.restlet.engine.log.LogFilter afterHandle INFO: 2014-11-12 08:37:35 00.00.00.00 - 00.00.00.00 8443 GET /JSSResource/distributionpoints - 401 424 0 2 https://00.00.00.00:8443 AutoPkgr/490 CFNetwork/673.3 Darwin/13.4.0 (x86_64) (VMware7%2C1) - Nov 12, 2014 8:37:35 AM org.restlet.engine.log.LogFilter afterHandle INFO: 2014-11-12 08:37:35 10.16.7.21 AutoPkgr 00.00.00.00 8443 GET /JSSResource/distributionpoints - 200 - 0 17 https://00.00.00.00 AutoPkgr/490 CFNetwork/673.3 Darwin/13.4.0 (x86_64) (VMware7%2C1) -

The tomcat log also shows lines with the identical output as expected, and also shows the successful API tests I did with Curl and via a web browser over the past few days.

eahrold commented 9 years ago

when you had tested with curl had you done the 

export HTTP_PROXY=0.0.0.0:8080

prior?

jmb03012 commented 9 years ago

Just did it with and without and got the same results.

ComputerName:~ UserName$ curl -v -u APIUSERNAME:APIPASSWORD https://00.00.00.00:8443/JSSResource/computers