acme/autocert error

[andpoi]

Hello,

I could be completely dumb here, but i've tried initializing the squirrel server, forwarded dns for the domain i've been using, set the right repo url and auth headers in the munki config script, still have the below error

TLS handshake error from 10.0.0.15:58825: acme/autocert: host not configured

Am I missing a step? I followed your directions to a T.

[andpoi]

TLS handshake error from x.x.x.x:49392: tls: client offered an unsupported, maximum protocol version of 301

[groob]

@andpoi when you're starting the server, are you passing the -tls-domain flag?

If so, this it the domain you need to use to connect to the server. Let's encrypt will only fetch a certificate if the server is running on a valid address.

See https://letsencrypt.org/how-it-works/ for more information.

[andpoi]

I am passing a domain with that flag. When I google the error I can see other people having the problem because of the version of TLS included with python <=2.7.5 is only 1.0. I could be off base here. What version of python are you using?

[groob]

I'm not using python. The language this is written in is Go.

Is the domain you're passing reachable publicly, or are you setting up behind a firewall?

[groob]

Or are you referring to the munki client. The Server is restricted to Modern TLS compatibility. I'm not seeing issues with my munki clients, but I'll investigate a little.

[groob]

@andpoi my python version on the clients is Python 2.7.10.

I'll confirm that 2.7.5 does not support the new TLS versions, and if so, I'll create a flag that relaxes TLS compatibility.

[groob]

Munki does not use system python for network communication, so I wonder what this could be. @andpoi can you give a bit more details about your environment?

1) Is this domain you are using publicly routable, or internal/behind a firewall?

2) Are you passing the domain as just -tls-domain=example.com or are you adding https:// in front?

3) Are you seeing this issue when munki clients are connecting?

4) Are you seeing this issue when you visit https://my-squirrel-server.com/repo/ in your browser?

5) Are you running something like munki-enroll as a prefligt/postflight script? If so, please describe/link to it if you're able.

6) What OS/munki version are you seeing these issues with?

Thanks for your patience and for reporting the issue.

[andpoi]

It is publicly accessible

[andpoi]

1.

Are you passing the domain as just -tls-domain=example.com or are you adding https:// in front?

Just the domain 2.

Are you seeing this issue when munki clients are connecting?

Yes 3.

Are you seeing this issue when you visit https://my-squirrel-server.com/repo/ in your browser? Yes 4.

Are you running something like munki-enroll as a prefligt/postflight script? If so, please describe/link to it if you're able.

No 5.

What OS/munki version are you seeing these issues with?

Latest of OSX and Munki stable release

[groob]

I tailed my logs today and I'm still not convinced this is an issue.

When a munki client connects, I do see a EOF error, but that's because munki doesn't terminate connections properly.

I also see acme/autocert: missing server name but they're not coming from my IPs. If you're server is public you're going to get a lot of random traffic which is safe to ignore.

What still concerns me is that you're seeing a maximum protocol version of 301 error in your browser. This should not happen with any modern OS, but I suspect your server is actually behind some reverse proxy/loadbalancer which requires TLSv1

I'll release an update with a --tls-compatibilty flag so you can downgrade to tlsv1 if needed.

[andpoi]

Thanks for the quick replies @groob,

This is the output from the Squirrel server after starting it and then immediately trying to update the software via managedsoftwareupdate.

Squirrel output: 2017/03/29 13:47:29 http: TLS handshake error from x.x.x.x:49266: EOF 2017/03/29 13:47:35 http: TLS handshake error from x.x.x.x:49267: acme: identifier authorization failed 2017/03/29 13:47:35 http: TLS handshake error from x.x.x.x:49268: acme/autocert: missing certificate 2017/03/29 13:47:35 http: TLS handshake error from x.x.x.x:49269: tls: client offered an unsupported, maximum protocol version of 301 2017/03/29 13:47:35 http: TLS handshake error from x.x.x.x:49270: tls: client offered an unsupported, maximum protocol version of 301

This is the error that the Munki client is receiving:

Managed Software Update Tool Copyright 2010-2016 The Munki Project https://github.com/munki/munki

Starting... No CA cert info provided, so nothing to add to System keychain. No client cert info provided, so no client keychain will be created. Checking for available updates... Manifest base URL is: redacted/Munki_repo/manifests/ Manifest base URL is: redacted/Munki_repo/manifests/ Getting manifest Loaner... Options: {'logging_function': <function display_debug2 at 0x109860cf8>, 'ignore_system_proxy': None, 'additional_headers': {u'Authorization': u'Basic redacted', 'User-Agent': u'managedsoftwareupdate/2.8.2.2855 Darwin/16.4.0'}, 'file': u'/Library/Managed Installs/manifests/client_manifest.plist.download', 'cache_data': None, 'url': u'redacted/Munki_repo/manifests/Loaner', 'follow_redirects': u'none', 'download_only_if_changed': False, 'can_resume': False} 2017-03-29 10:58:50.241 Python[641:10345] NSURLSession/NSURLConnection HTTP load failed (kCFStreamErrorDomainSSL, -9836) Download error -1200: An SSL error has occurred and a secure connection to the server cannot be made. SSL error detail: (-9836L, u'Bad protocol version') Keychain list

***Default keychain info***

ERROR: 1: security: SecKeychainCopyDomainDefault user: A default keychain could not be found. Headers: None ERROR: Could not retrieve manifest Loaner from the server: Error -1200: An SSL error has occurred and a secure connection to the server cannot be made. ERROR: Could not retrieve managed install primary manifest. Finishing... Getting info on currently installed applications... Done.

I hope this is helpful, I'm on a very very vanilla set-up, no reverse proxies or load balancers over here.

[groob]

Thanks, that's very helpful.

One thing to note is that that you have

Manifest base URL is: redacted/Munki_repo/manifests/

but squirrel explicitly expects the url to be .../repo/mannifests and not anything else. (the folder you point to can be named whatever you'd like, just the url has to match)

That should have no bearing on the issue, but you'll start seeing 404 errors even if the SSL is fixed.

[andpoi]

Thanks @groob. I changed the manifest base URL, still having SSL issues. I've confirmed the auth headers are correct.

SSL error detail: (-9836L, u'Bad protocol version') < == This is what made me originally think the TLS version was off

[groob]

there's definitely issues with your client. I just can't seem to duplicate. Our clients are also using 10.12.3-4 and 2.8.2 and there's no issues with the clients.

I'm continuing to look into this though.

[jessepeterson]

One thing I'd check is if there are firewalls or routers or transparent proxies that are messing with any of the traffic in between. If you can verify direct LAN<->IP connectivity that might rule out one avenue of potential issues.

[andpoi]

@jessepeterson I can connect to the server in chrome, just get an SSL error. That makes me think it's something related to SSL and not networking.

[groob]

@andpoi the fact that you're getting this issue in chrome shows that it's not really related to python or munki but something with your network.

is the https://website-url that you access in a browser the same as the one you set with -tls-domain flag?

[jessepeterson]

When you say you can 'connect' can you actually view content? If you're using Let's Encrypt there should be no certificate issues at all.

[andpoi]

@groob Yes when I browse to the same url i set as -tls-domain, i get this error in chrome: ERR_SSL_PROTOCOL_ERROR

@jessepeterson Can't actually see any content, just the SSL error. I'm assuming it's because I don't have the auth headers in my chrome browser?

[groob]

if you're seeing ERR_SSL_PROTOCOL_ERROR with a modern version of chrome on a modern version of macOS there's definitely a firewall/proxy issue.

What do you see if you're go to the munki url with your cellphone when you're not using wifi but just plain cellular?

[andpoi]

So if I quit the squirrel server, my chrome error turns from

This site can’t provide a secure connection ** sent an invalid response.

to

This site can’t be reached ***** refused to connect.

Would that be a firewall issue?

[groob]

That doesn't determine anything. If you stop the server from running, it won't connect.

The firewall issue could be because it's doing some MITM with certs. Try to connect on cellular and see if you have similar errors there.

[andpoi]

@groob same error over LTE

[groob]

I guess this is a serverside proxy then. Is the server running inside your network, or on a cloud provider?

[andpoi]

Inside my network on a mac

[groob]

For reference, this is running on my domain right now:

groob$ sudo squirrel serve -repo /tmp/munkirepo -basic-auth=abcd -tls-domain=dev.micromdm.io

    To connect your clients to the server, you will need to set the authentication headers.
    See https://github.com/munki/munki/wiki/Using-Basic-Authentication#configuring-the-clients-to-use-a-password
    for additional details:

    The headers header you should use is:

    Authorization: Basic c3F1aXJyZWw6YWJjZA==

    To configure manually, use:

    sudo defaults write /Library/Preferences/ManagedInstalls AdditionalHttpHeaders -array "Authorization: Basic c3F1aXJyZWw6YWJjZA=="

There's nothing in the repo, but you should be able to open the site and verify the connection.

[groob]

My money is on your firewall doing MITM on your network. If you can serve the connection without any firewall it would likely work. Unfortunately I don't have other suggestions for firewall issues.

Another thing you could check is to make sure that the system time on the mac you're running the server on is accurate.

[groob]

Closing this issue. I ended up investigating with @andpoi on slack.

Let's encrypt returns an error like

CAA record for munki.acme.co prevents issuance

which means Let's encrypt failed to issue a certificate because the DNS provider is blocking it. Nothing we can do here.

Note for anyone else: If you see acme: identifier authorization failed in the logs, that means that LE failed to complete the Let's Encrypt challenge for an unknown reason. I had a similar error, but caused beacause DNS didn't fully propagate:

      "type": "tls-sni-01",
      "status": "invalid",
      "error": {
        "type": "urn:acme:error:connection",
        "detail": "DNS problem: NXDOMAIN looking up A for repo.micromdm.io",
        "status": 400
      },