certbot fails on Ubuntu 14.0.4.5 (LTS)

rfrush commented 6 years ago

I'm getting an error trying to provision Let's Encrypt certificates with the error: OpenSSL.SSL.Error: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')]

I first reported this, and have a 'weak' workaround discussed here: https://discourse.mailinabox.email/t/error-during-provision-tls-missing-flags-for-certbot/3585

In short, when I rrn ‘sudo mailinabox’:

Mail-in-a-Box Version:  v0.28

Updating system packages...
Installing system packages...
Initializing system random number generator...
Firewall is active and enabled on system startup
Creating initial SSL certificate and perfect forward secrecy Diffie-Hellman parameters...
Generating DH parameters, 2048 bit long safe prime, generator 2
This is going to take a long time    ........................................+................................................................................+...............+.................................................................................................++*++*
Installing nsd (DNS server)...
Installing Postfix (SMTP server)...
Installing Dovecot (IMAP server)...
Installing OpenDKIM/OpenDMARC...
Installing SpamAssassin...
Installing Nginx (web server)...
Installing Roundcube (webmail)...
Installing Nextcloud (contacts/calendar)...
Nextcloud is already latest version
Installing Z-Push (Exchange/ActiveSync server)...
Installing Mail-in-a-Box system management daemon...
Installing Munin (system monitoring)...
updated DNS: rflm.net
web updated

-----------------------------------------------
Mail-in-a-Box uses Let's Encrypt to provision free certificates
to enable HTTPS connections to your box. You'll now be asked to agree
to Let's Encrypt's terms of service.

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Registering without email!
From cffi callback <function _verify_callback at 0x7ff588689bf8>:
Traceback (most recent call last):
  File "/usr/local/lib/python3.4/dist-packages/OpenSSL/SSL.py", line 309, in wrapper
    _lib.X509_up_ref(x509)
AttributeError: 'module' object has no attribute 'X509_up_ref'
An unexpected error occurred:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/contrib/pyopenssl.py", line 438, in wrap_socket
    cnx.do_handshake()
  File "/usr/local/lib/python3.4/dist-packages/OpenSSL/SSL.py", line 1907, in do_handshake
    self._raise_ssl_error(self._ssl, result)
  File "/usr/local/lib/python3.4/dist-packages/OpenSSL/SSL.py", line 1639, in _raise_ssl_error
    _raise_current_error()
  File "/usr/local/lib/python3.4/dist-packages/OpenSSL/_util.py", line 54, in exception_from_error_queue
    raise exception_type(errors)
OpenSSL.SSL.Error: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')]

I took the issue up with Let's Encrypt, and have had the following discussion which suggests that this is an issue with Python3.4 on Ubuntu 14.0.4.5 (LTS). The 'certbot' command, run from the command line fails in the same way.

https://community.letsencrypt.org/t/certificate-verify-failed/69444

Using curl -v https://acme-v01.api.letsencrypt.org/directory, the system can connect, but using:

$ python3.4
>>> import requests
>>> requests.get("https://acme-v01.api.letsencrypt.org/directory")

Generates the same errors about SSL3_GET_SERVER_CERTIFICATE as shown above.

I've re-installed 'certbot' (and related packages) on the Base system, which is a DigitalOcean Ubuntu 14.0.4.5 (LTS) droplet with no improvement in behavior.

-- Ray Frush

aroundmyroom commented 6 years ago

A looks like similar, but still different issue here:

Saving debug log to /var/log/letsencrypt/letsencrypt.log Registering without email!

Please read the Terms of Service at https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf. You must agree in order to register with the ACME server at https://acme-v02.api.letsencrypt.org/directory

(A)gree/(C)ancel: (A)gree/(C)ancel: An unexpected error occurred: Traceback (most recent call last): File "/usr/bin/certbot", line 11, in load_entry_point('certbot==0.26.1', 'console_scripts', 'certbot')() File "/usr/lib/python3/dist-packages/certbot/main.py", line 1364, in main return config.func(config, plugins) File "/usr/lib/python3/dist-packages/certbot/main.py", line 721, in register _determine_account(config) File "/usr/lib/python3/dist-packages/certbot/main.py", line 520, in _determine_account config, account_storage, tos_cb=_tos_cb) File "/usr/lib/python3/dist-packages/certbot/client.py", line 182, in register regr = perform_registration(acme, config, tos_cb) File "/usr/lib/python3/dist-packages/certbot/client.py", line 207, in perform_registration tos_cb) File "/usr/lib/python3/dist-packages/acme/client.py", line 777, in new_account_and_tos _assess_tos(self.client.directory.meta.terms_of_service) File "/usr/lib/python3/dist-packages/acme/client.py", line 768, in _assess_tos check_tos_cb(tos) File "/usr/lib/python3/dist-packages/certbot/main.py", line 498, in _tos_cb cli_flag="--agree-tos", force_interactive=True) File "/usr/lib/python3/dist-packages/certbot/display/util.py", line 223, in yesno no=_parens_around_char(no_label))) File "/usr/lib/python3/dist-packages/certbot/display/util.py", line 90, in input_with_timeout raise EOFError EOFError

rfrush commented 6 years ago

The folks over at Let's Encrypt called out that the certbot script is somehow using /usr/local/lib/python3.4/dist-packages/OpenSSL/SSL.py instead of the standard python libraries delivered by Ubuntu at /usr/lib/python3/dist-packages/OpenSSL/SSL.py (See https://community.letsencrypt.org/t/certificate-verify-failed/69444)

I haven't dug around in the code here to understand how mail-in-a-box is delivering it's own python, but something about that appears to be messing with 'certbot'

aroundmyroom commented 6 years ago

fixed my issue by running sudo mailinabox again .. (see forum)

rfrush commented 6 years ago

I've re-tried running sudo mailinabox several times with no change in behavior.

ChiefGyk3D commented 6 years ago

I am also getting this error `Mail-in-a-Box uses Let's Encrypt to provision free certificates to enable HTTPS connections to your box. You'll now be asked to agree to Let's Encrypt's terms of service.

Saving debug log to /var/log/letsencrypt/letsencrypt.log Registering without email!

Please read the Terms of Service at https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf. You must agree in order to register with the ACME server at https://acme-v02.api.letsencrypt.org/directory

(A)gree/(C)ancel: (A)gree/(C)ancel: An unexpected error occurred: Traceback (most recent call last): File "/usr/bin/certbot", line 11, in load_entry_point('certbot==0.26.1', 'console_scripts', 'certbot')() File "/usr/lib/python3/dist-packages/certbot/main.py", line 1364, in main return config.func(config, plugins) File "/usr/lib/python3/dist-packages/certbot/main.py", line 721, in register _determine_account(config) File "/usr/lib/python3/dist-packages/certbot/main.py", line 520, in _determine_account config, account_storage, tos_cb=_tos_cb) File "/usr/lib/python3/dist-packages/certbot/client.py", line 182, in register regr = perform_registration(acme, config, tos_cb) File "/usr/lib/python3/dist-packages/certbot/client.py", line 207, in perform_registration tos_cb) File "/usr/lib/python3/dist-packages/acme/client.py", line 777, in new_account_and_tos _assess_tos(self.client.directory.meta.terms_of_service) File "/usr/lib/python3/dist-packages/acme/client.py", line 768, in _assess_tos check_tos_cb(tos) File "/usr/lib/python3/dist-packages/certbot/main.py", line 498, in _tos_cb cli_flag="--agree-tos", force_interactive=True) File "/usr/lib/python3/dist-packages/certbot/display/util.py", line 223, in yesno no=_parens_around_char(no_label))) File "/usr/lib/python3/dist-packages/certbot/display/util.py", line 90, in input_with_timeout raise EOFError EOFError Please see the logfiles in /var/log/letsencrypt for more details.

Your Mail-in-a-Box is running. `

rfrush commented 6 years ago

aroundmyroom, ChiefGyk- The error you're getting is different than the error I'm reporting. I'm seeing: OpenSSL.SSL.Error: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')]

I'll let one of the developers weigh in to see if these problems need to be split into different issues.

ChiefGyk3D commented 6 years ago

I resolved it by manually running sudo mailinabox It then allowed me to agree to ToS, and everything is working as it should be. Seems the documented upgrade approach of piping commands just skips over allowing user input when agreeing to ToS. Maybe we need to add the flag to automatically agree to ToS in the script.

nomandera commented 6 years ago

Adding in the error message the system admin will receive for those that search here on that.

Provisioning TLS certificates for ...
error: ...
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None
You should register before running non-interactively, or provide --agree-tos and --email <email_address> flags.

Also fixed via sudo mailinabox and accepting the ToS. Thanks @ChiefGyk

ChrisScrumping commented 6 years ago

I was also having this issue, running it manually also solved it for me - thanks @ChiefGyk

BrewKeg commented 6 years ago

I had the same issue. 'sudo mailinabox' worked for me too.

ajankuv commented 6 years ago

sudo mailinabox worked, it allows you to accept the certbot

boralp commented 6 years ago

after sudo mailinabox the browser still shows me a self-signed ssl warning for the connection I had hence letsencrypt issued a valid certificate for the domain I had.

cd ~/mailinabox sudo management/ssl_certificates.py

updated my certificates and insecure connection label is gone. letsencrypt started to work

danyeaw commented 6 years ago

I was also having this issue, sudo mailinabox resolved it for me as well.

cooperaj commented 5 years ago

Sadly, sudo mailinabox is not solving it for me. Not sure what to do...

I had to run

sudo rm /home/user-data/ssl/lets_encrypt/accounts/acme-v02.api.letsencrypt.org/

to force the above command to regenerate the certbot config (and hence register the --agree-tos flag)

JoshData commented 5 years ago

@rfrush What version of Mail-in-a-Box are you running and are you still having the problem?

rfrush commented 5 years ago

Currently I'm on the master branch. The install was a fresh v.28 install

https://discourse.mailinabox.email/t/error-during-provision-tls-missing-flags-for-certbot/3585/8 " I found a non-optimal workaround for this issue. I added the following line to /etc/letsencrypt/cli.ini

no-verify-ssl = true " The certs are due to renew in about 5 days, and I'm keen to find out if the current install updates the certs without issue.

I have the 'no-verify-ssl' line commented out.

lilfade commented 5 years ago

Sadly, sudo mailinabox is not solving it for me. Not sure what to do...

I had to run
sudo rm /home/user-data/ssl/lets_encrypt/accounts/acme-v02.api.letsencrypt.org/
to force the above command to regenerate the certbot config (and hence register the --agree-tos flag)

it's a folder so add in the -rf flag, but this worked for me upgrading from v0.28 and kept getting the error about accepting terms. Even after updating to v0.29 and trying the sudo mailinabox command. Neither worked before removing the folder.

rfrush commented 5 years ago

My renewal failed... Same ssl verification error. To be clear, for everyone who keeps cross posting the "--agree-tos" this is not the same issue.

Provisioning TLS certificates for box.rflm.net, rflm.net, www.rflm.net.
error: box.rflm.net, rflm.net, www.rflm.net:
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None
From cffi callback <function _verify_callback at 0x7f887a838378>:
Traceback (most recent call last):
  File "/usr/local/lib/python3.4/dist-packages/OpenSSL/SSL.py", line 309, in wrapper
    _lib.X509_up_ref(x509)
AttributeError: 'module' object has no attribute 'X509_up_ref'
An unexpected error occurred:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/contrib/pyopenssl.py", line 438, in wrap_socket
    cnx.do_handshake()
  File "/usr/local/lib/python3.4/dist-packages/OpenSSL/SSL.py", line 1907, in do_handshake
    self._raise_ssl_error(self._ssl, result)
  File "/usr/local/lib/python3.4/dist-packages/OpenSSL/SSL.py", line 1639, in _raise_ssl_error
    _raise_current_error()
  File "/usr/local/lib/python3.4/dist-packages/OpenSSL/_util.py", line 54, in exception_from_error_queue
    raise exception_type(errors)
OpenSSL.SSL.Error: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 345, in _make_request
    self._validate_conn(conn)
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 846, in _validate_conn
    conn.connect()
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 326, in connect
    ssl_context=context)
  File "/usr/lib/python3/dist-packages/urllib3/util/ssl_.py", line 325, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "/usr/lib/python3/dist-packages/urllib3/contrib/pyopenssl.py", line 445, in wrap_socket
    raise ssl.SSLError('bad handshake: %r' % e)
ssl.SSLError: ("bad handshake: Error([('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')],)",)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/requests/adapters.py", line 440, in send
    timeout=timeout
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 630, in urlopen
    raise SSLError(e)
urllib3.exceptions.SSLError: ("bad handshake: Error([('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')],)",)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/bin/certbot", line 11, in <module>
    load_entry_point('certbot==0.26.1', 'console_scripts', 'certbot')()
  File "/usr/lib/python3/dist-packages/certbot/main.py", line 1364, in main
    return config.func(config, plugins)
  File "/usr/lib/python3/dist-packages/certbot/main.py", line 1238, in certonly
    le_client = _init_le_client(config, auth, installer)
  File "/usr/lib/python3/dist-packages/certbot/main.py", line 648, in _init_le_client
    return client.Client(config, acc, authenticator, installer, acme=acme)
  File "/usr/lib/python3/dist-packages/certbot/client.py", line 247, in __init__
    acme = acme_from_config_key(config, self.account.key, self.account.regr)
  File "/usr/lib/python3/dist-packages/certbot/client.py", line 50, in acme_from_config_key
    return acme_client.BackwardsCompatibleClientV2(net, key, config.server)
  File "/usr/lib/python3/dist-packages/acme/client.py", line 744, in __init__
    directory = messages.Directory.from_json(net.get(server).json())
  File "/usr/lib/python3/dist-packages/acme/client.py", line 1078, in get
    self._send_request('GET', url, **kwargs), content_type=content_type)
  File "/usr/lib/python3/dist-packages/acme/client.py", line 1027, in _send_request
    response = self.session.request(method, url, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 502, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 612, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/adapters.py", line 514, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: ("bad handshake: Error([('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')],)",)
Please see the logfiles in /var/log/letsencrypt for more details.

rfrush commented 5 years ago

I let this fail for a few days in a row and then re-set the 'no-verify-ssl = true' setting and I got my certs renewed with some interesting warnings from python about "AttributeError: 'module' object has no attribute 'X509_up_ref'"

* Stopping Postfix Mail Transport Agent postfix
   ...done.
 * Starting Postfix Mail Transport Agent postfix
   ...done.
dovecot stop/waiting
dovecot start/running, process 29542
 * Reloading nginx configuration nginx
   ...done.
Provisioning TLS certificates for box.rflm.net, rflm.net, www.rflm.net.
installed: box.rflm.net, rflm.net, www.rflm.net:
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None
From cffi callback <function _verify_callback at 0x7fda178af378>:
Traceback (most recent call last):
  File "/usr/local/lib/python3.4/dist-packages/OpenSSL/SSL.py", line 309, in wrapper
    _lib.X509_up_ref(x509)
AttributeError: 'module' object has no attribute 'X509_up_ref'
/usr/lib/python3/dist-packages/urllib3/connectionpool.py:854: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
/usr/lib/python3/dist-packages/urllib3/connectionpool.py:854: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
/usr/lib/python3/dist-packages/urllib3/connectionpool.py:854: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
/usr/lib/python3/dist-packages/urllib3/connectionpool.py:854: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
/usr/lib/python3/dist-packages/urllib3/connectionpool.py:854: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
/usr/lib/python3/dist-packages/urllib3/connectionpool.py:854: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
Performing the following challenges:
http-01 challenge for box.rflm.net
http-01 challenge for rflm.net
http-01 challenge for www.rflm.net
Using the webroot path /home/user-data/ssl/lets_encrypt/webroot for all unmatched domains.
Waiting for verification...
/usr/lib/python3/dist-packages/urllib3/connectionpool.py:854: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
/usr/lib/python3/dist-packages/urllib3/connectionpool.py:854: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
/usr/lib/python3/dist-packages/urllib3/connectionpool.py:854: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
/usr/lib/python3/dist-packages/urllib3/connectionpool.py:854: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
/usr/lib/python3/dist-packages/urllib3/connectionpool.py:854: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
/usr/lib/python3/dist-packages/urllib3/connectionpool.py:854: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
Cleaning up challenges
/usr/lib/python3/dist-packages/urllib3/connectionpool.py:854: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
/usr/lib/python3/dist-packages/urllib3/connectionpool.py:854: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
/usr/lib/python3/dist-packages/urllib3/connectionpool.py:854: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
Server issued certificate; certificate written to /tmp/tmpmhsjbbq9/cert
Cert chain written to 10
Cert chain written to 11
IMPORTANT NOTES:
 - Congratulations! Your certificate and chain have been saved at:
   /tmp/tmpmhsjbbq9/cert_and_chain.pem
   Your cert will expire on 2019-01-31. To obtain a new or tweaked
   version of this certificate in the future, simply run certbot
   again. To non-interactively renew *all* of your certificates, run
   "certbot renew"
 - If you like Certbot, please consider supporting our work by:

   Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
   Donating to EFF:                    https://eff.org/donate-le

ozerman2 commented 5 years ago

Sadly, sudo mailinabox is not solving it for me. Not sure what to do... I had to run
sudo rm /home/user-data/ssl/lets_encrypt/accounts/acme-v02.api.letsencrypt.org/
to force the above command to regenerate the certbot config (and hence register the --agree-tos flag)
it's a folder so add in the -rf flag, but this worked for me upgrading from v0.28 and kept getting the error about accepting terms. Even after updating to v0.29 and trying the sudo mailinabox command. Neither worked before removing the folder.

This was the only thing that could get it working again for me.

backwoodsman commented 5 years ago

I have been running MIAB for a couple of years. Now it has begun complaining about its failure to provision the TLS/SSL certificates. Today, the certificates for a dozen domains have expired. I have seen a stack of threads about this or similar problems, but no definitive answer. Most seem to suggest that it is enough to run sudo mailinabox. Surely someone must by now have identified the bug and be able to explain how to get new certificates? I have a few weeks until the most important certificates expire, but I don't relish having to find a replacement for MIAB.

An typical example of the logs on my box:

2018-11-13 14:23:10,247:DEBUG:certbot.main:certbot version: 0.26.1
2018-11-13 14:23:10,247:DEBUG:certbot.main:Arguments: ['--non-interactive', '-d', 'list-of-domains.eu', '--csr', '/tmp/tmpia69vw1c', '--cert-path', '/tmp/tmpuuh7b8hg/cert', '--chain-path', '/tmp/tmpuuh7b8hg/chain', '--fullchain-path', '/tmp/tmpuuh7b8hg/cert_and_chain.pem', '--webroot', '--webroot-path', '/home/user-data/ssl/lets_encrypt/webroot', '--config-dir', '/home/user-data/ssl/lets_encrypt']
2018-11-13 14:23:10,247:DEBUG:certbot.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#manual,PluginEntryPoint#null,PluginEntryPoint#standalone,PluginEntryPoint#webroot)
2018-11-13 14:23:10,257:DEBUG:certbot.log:Root logging level set at 20
2018-11-13 14:23:10,258:INFO:certbot.log:Saving debug log to /var/log/letsencrypt/letsencrypt.log
2018-11-13 14:23:10,259:DEBUG:certbot.plugins.selection:Requested authenticator webroot and installer None
2018-11-13 14:23:10,260:DEBUG:certbot.plugins.selection:Single candidate plugin: * webroot
Description: Place files in webroot directory
Interfaces: IAuthenticator, IPlugin
Entry point: webroot = certbot.plugins.webroot:Authenticator
Initialized: <certbot.plugins.webroot.Authenticator object at 0x7fa27383ba90>
Prep: True
2018-11-13 14:23:10,260:DEBUG:certbot.plugins.selection:Selected authenticator <certbot.plugins.webroot.Authenticator object at 0x7fa27383ba90> and installer None
2018-11-13 14:23:10,260:INFO:certbot.plugins.selection:Plugins selected: Authenticator webroot, Installer None
2018-11-13 14:23:10,261:DEBUG:certbot.log:Exiting abnormally:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/certbot/display/ops.py", line 50, in get_email
    force_interactive=True)
  File "/usr/lib/python3/dist-packages/certbot/display/util.py", line 529, in input
    self._interaction_fail(message, cli_flag)
  File "/usr/lib/python3/dist-packages/certbot/display/util.py", line 474, in _interaction_fail
    raise errors.MissingCommandlineFlag(msg)
certbot.errors.MissingCommandlineFlag: Missing command line flag or config entry for this setting:
Enter email address (used for urgent renewal and security notices)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/bin/certbot", line 11, in <module>
    load_entry_point('certbot==0.26.1', 'console_scripts', 'certbot')()
  File "/usr/lib/python3/dist-packages/certbot/main.py", line 1364, in main
    return config.func(config, plugins)
  File "/usr/lib/python3/dist-packages/certbot/main.py", line 1238, in certonly
    le_client = _init_le_client(config, auth, installer)
  File "/usr/lib/python3/dist-packages/certbot/main.py", line 641, in _init_le_client
    acc, acme = _determine_account(config)
  File "/usr/lib/python3/dist-packages/certbot/main.py", line 517, in _determine_account
    config.email = display_ops.get_email()
  File "/usr/lib/python3/dist-packages/certbot/display/ops.py", line 54, in get_email
    raise errors.MissingCommandlineFlag(msg)
certbot.errors.MissingCommandlineFlag: You should register before running non-interactively, or provide --agree-tos and --email <email_address> flags.

I have tried running lets_encrypt and certbot from cli, but the response seems to be

Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

No renewals were attempted.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Can anyone tell me the correct procedure to overcome this bug?

ozerman2 commented 5 years ago

Did you even read this thread? The solution is here...

backwoodsman commented 5 years ago

By my reading of the thread, there are three solutions suggested. (1) sudo mailinabox -- worked for 6 people, failed for 3 (2) no-verify-ssl=true -- worked for one person (3) rm /home/user-data/.../acme-v02.api.letsencrypt.org/ seems to have worked for two people and perhaps another. Nothing I read here suggests these are definitive bug-fixes, and two of them look like kludges. None has helped on my box. Other threads suggest other solutions, such as pip3 uninstall pyOpenSSL followed by pip3 install pyOpenSSL, which produced unexpected messages on my box, but again seems like someone speculating and investigating rather than a clear answer from the developers, which is what a user who is not a python programmer needs. A user installing xxx-in-a-box really does not expect to have to dismantle all the parts in the box.

But thanks for troubling to reply.

ozerman2 commented 5 years ago

It says right in the error you haven't agreed to the TOS. This works in the latest verion of MIAB. In order to fix it you need to manually agree to the TOS or get the installer to agree to the TOS.

backwoodsman commented 5 years ago

I have not been asked for agreement to anything. I always agree. But if the app doesn't ask, it will never know that.

I did try manually running certbot for one domain, and for that, it did ask. But I had no idea whether to use webroot or a temporary instance. For webroot, it asked me for the path, which I was unable to find in any setup file. And the temporary instance appeared to succeed but has apparently put the certificate somewhere MIAB does not expect it to be.

ozerman2 commented 5 years ago

That's why deleting the folder will force it to ask the next time...

backwoodsman commented 5 years ago

Then why does MIAB not delete the folder? leaving users to search for 4 or 5 hours is no way for an automated system to run, IMHO. /rant Ok, I will try deleting the folder. Then, I guess, I need to run management/ssl_certificates.py or sudo mailinabox or letsencrypt -renew or something...

ozerman2 commented 5 years ago

IDK but this is a minor problem compared to the fact we aren't allowed to customize nextcloud, use document editing plugins, or the fact there doesn't seem to be any working CalDav calendar for windows.

backwoodsman commented 5 years ago

I can see that not being able to use your favourite editor, or to customize nextcloud would be more serious for some people. I gave up on windows back in 1998, so that is certainly not on my horizon as a problem. But my trivial problem threatens the viability of my mailserver, so it is to me non-trivial. I appologise for getting a little het-up about it.

Anyway, I deleted the directory and reran mailinabox for the umpteenth time. This time it responded

Updating system packages...
Installing system packages...
Initializing system random number generator...
[...]
updated DNS: OpenDKIM configuration

-----------------------------------------------
Mail-in-a-Box uses Let's Encrypt to provision free SSL/TLS certificates
to enable HTTPS connections to your box. We're automatically
agreeing you to their subscriber agreement. See https://letsencrypt.org.

usage: 
  certbot [SUBCOMMAND] [options] [-d DOMAIN] [-d DOMAIN] ...

Certbot can obtain and install HTTPS/TLS/SSL certificates.  By default,
it will attempt to use a webserver both for obtaining and installing the
certificate. 
certbot: error: unrecognized arguments: --PRIVATE_IPV6 2001:41c9:1:424::159

-----------------------------------------------

Your Mail-in-a-Box is running.
[...]

So the agreement issue seems to be dealt with. I have grepped that IPV6 and am trying adding back all the extra zeros to see whether that resolves that issue... There must be a way to force certbot to reload the config... Oh well... rerun mailinabox for the (umpteen+1)th time... No, that does not work, since MIAB accepts the PUBLIC_IPV6 address with the additional zeros, but prunes them out to the same "unrecognised argument" for the PRIVATE_IPV6. I shall have to find the other thread that deals with deleting IPV6 from the configuration as a solution to -- perhaps this very issue...

JoshData commented 5 years ago

Folks, I'm sorry there are bugs. I try to fix as many as I can --- and I try to merge bug-fixing pull requests whenever people submit fixes. It's hard to make Mail-in-a-Box work flawlessless. :) We're all trying.

But if you think that it's a major problem that you aren't "allowed" to customize components, please take your commentary elsewhere! Those of us volunteering to keep this ship afloat are not doing anything that prevents anyone from doing anything. Log into your server and make whatever changes you need, and if you don't know how --- don't expect we do either!

backwoodsman commented 5 years ago

@JoshData I apologise for my negative tone. I truly do appreciate the work done by the developers, and in general I have found MIAB to be brilliant. I lost my cool when I had spent more than half a day searching for a way to keep it running. Happily, it has just now "come right".

Replying to my own previous comment, it seemed to be impossible to eliminate the IPV6 address data -- I assume MIAB gets it from the DNS. However, after I removed the line PRIVATE_IPV6=... from /etc/letsencrypt/cli.ini and ran sudo mailinabox another two or three times there was a magical recovery, and the missing certificates all renewed from the admin interface. The automatic agreement to Lets Encrypt subscriber agreement apparently cleared the way. A new set of InsecureRequestWarnings has popped up (/usr/local/lib/python3.4/dist-packages/urllib3/connectionpool.py:858: InsecureRequestWarning: Unverified HTTPS request is being made...) three iterations for each certificate, but the certificates are provisioned and the miab server is fully functional again.

JoshData commented 5 years ago

I was really looking at ozerman2. :)

wuxmedia commented 5 years ago

so is there a fix, or is it YMMV ? deleting the folder nor upgrading mailinabox - I tried running certbot register which completed and --agree-tos but none (and I've tried everything on this thread and beyond) seems to have fixed it :/

wuxmedia commented 5 years ago

Ok so I persevered (it's my job, literally) and after re-reading the whole thread slowly, I did this which worked: root@mail:/home/user-data/ssl# mv * /root/ssl/ which cleared that dir great. then reran mailinabox. Maybe because I had already agreed to it on the command line it did not ask me to accept anything, IT did recreate the DHkey - then just completed. Once miab had finished I reran /root/mailinabox/management/ssl_certificates.py and it WORKED! thanks @JoshData this has been so far the least hassly way of running a very well configured mail server. I would propose a fix of backing up then removing that ssl dir, then running the cert script, but what do I know - I'm just a sysadmin grunt :D

dddcooke commented 5 years ago

Ok so I persevered (it's my job, literally) and after re-reading the whole thread slowly, I did this which worked: root@mail:/home/user-data/ssl# mv * /root/ssl/ which cleared that dir great. then reran mailinabox. Maybe because I had already agreed to it on the command line it did not ask me to accept anything, IT did recreate the DHkey - then just completed. Once miab had finished I reran /root/mailinabox/management/ssl_certificates.py and it WORKED! thanks @JoshData this has been so far the least hassly way of running a very well configured mail server. I would propose a fix of backing up then removing that ssl dir, then running the cert script, but what do I know - I'm just a sysadmin grunt :D

This worked for me with my renewal of certificates, after running MIAB and rebooting many times. Although it never asked me to accepted anything it automatically excepted after the install...

"Mail-in-a-Box uses Let's Encrypt to provision free SSL/TLS certificates to enable HTTPS connections to your box. We're automatically agreeing you to their subscriber agreement. See https://letsencrypt.org."

Thanks Dave

mail-in-a-box / mailinabox

certbot fails on Ubuntu 14.0.4.5 (LTS) #1415