create authority fails with nonetype

[JohnTheodore]

I'm deploying lemur in my environment now to test it out that is works.

We're using oauth2 with a oauth2 identity provider written in go. So oauth2 was broken for me until I changed the lowercase b to a capital B here: https://github.com/Netflix/lemur/compare/master...JohnTheodore:master

Here if the RFC and the built in library/function for golang: RFC: https://tools.ietf.org/html/rfc7617#section-2 and golang link: https://golang.org/src/net/http/request.go?s=27803:27869#L857

I think it's supposed to be capital Basic.

So once I fixed that, oauth2 works now.

I'm having an additional problem now. So my lemur.conf.py variables that matter more are here: https://gist.github.com/JohnTheodore/bfbd98105c7aa66b44d895018b9a4e35

After I can login as the first time (with my oauth2 user having the 'admin' role attached), I believe the next step I need to take is to create an authority. When I try to create an authority I get this error in the lemur journal: https://gist.github.com/JohnTheodore/19e8803f9c50a65be1b861aa67ce5222

There are dozens of fields in the form for creating an authority. I'm not sure which part of is is causing this error: TypeError: 'NoneType' object cannot be interpreted as an integer

Note, once I have things working and can cut a certificate, I'll make a small PR with any fixes or docs that can make things more clear.

[JohnTheodore]

Here are my kwargs from lemur/certificates/service.py(237)

https://gist.github.com/JohnTheodore/baae0d3cfd24881d5f3b88ac7af96f89

I can see 'body' is empty. When I make the authority am I also supposed to make a new cert as well?

[kevgliss]

For authorities, these are typically configured via the plugin itself: http://lemur.readthedocs.io/en/latest/administration.html#plugin-specific-options

You can typically configure which root/intermediate you need from your CA. When you create an authority for that plugin it will use these values. Some authorities create their values dynamically. Symantec/Verisign does not.

[JohnTheodore]

I set all those options in my lemur.conf.py: https://gist.github.com/JohnTheodore/bfbd98105c7aa66b44d895018b9a4e35

Though there are no authorities listed when I sign in.

[kevgliss]

I believe VERISIGN_INTERMEDIATE and VERISIGN_ROOT both need to be set, and be pem formatted strings. You can easily do open('path-to-pem', 'r').readlines() if thats what if you prefer.

[JohnTheodore]

hmm, I set all the variables for VERISIGN_* now.

VERISIGN_URL = 'certmanager-webservices.websecurity.symantec.com/vswebservices'
VERISIGN_PEM_PATH = '/etc/lemur/jt-vice-mpki-api.p7b'
VERISIGN_FIRST_NAME = 'lemur'
VERISIGN_LAST_NAME = 'viceapi'
VERISIGN_EMAIL = 'do_not_reply@example.com'
VERISIGN_INTERMEDIATE = '/etc/lemur/digitcert.intermediate.pem'
VERISIGN_ROOT = '/etc/lemur/digitcert.root.pem'

and these are set too:

# set the default plugin
LEMUR_DEFAULT_ISSUER_PLUGIN = 'verisign-issuer'
LEMUR_DEFAULT_AUTHORITY = 'verisign'```

but.. there is still no authority listed.

[kevgliss]

1) Setting those files only allows the plugin to work, you still need to create an authority and select that plugin. 2) The values need to be PEM strings not file paths (you have paths listed atm).

[JohnTheodore]

ok, these settings worked for creating an authority:

# MPKI settings
VERISIGN_URL = 'https://certmanager-webservices.websecurity.symantec.com/vswebservices'
VERISIGN_PEM_PATH = '/etc/lemur/jt-vice-mpki-api.p7b'
VERISIGN_FIRST_NAME = 'lemur'
VERISIGN_LAST_NAME = 'viceapi'
VERISIGN_EMAIL = 'do_not_reply@example.com'
VERISIGN_INTERMEDIATE = open('/etc/lemur/digitcert.intermediate.pem', 'r').read()
VERISIGN_ROOT = open('/etc/lemur/digitcert.root.pem', 'r').read()

When trying to create my first test certificate, I get this error:

[2018-05-24 07:42:14,716] ERROR in schema: [('PEM routines', 'PEM_read_bio', 'no start line'), ('SSL routines', 'SSL_CTX_use_certificate_file', 'PEM lib')]
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/lemur/common/schema.py", line 160, in decorated_function
    resp = f(*args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/lemur/certificates/views.py", line 270, in post
    cert = service.create(**data)
  File "/usr/local/lib/python3.5/dist-packages/lemur/certificates/service.py", line 249, in create
    cert_body, private_key, cert_chain, external_id, csr = mint(**kwargs)
  File "/usr/local/lib/python3.5/dist-packages/lemur/certificates/service.py", line 196, in mint
    cert_body, cert_chain, external_id = issuer.create_certificate(csr, kwargs)
  File "/usr/local/lib/python3.5/dist-packages/lemur/plugins/lemur_verisign/plugin.py", line 197, in create_certificate
    response = self.session.post(url, data=data)
  File "/usr/local/lib/python3.5/dist-packages/requests/sessions.py", line 522, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/requests/sessions.py", line 475, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.5/dist-packages/raven/breadcrumbs.py", line 326, in send
    resp = real_send(self, request, *args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/requests/sessions.py", line 596, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/requests/adapters.py", line 423, in send
    timeout=timeout
  File "/usr/local/lib/python3.5/dist-packages/requests/packages/urllib3/connectionpool.py", line 595, in urlopen
    chunked=chunked)
  File "/usr/local/lib/python3.5/dist-packages/requests/packages/urllib3/connectionpool.py", line 352, in _make_request
    self._validate_conn(conn)
  File "/usr/local/lib/python3.5/dist-packages/requests/packages/urllib3/connectionpool.py", line 831, in _validate_conn
    conn.connect()
  File "/usr/local/lib/python3.5/dist-packages/requests/packages/urllib3/connection.py", line 289, in connect
    ssl_version=resolved_ssl_version)
  File "/usr/local/lib/python3.5/dist-packages/requests/packages/urllib3/contrib/pyopenssl.py", line 321, in ssl_wrap_socket
    ctx.use_certificate_file(certfile)
  File "/usr/local/lib/python3.5/dist-packages/OpenSSL/SSL.py", line 817, in use_certificate_file
    _raise_current_error()
  File "/usr/local/lib/python3.5/dist-packages/OpenSSL/_util.py", line 54, in exception_from_error_queue
    raise exception_type(errors)

from the debugger:

ipdb> data
{'validityPeriod': '1Y', 'email': 'do_not_reply@example.com', 'certProductType': 'Server', 'specificEndDate': '05/25/2018', 'csr': '-----BEGIN CERTIFICATE REQUEST-----\morestuffhere\n-----END CERTIFICATE REQUEST-----\n', 'signatureAlgorithm': 'sha256WithRSAEncryption', 'subject_alt_names': '', 'challenge': '12345', 'serverType': 'Apache', 'firstName': 'lemur', 'lastName': 'viceapi'}
ipdb> url
'https://certmanager-webservices.websecurity.symantec.com/vswebservices/rest/services/enroll'
ipdb> response = self.session.post(url, data=data)
*** OpenSSL.SSL.Error: [('PEM routines', 'PEM_read_bio', 'no start line'), ('SSL routines', 'SSL_CTX_use_certificate_file', 'PEM lib')]
ipdb>

[kevgliss]

I think you want readlines() not read() as you actually want the whole file read.

[kevgliss]

You will likely have to re-create the authority as it probably wasn't set correctly.

[JohnTheodore]

readlines puts it into an array. and the cryptography library that parse_certificate runs wants it to be a single string blob I think?. There were errors creating the authority when parse_certificates had the input as an array of strings, versus just one string for the body. read() does read the whole file as a single string.

This is the function that I think doesn't want an array of strings: https://github.com/Netflix/lemur/blob/master/lemur/common/utils.py#L54 https://cryptography.io/en/latest/x509/reference/#loading-certificates

[JohnTheodore]

I changed it to readlines(), and made a new authority, here is the error from that:

[2018-05-24 18:49:00,591] ERROR in schema: initializer for ctype 'char' must be a bytes of length 1, not str
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/lemur/common/schema.py", line 160, in decorated_function
    resp = f(*args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/lemur/authorities/views.py", line 223, in post
    return service.create(**data)
  File "/usr/local/lib/python3.5/dist-packages/lemur/authorities/service.py", line 108, in create
    cert = upload(**kwargs)
  File "/usr/local/lib/python3.5/dist-packages/lemur/certificates/service.py", line 235, in upload
    cert = Certificate(**kwargs)
  File "<string>", line 4, in __init__
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/orm/state.py", line 417, in _initialize_instance
    manager.dispatch.init_failure(self, args, kwargs)
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__
    compat.reraise(exc_type, exc_value, exc_tb)
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/util/compat.py", line 187, in reraise
    raise value
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/orm/state.py", line 414, in _initialize_instance
    return manager.original_init(*mixed[1:], **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/lemur/certificates/models.py", line 144, in __init__
    cert = lemur.common.utils.parse_certificate(kwargs['body'])
  File "/usr/local/lib/python3.5/dist-packages/lemur/common/utils.py", line 54, in parse_certificate
    return x509.load_pem_x509_certificate(body, default_backend())
  File "/usr/local/lib/python3.5/dist-packages/cryptography/x509/base.py", line 43, in load_pem_x509_certificate
    return backend.load_pem_x509_certificate(data)
  File "/usr/local/lib/python3.5/dist-packages/cryptography/hazmat/backends/openssl/backend.py", line 1132, in load_pem_x509_certificate
    mem_bio = self._bytes_to_bio(data)
  File "/usr/local/lib/python3.5/dist-packages/cryptography/hazmat/backends/openssl/backend.py", line 436, in _bytes_to_bio
    data_char_p = self._ffi.new("char[]", data)
TypeError: initializer for ctype 'char' must be a bytes of length 1, not str
initializer for ctype 'char' must be a bytes of length 1, not str
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/lemur/common/schema.py", line 160, in decorated_function
    resp = f(*args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/lemur/authorities/views.py", line 223, in post
    return service.create(**data)
  File "/usr/local/lib/python3.5/dist-packages/lemur/authorities/service.py", line 108, in create
    cert = upload(**kwargs)
  File "/usr/local/lib/python3.5/dist-packages/lemur/certificates/service.py", line 235, in upload
    cert = Certificate(**kwargs)
  File "<string>", line 4, in __init__
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/orm/state.py", line 417, in _initialize_instance
    manager.dispatch.init_failure(self, args, kwargs)
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__
    compat.reraise(exc_type, exc_value, exc_tb)
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/util/compat.py", line 187, in reraise
    raise value
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/orm/state.py", line 414, in _initialize_instance
    return manager.original_init(*mixed[1:], **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/lemur/certificates/models.py", line 144, in __init__
    cert = lemur.common.utils.parse_certificate(kwargs['body'])
  File "/usr/local/lib/python3.5/dist-packages/lemur/common/utils.py", line 54, in parse_certificate
    return x509.load_pem_x509_certificate(body, default_backend())
  File "/usr/local/lib/python3.5/dist-packages/cryptography/x509/base.py", line 43, in load_pem_x509_certificate
    return backend.load_pem_x509_certificate(data)
  File "/usr/local/lib/python3.5/dist-packages/cryptography/hazmat/backends/openssl/backend.py", line 1132, in load_pem_x509_certificate
    mem_bio = self._bytes_to_bio(data)
  File "/usr/local/lib/python3.5/dist-packages/cryptography/hazmat/backends/openssl/backend.py", line 436, in _bytes_to_bio
    data_char_p = self._ffi.new("char[]", data)
TypeError: initializer for ctype 'char' must be a bytes of length 1, not str

[JohnTheodore]

So the api credential (certificate) I got from Symantec (digicert) is a p7b file. The issue was the p7b needs to be in pem format. I can make a certificate now!

[JohnTheodore]

Is there a way I can force all certificates to never expires more than say 2 months after today's date?

also are there little scripts for renewing a cert for say apache, tomcat, and other services as cronjobs?

[JohnTheodore]

also what is the command to sync all the certs from mpki to lemur? meaning the older certs

[JohnTheodore]

hmm, if wanted to make API keys for say somesub.domain.com, and then have a script run that renews that cert every week say. Do I have to make a role, user, and API key each time I want to do that? the goal is to make all certificate renewals automated, run by cronjobs or something similar.