Couldn't create certificate PEM routines:PEM_read_bio:no start line:pem_lib.c:703:Expecting: TRUSTED CERTIFICATE

[metal3d]

Hi, We created account_key via dehydrated:

./dehydrated --register --accept-terms

That created our account_key RSA file (4096 bits) The we added this key as secret.

Afterward, we've launched your application and we can see in logs:

watching routes with selector butter.sh/letsencrypt-managed=yes
Processing route /oapi/v1/namespaces/XXX/XXX/XXX with domain api.XXXX.fr.
unable to load certificate
139686420318112:error:0906D06C:PEM routines:PEM_read_bio:no start line:pem_lib.c:703:Expecting: TRUSTED CERTIFICATE
Getting new certificate for api.XXXXX.fr
Deleting well-known route.
Adding well-known route.

We checked rsa pem key and we have good headers:

-----BEGIN RSA PRIVATE KEY-----
...
-----END RSA PRIVATE KEY-----

We've checked in Pod if the RSA key is well monted, and it's ok.

At this time, we're not able to find why it won't work, and how to fix it.

Any help will be appreciated.

Thanks

[metal3d]

EDIT:

After a while, a new line appears:

...
Adding well-known route.
Route letsencrypt-api.XXXXXX.fr not admitted.

[ibotty]

The first problem is regarding the certificate in your api.XXXX.fr route. I suspect that's no pem formated certificate. Can you check please.

> oc rsh letsencrypt-...
sh-4.2$ . /usr/share/letsencrypt-container/common.sh
sh-4.2$ api_call /oapi/v1/namespaces/XXX/routes/XXXX | jq -r '.spec.tls.certificate' > /tmp/existing-cert
sh-4.2$ openssl x509 -in /tmp/existing-cert -noout -dates

That should tell you, if the certificate is valid. I assume it won't be. If it is not, try to delete the certificate from your route and try again. You can do that without watching by doing, in the same shell:

sh-4.2$ get_certificate api.XXX.fr /oapi/v1/namespaces/XXX/routes/XXX

Regarding the second problem (I noted the hostname is different), I assume that DNS does not point to your openshift router, or the hostname is already used in a different name space. Try to create a route with that hostname to any other service in that namespace and check whether that works.

[metal3d]

Thanks for your help. I tried:

$ . /usr/share/letsencrypt-container/common.sh
$ api_call /oapi/v1/namespaces/chaine-iot/routes/kong-api | jq -r '.spec.tls.certificate'
null

[metal3d]

To precise:

api_call /oapi/v1/namespaces/chaine-iot/routes/kong-api
{
  "kind": "Route",
  "apiVersion": "v1",
  "metadata": {
    "name": "kong-api",
    "namespace": "chaine-iot",
    "selfLink": "/oapi/v1/namespaces/chaine-iot/routes/kong-api",
    "uid": "845e51f7-5593-11e7-8106-de196c321004",
    "resourceVersion": "1082871",
    "creationTimestamp": "2017-06-20T08:36:12Z",
    "labels": {
      "butter.sh/letsencrypt-managed": "yes"
    }
  },
  "spec": {
    "host": "api.XXXXXX.fr",
    "to": {
      "kind": "Service",
      "name": "kong-proxy",
      "weight": 100
    },
    "port": {
      "targetPort": "kong-proxy-ssl"
    },
    "wildcardPolicy": "None"
  },
  "status": {
    "ingress": [
      {
        "host": "api.XXXXXX.fr",
        "routerName": "router",
        "conditions": [
          {
            "type": "Admitted",
            "status": "True",
            "lastTransitionTime": "2017-06-20T08:36:12Z"
          }
        ],
        "wildcardPolicy": "None"
      }
    ]
  }
}

note: I replace our domain by XXXXXX of course

[ibotty]

I see, in that case that should not be the problem. I wonder why it even goes to that code part. Can you ls -hl your /var/lib/openshift-letsencrypt directory? I assume there is an invalid cert there.

[JeremyBesson]

Hi, I work on the same project as @metal3d and I list the /var/lib/letsencrypt-container/ with ls -hl Result is:

total 4.0K                                                                                                                                                                                           
drwxr-sr-x 2 root root 4.0K Jun 21 10:09 api.XXX.fr                                                                                                                                             
-rw-r--r-- 1 root root    0 Jun 21 12:26 get_certificate.pid  

[ibotty]

Sorry: I meant an ls -l in your api.XXX.fr directory.

[JeremyBesson]

Ok sorry,

This time the result is:

total 12                                                                                                                                                                                             
-rw-r--r-- 1 root root 5 Jun 21 10:09 crt                                                                                                                                                            
-rw-r--r-- 1 root root 5 Jun 21 10:09 fullchain                                                                                                                                                      
-rw-r--r-- 1 root root 5 Jun 21 10:09 key           

Something is missing ?

[ibotty]

That looks wrong. For reference: that's what it should look like:

sh-4.2$ ls -l /var/lib/letsencrypt-container/tungdojo.de/
total 12
-rw-r--r--. 1 1000110000 1000110000 3815 Jun 15 10:50 crt
-rw-r--r--. 1 1000110000 1000110000 3815 Jun 15 10:50 fullchain
-rw-r--r--. 1 1000110000 1000110000 3244 Jun 15 10:50 key

You can try deleting them and try get_certificate ... (see above).

[metal3d]

That's strange that you, @ibotty, have got UUID as if you didn't set anyuid scc to letsencrypt user. Something seems strange.

[metal3d]

@ibotty one more time, thanks to take time for us. This is what I tried:

sh-4.2# rm -f /var/lib/letsencrypt-container/api.XXX.fr/*
sh-4.2# get_certificate api.XXX.fr /oapi/v1/namespaces/chaine-iot/routes/kong-api                                                                                                                                          
unable to load certificate                                                                                                                                                                                                      
139856848533408:error:0906D06C:PEM routines:PEM_read_bio:no start line:pem_lib.c:703:Expecting: TRUSTED CERTIFICATE                                                                                                             
Getting new certificate for api.XXX.fr                                                                                                                                                                                     
Deleting well-known route.                                                                                                                                                                                                      
Adding well-known route.  

[JeremyBesson]

I think we made an error with users roles in another service that affect this one. I fix that and I tell you if the problem persist.

Thanks for your help.

[JeremyBesson]

Hi, it was effectively a user role issue in Openshift config. The template work well after all. Thanks again for your help.

[metal3d]

We can close that issue, thank @ibotty for your help. That was actually an error made on user role that were modified namespace wide.