Failed to set DER cert - Githubissues

Venthe commented 2 years ago

NGINX Ingress controller version (exec into the pod and run nginx-ingress-controller --version.):

NGINX Ingress controller Release: v1.1.1 Build: a17181e43ec85534a6fea968d95d019c5a4bc8cf Repository: https://github.com/kubernetes/ingress-nginx nginx version: nginx/1.19.9

Kubernetes version (use kubectl version): Server Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.3", GitCommit:"816c97ab8cff8a1c72eccca1026f7820e93e0d25", GitTreeState:"clean", BuildDate:"2022-01-25T21:19:12Z", GoVersion:"go1.17.6", Compiler:"gc", Platform:"linux/amd64"}

Environment:

Cloud provider or hardware configuration:
OS (e.g. from /etc/os-release): Debian GNU/Linux 11 (bullseye)
Kernel (e.g. uname -a): Linux truenas.local 5.10.70+truenas #1 SMP Mon Nov 22 21:32:24 UTC 2021 x86_64 GNU/Linux
Install tools:
- Please mention how/where was the cluster created like kubeadm/kops/minikube/kind etc.
- Manual installation following own baremetal scripts, see https://github.com/Venthe/Personal-Development-Pipeline/tree/develop/provisioning/cluster_vagrant/provisioning
Basic cluster related info:
- kubectl version Client Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.1", GitCommit:"86ec240af8cbd1b60bcc4c03c20da9b98005b92e", GitTreeState:"clean", BuildDate:"2021-12-16T11:41:01Z", GoVersion:"go1.17.5", Compiler:"gc", Platform:"linux/amd64"}
- kubectl get nodes -o wide truenas Ready control-plane,master 4d21h v1.23.3 192.168.0.105 Debian GNU/Linux 11 (bullseye) 5.10.70+truenas containerd://1.4.12
How was the ingress-nginx-controller installed:
- ingress ingress 1 2022-02-09 03:16:19.367640806 +0100 CET deployed ingress-nginx-4.0.17 1.1.1
```
USER-SUPPLIED VALUES:
controller:
service:
annotations:
external-dns.alpha.kubernetes.io/hostname: home.arpa
```
Others:
- Any other related information like ; Installed Cert-Manager cert-manager cert-manager 1 2022-02-09 03:17:10.314664758 +0100 CET deployed cert-manager-v1.7.1 v1.7.1

➜  ~ kubectl describe ingress/jenkins -n jenkins
Name:             jenkins
Labels:           app.kubernetes.io/component=jenkins-controller
                  app.kubernetes.io/instance=jenkins
                  app.kubernetes.io/managed-by=Helm
                  app.kubernetes.io/name=jenkins
                  helm.sh/chart=jenkins-3.11.4
Namespace:        jenkins
Address:          192.168.10.2
Default backend:  default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
TLS:
  jenkins-ingress-cert terminates jenkins.home.arpa
Rules:
  Host               Path  Backends
  ----               ----  --------
  jenkins.home.arpa
                        jenkins:80 (10.0.85.47:8080)
Annotations:         cert-manager.io/cluster-issuer: ca-issuer
                     kubernetes.io/ingress.class: nginx
                     meta.helm.sh/release-name: jenkins
                     meta.helm.sh/release-namespace: jenkins
Events:              <none>
➜  jacek curl https://jenkins.home.arpa -v
*   Trying 192.168.10.2:443...
* TCP_NODELAY set
* Connected to jenkins.home.arpa (192.168.10.2) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS alert, internal error (592):
* error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error
* Closing connection 0
curl: (35) error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error
➜  jacek kubectl logs -n ingress \
    `kubectl get pod -n ingress |grep nginx |awk '{print $1}'`
192.168.0.105 - - [11/Feb/2022:21:12:50 +0000] "GET / HTTP/1.1" 308 164 "-" "curl/7.68.0" 81 0.000 [jenkins-jenkins-80] [] - - - - 50f6b627bc07e35779d63e3d9c6ca3f6
2022/02/11 21:12:56 [error] 49#49: *5483907 [lua] certificate.lua:259: call(): failed to set DER cert: SSL_add0_chain_cert() failed, context: ssl_certificate_by_lua*, client: 192.168.0.105, server: 0.0.0.0:443
➜  ~ kubectl get secret/jenkins-ingress-cert -n jenkins -oyaml
apiVersion: v1
data:
  ca.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUMvakNDQWVhZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeU1ESXdOakl6TkRnd05Gb1hEVE15TURJd05ESXpORGd3TkZvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTHUwClVLTHlqUEwvMjMrNC8zOHhSTXkweE9xWTFQc1dZZXM1TmxSZis4WTF2RzNiYlQzWUk2bkdZcDU4cWJiK3dNOXUKZDQxelA4U0RnbUhBc21wZHhqcHNWa1lVY0lua0dDTS9Td3JRcGEwMzAxYWhKLytmL2pKSWV4cHhlN3ZXZHRJMApHT2VYaTMrbURyUmpvSTNsdUJ1eTI0TFFnZERJcUFmRFpzWlI0VXVSWTBvR3BTWkMxVUU1dWkyRk9JSXJ0T2VPCnljRjdBUE9USjc1UWNGb3AxY0sxMEtiOGRhVGtRYjk5NitNRkc4ZWI3TElTcXhUNVdmUzhYWW1nMGVYMW05RnkKbEptYTJNRXJOd0xvWFpQQU8yNWp5bFg1VFZ6anVBT000bTJEZHpiWFJHbm9pNk9wQzZ5TFdycGFvQnBZd0hpWQpIb1hEZlNmQ0wyem1DanN6OC8wQ0F3RUFBYU5aTUZjd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0hRWURWUjBPQkJZRUZDUkJ5alErcEVTOVIrb0oxSzIzS2Fvc0hxZ3BNQlVHQTFVZEVRUU8KTUF5Q0NtdDFZbVZ5Ym1WMFpYTXdEUVlKS29aSWh2Y05BUUVMQlFBRGdnRUJBRmxyNm81K2dOUTZLUlRSbW1YVwpFMXE4Y3ZDVWFtL1BuMnZUbjgxQjZVMjltZ2VaMVgwdWtxb3orTE4vclhORU96UWdtM251N25za0sxNFk2Y2hvCkY1ODJkVDA4bWZqak1lMnFyZ0Y2QTErOHBjeVpHVkx1eW8zV2RrVm5SdVZVU3BSODBrTnVPZ3B0VjdyRUtOWDQKNmtkcTM2MFBXOU9FcjhVUlFQbVl3d3ZHcmJDZitMdkM0UGF6TGFZVDNvaGRGc09sU2l0R3ZQMVN3TUpVclhxRQpYSUVoSUt0MElqa2JicTM0cUZWVmtaOThGVkJ5ZDVuZWJPQmVsNmhPcHMzNW9ML0R3YmtSY2ZYV1p2cUg3TWZrCkhqN1A4MExNbnQzcDZ4ZGUrUkhIUUFic1ZZaWFXNmJITmtNOEJ6bk9OZlBFdVUzYW9xVjI4TXcvNEVZWlFyVVUKemk4PQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
  tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN1akNDQW1TZ0F3SUJBZ0lRQko2dDZGREZrR0dFS0tYRXhUU0FTekFOQmdrcWhraUc5dzBCQVFzRkFEQ0IKanpFTE1Ba0dBMVVFQmhNQ1VFd3hFREFPQmdOVkJBZ01CMDFoZW05MmFXRXhFVEFQQmdOVkJBY01DRkJ5ZFhONgphMjkzTVEwd0N3WURWUVFLREFSSWIyMWxNU0l3SUFZRFZRUUREQmxvYjIxbExtRnljR0VnYVc1MFpYSnRaV1JwCllYUmxJRU5CTVNnd0pnWUpLb1pJaHZjTkFRa0JGaGxxWVdObGF5NXNhWEJwWldNdVltTkFaMjFoYVd3dVkyOXQKTUI0WERUSXlNREl4TURFeU1URXpOMW9YRFRJeU1EVXhNVEV5TVRFek4xb3dBRENDQVNJd0RRWUpLb1pJaHZjTgpBUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTTJvUEd1eENVbXRadXdUWTFQV2xjeG5KQ3lsYWJ5OWRUcmVmVzVXCjNRZVh6MTN0cmxFVFFzSUxWRm90TjBScXhCYWVqSE1OcnAvWXdUZXV1OG5SZEQxYnpCOWloeHF3cTdzK1pvblMKdk1nNmx3YmtDZkVuY1o3RDR2RDZLQmM0N29BdHQvWEFSM1NSc2xBNTU3aVZiOW84ellMNEF5WTVMenFFVEZvZwplQm1RQ3RObGFnSHBuTGFaS2tLQndEcDFSZkdFRUNhRVl2eU5GbXZHcVM0YXF3RmRsaW10MjR1N1h6Y3VRM0Z1Clp3Rm15bmhjZmp3aFB4dzRDUE40cG16VGprS0Y0Y0o4M29HQ3dtQTF0a0hQN1dsSjVNbUJNYzlZRVVHZlAzTlQKUDZKaXVHZllvajh2ZjN2cGRpRVJ1L1lHSzd4b2pqYWduZmtkUFhzdndFZXlwTXNDQXdFQUFhTmlNR0F3RGdZRApWUjBQQVFIL0JBUURBZ1dnTUF3R0ExVWRFd0VCL3dRQ01BQXdId1lEVlIwakJCZ3dGb0FVMlhqK2Z5UytrSFBBCnQ5VnFkSXk0T1JjRnF3NHdId1lEVlIwUkFRSC9CQlV3RTRJUmFtVnVhMmx1Y3k1b2IyMWxMbUZ5Y0dFd0RRWUoKS29aSWh2Y05BUUVMQlFBRFFRQ0lLdFlBQ3Y0b1B4TGZmTzh4cmtyaDNWMERHM1g5MjAxR2RWWHpFV1FDdzl3MwpxRytHOVNGZjRoZ2R4ek5OeitwbFIvTWV1cjBZOEhheTFYRm9uUzE1Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0KLS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUMwVENDQWJtZ0F3SUJBZ0lVUHg1MURHSFMyQ3Rsb0swRUVMME5QbHV5T1ZRd0RRWUpLb1pJaHZjTkFRRU4KQlFBd0ZURVRNQkVHQTFVRUF4TUthM1ZpWlhKdVpYUmxjekFlRncweU1qQXlNVEF4TVRRME1qSmFGdzB5TXpBMgpNalV4TVRRME1qSmFNSUdQTVFzd0NRWURWUVFHRXdKUVRERVFNQTRHQTFVRUNBd0hUV0Y2YjNacFlURVJNQThHCkExVUVCd3dJVUhKMWMzcHJiM2N4RFRBTEJnTlZCQW9NQkVodmJXVXhJakFnQmdOVkJBTU1HV2h2YldVdVlYSncKWVNCcGJuUmxjbTFsWkdsaGRHVWdRMEV4S0RBbUJna3Foa2lHOXcwQkNRRVdHV3BoWTJWckxteHBjR2xsWXk1aQpZMEJuYldGcGJDNWpiMjB3WERBTkJna3Foa2lHOXcwQkFRRUZBQU5MQURCSUFrRUEyWkU5NFhnS3pFeHcvVHdECjk2Y055TnhYTlZiTFB3NDc2SlFPcVhuNnN2V1ZQR0ZScVd5eFBRNGZtWUtYYkpic01EWWZsdHhlMHJZdWp4ZnkKS2ZzQm9RSURBUUFCbzJZd1pEQWRCZ05WSFE0RUZnUVUyWGorZnlTK2tIUEF0OVZxZEl5NE9SY0ZxdzR3SHdZRApWUjBqQkJnd0ZvQVVKRUhLTkQ2a1JMMUg2Z25VcmJjcHFpd2VxQ2t3RWdZRFZSMFRBUUgvQkFnd0JnRUIvd0lCCkFEQU9CZ05WSFE4QkFmOEVCQU1DQVlZd0RRWUpLb1pJaHZjTkFRRU5CUUFEZ2dFQkFIQXJ4UDRUZFE1TmZpanMKcnJ2UGp4MXhNREQvRzNueEkyNGFWTlY0NXBaVFY3L2FBT1dPbmx3R1JSVXVPUlNTQzNSY2VaYUc5ZVErZmFhKwphVktibmlOM2luWDYzbzl5TWx3SWE0Yk9xRVdxU1hTTzFtUjFpSGNac2JzQW8wZ2RKYzQ0ZXJaTlUrcjhEUGUzCjFGVmZTQUdJMmtGSThjK3ErcjF0enNpbEtkOUN6b1MrWmJtMG9XQ1pjeUpobVVQSkpidk5PMTFwSGFJa1dTNksKVmRiUmk5b3h6RzJ4L2gzZFh1VC92UWNNeTZRSEFhclRiOVhVdmE2bzFYMkZMbmFOemNuVlZ3TkNqSEQ0UlEzMQpFcHVEeFhhckdvMzFEYlQzVVoyNGNpcnBpUE1UcEVjRG9Ha0NSTExIS1o3bndYd0pTQzcvakpXcmRtaWdaRE1rCnZqWjVqU2s9Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
  tls.key: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFcEFJQkFBS0NBUUVBemFnOGE3RUpTYTFtN0JOalU5YVZ6R2NrTEtWcHZMMTFPdDU5YmxiZEI1ZlBYZTJ1ClVSTkN3Z3RVV2kwM1JHckVGcDZNY3cydW45akJONjY3eWRGMFBWdk1IMktIR3JDcnV6NW1pZEs4eURxWEJ1UUoKOFNkeG5zUGk4UG9vRnpqdWdDMjM5Y0JIZEpHeVVEbm51SlZ2Mmp6Tmd2Z0RKamt2T29STVdpQjRHWkFLMDJWcQpBZW1jdHBrcVFvSEFPblZGOFlRUUpvUmkvSTBXYThhcExocXJBVjJXS2EzYmk3dGZOeTVEY1c1bkFXYktlRngrClBDRS9IRGdJODNpbWJOT09Rb1hod256ZWdZTENZRFcyUWMvdGFVbmt5WUV4ejFnUlFaOC9jMU0vb21LNFo5aWkKUHk5L2UrbDJJUkc3OWdZcnZHaU9OcUNkK1IwOWV5L0FSN0treXdJREFRQUJBb0lCQVFDSE44QlNSbHFHK040ZApCanBlbTA1dTBMbjJLT1EyMXBMSG5CWDRnR0JmWjY5T1lNSW1TSFd5UStTNFRkWnl3NGdlSHl6VlVQK3AxY2hrCnBoeU9lNHJqOVFGbStVOC9lejNPcFI2aUs4ODREMTNSaUFvSlp4ZkxmWnd2T1NzMjZaWFc3VGh3K25XQTdYTEMKVVBCTWF1VGFKdlF3MG9Tdmt4VzhjZFFKeGlOeEdoaFV0SU5NSmNDWFhWMFdManpDYlhqOEhXNUVWTXZaZkg0Swp4RkdYTjF5UHlLTUJZWHpRL1VaSGtCdGdNZGk2Mjh2YWxxbytsYVl5OHh2cENWVW1qZktrY1lBV0ZJVkNjMjY4CnhibTlXMHpVdVkwcHZIb1VheFNBWXNrOHV2OENCcGtaUld1Q0xoMkZ6ZzFGUC9Wb0czb2ZlUmxMaVlXcFJaRmgKMXl0K1QxZ1JBb0dCQU9BNVJtTTF0aDEvaldIUFdWalgrcWRoNnhrSlpZSGpGRkkyQnpZOUV0cW9GSlpwa2UzQwpyaDJHTVFobjVTNlZabnQ4Rlh6SktFRDZzcEV4d0lWSE1WMnphd2M5SHJOWm5oYWk5OVp4WDhJQzlzRmk0Q09DClNWRS9kUWdpVDFCZ3loMzBvM1ZCMW45T0ZFWjh6bFVLU2RXNEhRSlU2NWg4UXQzZ1Jla1pjOFhYQW9HQkFPck4KWUNjQmtjRFlXNWVxSSs0N0U1ZldqaFE1OExkYklWYzRROFFaWFZlejgwenl5SW4vSUhCTXhmQUdFakVjTmJETAo4eU1GUVA1ZGNtcHBpT1doVk9hK3VqKzVMcDdNY2NhVVptL09taVhjekluWFBPMHJHb0lDdzJQNzh6RzdCWTQ1Clg5b0U3a0RzMUFuOXpLOGFzRU10a1BnUWdZRmdzbThPTGJEN2xsSXRBb0dCQUtEOW5xTTIrbVpkb2RhYmVPeEkKVG9rRnpqWXd2L2d0WWZiUklnWjRmV1FQa0pDYWhoTnZWM0tOV2dGT0Q0aU43b0kxQ255Sk5lVEoyNUpTWlllTQpJVTdFdHRFQXVsMU51Q3RLb2FZUHl4QlM3bmJWbGxjSWdibmJWUjBjL1ZPWjJ5VDg5S3dKTklnOU9HT1AwdG5PClJFczJJV1grTVJaUUNmYWl3NjlrL2ZJTEFvR0FiSHB4LzlOanlWcXNCa2ZlemNPUmE4YitZUzNibm5xRnpzdjUKMHZGOUJIWG81QllPdkdLWVZBbkVNUDQ5RFY5aEZzdERDMXowQ0JDWU9iMkVOOEx2ZktPRlhNZHJXSVpQMGxUaApkcXBXNFVNL1pEZUN1SGYvaVMxSnBEblV3NWNLSEFrNTFYb05zUVU1ZVZ3K1ltL01HMTFENnJlRmZZWjFLWVc5CmFsVHh3QWtDZ1lBT1YxdjBScVpUbXZVVXVrUmFyN1JTbmhhS3ZPalVrV2I4N2RtT211TEpsYlk4RUpCR2lMQ2QKMC83eCtLRHIxdmlSRUtXbTJpU1NaLzJEZGhOczJGM1ljZ1RkaXBZMUxxTnRUTlo0QTZoczB3ZGZ5NDd5SzRSNwo1OTdyazVWWVMzR3d2VWZxdEhZR3FwZzR0S1JWK3ZQVVNUeXpNL2NFb3grT3hRZmYrL2ZuWFE9PQotLS0tLUVORCBSU0EgUFJJVkFURSBLRVktLS0tLQo=
kind: Secret
metadata:
  annotations:
    cert-manager.io/alt-names: jenkins.home.arpa
    cert-manager.io/certificate-name: jenkins-ingress-cert
    cert-manager.io/common-name: ""
    cert-manager.io/ip-sans: ""
    cert-manager.io/issuer-group: cert-manager.io
    cert-manager.io/issuer-kind: ClusterIssuer
    cert-manager.io/issuer-name: ca-issuer
    cert-manager.io/uri-sans: ""
  creationTimestamp: "2022-02-10T12:11:37Z"
  name: jenkins-ingress-cert
  namespace: jenkins
  resourceVersion: "434085"
  uid: 06c3c303-347c-4345-8d07-4c3f6cc2810a
type: kubernetes.io/tls

What happened:

When supplied with certificates, ingress cannot serve them with error curl: (35) error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error

What you expected to happen:

Certificate chain is provided

k8s-ci-robot commented 2 years ago

@Venthe: This issue is currently awaiting triage.

If Ingress contributors determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.

Venthe commented 2 years ago

Certificates validated via openssl verify -CAfile ./ca.crt int.crt

CA cert is from the K8s cluster. Cluster is ephemeral as of now, so no harm in giving real certs

Venthe commented 2 years ago

As it turns out, the problem lied in the public key part of the intermediate CA being 512 bits -> when it should be at least 2048. In essence, default configuration hid the problem of SECLEVEL=2. To actually find the reason, I had to install ingress-nginx from the ingress itself, wherein I got 'ca key too small'

192.168.0.105 - - [11/Feb/2022:21:12:50 +0000] "GET / HTTP/1.1" 308 164 "-" "curl/7.68.0" 81 0.000 [jenkins-jenkins-80] [] - - - - 50f6b627bc07e35779d63e3d9c6ca3f6
2022/02/11 21:12:56 [error] 49#49: *5483907 [lua] certificate.lua:259: call(): failed to set DER cert: SSL_add0_chain_cert() failed, context: ssl_certificate_by_lua*, client: 192.168.0.105, server: 0.0.0.0:443

longwuyuan commented 2 years ago

/remove-kind bug Does your last message mean this is resolved. If yes, then consider closing the issue. Thank you for updating the resolution.

Venthe commented 2 years ago

Not really, because ingress-ngings/kubernetes is incapable of telling me what's the problem. Either way, closing

naofel1 commented 2 years ago

Hello @Venthe , I have exactly the same problem, do you have any update to fix this problem ?

2022/10/02 14:40:10 [error] 29#29: *542 [lua] certificate.lua:263: call(): failed to set DER private key: d2i_PrivateKey_bio() failed, context: ssl_certificate_by_lua*, client: 10.65.52.35, server: 0.0.0.0:443
24
2022/10/02 14:40:10 [alert] 22#22: worker process 29 exited on signal 7 (core dumped)

➜  ~ openssl s_client -connect XXX:443 -prexit
CONNECTED(00000005)
0071125DF87F0000:error:0A000438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error:ssl/record/rec_layer_s3.c:1584:SSL alert number 80
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 7 bytes and written 323 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 7 bytes and written 323 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---

Thank you!

Venthe commented 2 years ago

@naofel1 in my case it was cert that was encrypted below 2048. I've used alternative nginx ingress provider, and it has showed me this info in logs

Try cert with at least 2048 (or 4096?) Bits

E: see https://github.com/Venthe/Personal-Development-Pipeline/blob/single-file-provision/provision-kubernetes.yml#L1090

naofel1 commented 2 years ago

@Venthe Oh ok very interesting, I was using bitnami Nginx ingress controller. I just now try to switch to Nginx ingress from Kubernetes and the problem disappears. I think there is a problem with previous Nginx version.

Thank you, for your fast response!

kubernetes / ingress-nginx

Failed to set DER cert #8238

NGINX Ingress controller version (exec into the pod and run nginx-ingress-controller --version.):