search

apache / apisix-ingress-controller

APISIX Ingress Controller for Kubernetes
https://apisix.apache.org/
Apache License 2.0
972 stars 337 forks source link

bug: failed to decrypt previous encrypted key, status code 400 #2192

Open zt4123 opened 3 months ago

zt4123 commented 3 months ago

Current Behavior

I deployed apisix and apisix ingress controller on GCP kubernetes cluster. In pod log for apisix-ingress-controller, there are always errors about "failed to create ssl: unexpected status code 400; error message: {"error_msg":"failed to decrypt previous encrypted key"}"

Expected Behavior

No such errors.

Error Logs

2024-03-22T01:54:46+08:00 error apisix/ssl.go:139 failed to create ssl: unexpected status code 400; error message: {"error_msg":"failed to decrypt previous encrypted key"}

2024-03-22T01:54:46+08:00 error apisix/apisix_tls.go:179 failed to sync SSL to APISIX {"error": "unexpected status code 400; error message: {\"error_msg\":\"failed to decrypt previous encrypted key\"}\n", "errorCauses": [{"error": "unexpected status code 400"}, {"error": "error message: {\"error_msg\":\"failed to decrypt previous encrypted key\"}\n"}], "ssl": {"id":"8db0ab63","snis":["gke-sea1-pragma-dev-apisix-dashboard.concentrix.com"],"cert":"-----BEGIN CERTIFICATE-----\r\nMIIFeTCCBP6gAwIBAgIQDAD9d20jevNIsWSOM3QKtjAKBggqhkjOPQQDAzBWMQsw\r\nCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMTAwLgYDVQQDEydEaWdp\r\nQ2VydCBUTFMgSHlicmlkIEVDQyBTSEEzODQgMjAyMCBDQTEwHhcNMjMwNjA2MDAw\r\nMDAwWhcNMjQwNzA1MjM1OTU5WjBwMQswCQYDVQQGEwJVUzETMBEGA1UECBMKQ2Fs\r\naWZvcm5pYTEQMA4GA1UEBxMHRnJlbW9udDEfMB0GA1UEChMWQ29uY2VudHJpeCBD\r\nb3Jwb3JhdGlvbjEZMBcGA1UEAwwQKi5jb25jZW50cml4LmNvbTBZMBMGByqGSM49\r\nAgEGCCqGSM49AwEHA0IABLdwc14ZsyTqHeAWrBksbuqqUpOHTNsRl0ZReJvLquVb\r\ndIlxCTDkKWWBCDCe8kC9fsYR5r2vGj3TWXtwJWsPlNKjggOSMIIDjjAfBgNVHSME\r\nGDAWgBQKvAgpF4ylOW16Ds4zxy6z7fvDejAdBgNVHQ4EFgQUcSI3ZsxDkoOxSu16\r\nhTW7tZNMyNIwKwYDVR0RBCQwIoIQKi5jb25jZW50cml4LmNvbYIOY29uY2VudHJp\r\neC5jb20wDgYDVR0PAQH/BAQDAgeAMB0GA1UdJQQWMBQGCCsGAQUFBwMBBggrBgEF\r\nBQcDAjCBmwYDVR0fBIGTMIGQMEagRKBChkBodHRwOi8vY3JsMy5kaWdpY2VydC5j\r\nb20vRGlnaUNlcnRUTFNIeWJyaWRFQ0NTSEEzODQyMDIwQ0ExLTEuY3JsMEagRKBC\r\nhkBodHRwOi8vY3JsNC5kaWdpY2VydC5jb20vRGlnaUNlcnRUTFNIeWJyaWRFQ0NT\r\nSEEzODQyMDIwQ0ExLTEuY3JsMD4GA1UdIAQ3MDUwMwYGZ4EMAQICMCkwJwYIKwYB\r\nBQUHAgEWG2h0dHA6Ly93d3cuZGlnaWNlcnQuY29tL0NQUzCBhQYIKwYBBQUHAQEE\r\neTB3MCQGCCsGAQUFBzABhhhodHRwOi8vb2NzcC5kaWdpY2VydC5jb20wTwYIKwYB\r\nBQUHMAKGQ2h0dHA6Ly9jYWNlcnRzLmRpZ2ljZXJ0LmNvbS9EaWdpQ2VydFRMU0h5\r\nYnJpZEVDQ1NIQTM4NDIwMjBDQTEtMS5jcnQwCQYDVR0TBAIwADCCAX0GCisGAQQB\r\n1nkCBAIEggFtBIIBaQFnAHcA7s3QZNXbGs7FXLedtM0TojKHRny87N7DUUhZRnEf\r\ntZsAAAGIj+50AAAABAMASDBGAiEAiPVe7X9Fgw6x+A5xb+xXKKrxiEHHRMCrsndI\r\nxrpzVUUCIQC+54rTQryylaHCWgDtXap3N0XUYfCmMWzJWrFwCE5KfwB1AEiw42va\r\npkc0D+VqAvqdMOscUgHLVt0sgdm7v6s52IRzAAABiI/udCwAAAQDAEYwRAIgXrXf\r\n+lyTEp+BxDvqSYgOLogRqTwZLjnUl3xpkkhD6dUCIDo7Fgx90AgdYQHGfSyYW5ue\r\nGmnbtn8WWazf6MmX0eaFAHUA2ra/az+1tiKfm8K7XGvocJFxbLtRhIU0vaQ9MEjX\r\n+6sAAAGIj+5z6QAABAMARjBEAiBiQ2aRojIFTGKtEh1LaE7u//XYoW7hPCSsVMKN\r\nhel2WQIgMS+r70gKodGSohlc/zLIArKukObwV2tkmTcXtJdzZigwCgYIKoZIzj0E\r\nAwMDaQAwZgIxAPKpY9qB+WzjowQT+S065L7wuiNgA2y5THh892oVKeMz/UJm94aM\r\nF0AGTRb6wTpVLQIxALQP5QisAeSVfpqWAbKmX6XgxeLn6fKGYg4VgYRDbDPCMSe2\r\nbDsIKBes7Cu1KB4ebQ==\r\n-----END CERTIFICATE-----\r\n","key":"Bag Attributes\r\n Microsoft Local Key set: \r\n localKeyID: 01 00 00 00 \r\n friendlyName: te-2d33dfef-2403-4eb5-9dfb-a25900162c4c\r\n Microsoft CSP Name: Microsoft Software Key Storage Provider\r\nKey Attributes\r\n X509v3 Key Usage: 80 \r\n-----BEGIN PRIVATE KEY-----\r\nMIGHAgEAMBMGByqGSM49AgEGCCqGSM49AwEHBG0wawIBAQQgfehuLux8Peq8nE/j\r\nLmmU09MMF8dvKgcPM3ScYxCp1zKhRANCAAS3cHNeGbMk6h3gFqwZLG7qqlKTh0zb\r\nEZdGUXiby6rlW3SJcQkw5CllgQgwnvJAvX7GEea9rxo901l7cCVrD5TS\r\n-----END PRIVATE KEY-----\r\n","status":1,"labels":{"managed-by":"apisix-ingress-controller","meta_secret_name":"concentrix-com","meta_secret_namespace":"ingress-apisix"}}} 2024-03-22T01:54:46+08:00 warn apisix/apisix_tls.go:279 sync ApisixTls failed, will retry {"object": {"Type":4,"Object":{"Key":"ingress-apisix/apisix-dashboard","OldObject":null,"GroupVersion":"apisix.apache.org/v2"},"OldObject":null,"Tombstone":null}, "error": "unexpected status code 400; error message: {\"error_msg\":\"failed to decrypt previous encrypted key\"}\n", "errorCauses": [{"error": "unexpected status code 400"}, {"error": "error message: {\"error_msg\":\"failed to decrypt previous encrypted key\"}\n"}]}

Steps to Reproduce

  1. Deploy apisix and apisix-ingress-controller on GCP k8s by using helm chart apisix "2.6.0" and apisix-ingress-controller "0.14.0"
  2. run kubectl logs -f or run kubectl describe apisixtls apisix-admin-api, will see errors.

Environment

sereneshikari commented 2 weeks ago

I'm facing a similar error with the current latest version of APISIX (v3.9.1). Please let me know if there's any information you'd like me to provide.

sereneshikari commented 2 weeks ago

Update: apologies, it turns out I had a misconfiguration. My APISIX control plane was on v3.9.1 and ingress controller was on v1.8.2 but my data plane was on v3.7.0. After moving my data plane to v3.9.1, this error disappeared. Wondering if it's related to https://github.com/apache/apisix/pull/10724 which was merged in v3.8.0?