[BUG] JWT Authentication Backend Fails to parse public keys in security-config.yml

trevorlyman commented 3 weeks ago

What is the bug? The jwt_auth_domain authentication backend fails to parse the signing_key value when the key is formatted in as a PEM as shown in the documentation. The key can be read if the PEM headers are stripped out and the base64 content of the key is provided as a single line.

How can one reproduce the bug? Steps to reproduce the behavior:

Configure a jwt_auth_domain authentication backend with the following key:

...
  jwt_auth_domain:
    description: "Authenticate via Json Web Token"
    http_enabled: true
    transport_enabled: true
    order: 0
    http_authenticator:
      type: jwt
      challenge: false
      config:
        signing_key: |-
          -----BEGIN PUBLIC KEY-----
          MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA64L6PDUm1ySNWIJiuxEv
          OA92MDmrevshZEWQsN98pkrp8pbCOVjA5OseddXOYVlXunbv1FMooGwVkOCAk6s3
          rxffHP2tN4NJRvvEDr/uoWKj22mV9cqyVar13HvdKkTm0MN1nDjo1fv1YBL5Xq7E
          2Ak6d0soKIg/RFzzv7JqLkUCsQ5krhm08A11cBbI8jqsebsiRCa7HkRyUalYLRVh
          NHf9Dxf3rmVoKtHs1jKRMm9NiAl1aa6U/BDzlM45uX9yNaOYiqNVbQIufwxiy4/a
          gbmkvuqTf/Pm6jzX09h0nb7CIiy+3AbbnncaZ8UjwKZ8iJJ8XTDtHuerM8eN728F
          NwIDAQAB
          -----END PUBLIC KEY-----
        jwt_header: "Authorization"
...

Reload the configuration if needed

$ cd /usr/share/opensearch/plugins/opensearch-security/tools
$ ./securityadmin.sh -icl -nhnv -cacert ../../../config/root-ca.pem -cert ../../../config/kirk.pem -key ../../../config/kirk-key.pem -cd /usr/share/opensearch/config/opensearch-security/

Observe the logs to see errors reading the public key

2024-06-05 10:54:13 [2024-06-05T16:54:13,013][ERROR][c.a.d.a.h.j.HTTPJwtAuthenticator] [opensearch-node1] Error while creating JWT authenticator
2024-06-05 10:54:13 java.lang.IllegalArgumentException: Illegal base64 character a
2024-06-05 10:54:13 at java.base/java.util.Base64$Decoder.decode0(Base64.java:852) ~[?:?]
2024-06-05 10:54:13 at java.base/java.util.Base64$Decoder.decode(Base64.java:570) ~[?:?]
2024-06-05 10:54:13 at java.base/java.util.Base64$Decoder.decode(Base64.java:593) ~[?:?]
2024-06-05 10:54:13 at org.opensearch.security.util.KeyUtils$1.run(KeyUtils.java:58) [opensearch-security-2.14.0.0.jar:2.14.0.0]
2024-06-05 10:54:13 at org.opensearch.security.util.KeyUtils$1.run(KeyUtils.java:45) [opensearch-security-2.14.0.0.jar:2.14.0.0]
2024-06-05 10:54:13 at java.base/java.security.AccessController.doPrivileged(AccessController.java:319) [?:?]
2024-06-05 10:54:13 at org.opensearch.security.util.KeyUtils.createJwtParserBuilderFromSigningKey(KeyUtils.java:45) [opensearch-security-2.14.0.0.jar:2.14.0.0]
2024-06-05 10:54:13 at com.amazon.dlic.auth.http.jwt.HTTPJwtAuthenticator.<init>(HTTPJwtAuthenticator.java:88) [opensearch-security-2.14.0.0.jar:2.14.0.0]
2024-06-05 10:54:13 at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:62) ~[?:?]
2024-06-05 10:54:13 at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:502) ~[?:?]
2024-06-05 10:54:13 at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:486) ~[?:?]
...

OpenSearch-key-load.log

What is the expected behavior? The key should be loaded correctly

What is your host/environment?

OS: Docker Container running on WSL2

Version

{
"name": "opensearch-node1",
"cluster_name": "opensearch-cluster",
"cluster_uuid": "uC4_Zj7vS5Cv2n87CIcQzA",
"version": {
    "distribution": "opensearch",
    "number": "2.14.0",
    "build_type": "tar",
    "build_hash": "aaa555453f4713d652b52436874e11ba258d8f03",
    "build_date": "2024-05-09T18:51:00.973564994Z",
    "build_snapshot": false,
    "lucene_version": "9.10.0",
    "minimum_wire_compatibility_version": "7.10.0",
    "minimum_index_compatibility_version": "7.0.0"
},
"tagline": "The OpenSearch Project: https://opensearch.org/"
}

Image opensearchproject/opensearch:sha256:756d2401537847f8bfb158a02a649a46adf7e7d15303a3692ed3d76586189d12

Do you have any additional context? This is a valid key. The code for parsing the public keys seems a bit buggy. The key loads correctly if it is formatted as a single line string without the PEM headers.

...
config:
            signing_key: "MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA64L6PDUm1ySNWIJiuxEvOA92MDmrevshZEWQsN98pkrp8pbCOVjA5OseddXOYVlXunbv1FMooGwVkOCAk6s3rxffHP2tN4NJRvvEDr/uoWKj22mV9cqyVar13HvdKkTm0MN1nDjo1fv1YBL5Xq7E2Ak6d0soKIg/RFzzv7JqLkUCsQ5krhm08A11cBbI8jqsebsiRCa7HkRyUalYLRVhNHf9Dxf3rmVoKtHs1jKRMm9NiAl1aa6U/BDzlM45uX9yNaOYiqNVbQIufwxiy4/agbmkvuqTf/Pm6jzX09h0nb7CIiy+3AbbnncaZ8UjwKZ8iJJ8XTDtHuerM8eN728FNwIDAQAB"
            jwt_header: "Authorization"
...

Code in question The parsing of the public key looks to happen here.

Additional Concerns I'm also concerned that because of the way the parsing code works that a public key may get misinterpreted as a HMAC key if parsing of the public key fails. By immediately clearing the PEM headers when parsing important context from the user is lost. Consider a situation where a public key was slightly malformed and could not be parsed as RSA or ECC and got interpreted as an HMAC string.

scrawfor99 commented 3 weeks ago

[Triage] Hi @trevorlyman, thank you for filing this issue. Quickly looking at your configuration, the issue you're experiencing is likely caused by the format of your public key. You have

 |-
              -----BEGIN PUBLIC KEY-----
              MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA64L6PDUm1ySNWIJiuxEv
              OA92MDmrevshZEWQsN98pkrp8pbCOVjA5OseddXOYVlXunbv1FMooGwVkOCAk6s3
              rxffHP2tN4NJRvvEDr/uoWKj22mV9cqyVar13HvdKkTm0MN1nDjo1fv1YBL5Xq7E
              2Ak6d0soKIg/RFzzv7JqLkUCsQ5krhm08A11cBbI8jqsebsiRCa7HkRyUalYLRVh
              NHf9Dxf3rmVoKtHs1jKRMm9NiAl1aa6U/BDzlM45uX9yNaOYiqNVbQIufwxiy4/a
              gbmkvuqTf/Pm6jzX09h0nb7CIiy+3AbbnncaZ8UjwKZ8iJJ8XTDtHuerM8eN728F
              NwIDAQAB
              -----END PUBLIC KEY-----

Which has a leading |- and will cause problems.

That being said, you are welcome to open a PR to improve the key parsing functionality for the JWT auth backend.

trevorlyman commented 3 weeks ago

Thanks for looking at this bug.

With and without the - is something that I tested. Either way the key can't be parsed.

This config:

...
      jwt_auth_domain:
        description: "Authenticate via Json Web Token"
        http_enabled: true
        transport_enabled: true
        order: 0
        http_authenticator:
          type: jwt
          challenge: false
          config:
            signing_key: |
              -----BEGIN PUBLIC KEY-----
              MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA64L6PDUm1ySNWIJiuxEv
              OA92MDmrevshZEWQsN98pkrp8pbCOVjA5OseddXOYVlXunbv1FMooGwVkOCAk6s3
              rxffHP2tN4NJRvvEDr/uoWKj22mV9cqyVar13HvdKkTm0MN1nDjo1fv1YBL5Xq7E
              2Ak6d0soKIg/RFzzv7JqLkUCsQ5krhm08A11cBbI8jqsebsiRCa7HkRyUalYLRVh
              NHf9Dxf3rmVoKtHs1jKRMm9NiAl1aa6U/BDzlM45uX9yNaOYiqNVbQIufwxiy4/a
              gbmkvuqTf/Pm6jzX09h0nb7CIiy+3AbbnncaZ8UjwKZ8iJJ8XTDtHuerM8eN728F
              NwIDAQAB
              -----END PUBLIC KEY-----
            jwt_header: "Authorization"
...

Produces this error message:

2024-06-10 11:05:03 [2024-06-10T17:05:03,918][ERROR][c.a.d.a.h.j.HTTPJwtAuthenticator] [opensearch-node1] Error while creating JWT authenticator
2024-06-10 11:05:03 java.lang.IllegalArgumentException: Illegal base64 character a
2024-06-10 11:05:03 at java.base/java.util.Base64$Decoder.decode0(Base64.java:852) ~[?:?]
2024-06-10 11:05:03 at java.base/java.util.Base64$Decoder.decode(Base64.java:570) ~[?:?]
2024-06-10 11:05:03 at java.base/java.util.Base64$Decoder.decode(Base64.java:593) ~[?:?]
2024-06-10 11:05:03 at org.opensearch.security.util.KeyUtils$1.run(KeyUtils.java:58) [opensearch-security-2.14.0.0.jar:2.14.0.0]
2024-06-10 11:05:03 at org.opensearch.security.util.KeyUtils$1.run(KeyUtils.java:45) [opensearch-security-2.14.0.0.jar:2.14.0.0]
2024-06-10 11:05:03 at java.base/java.security.AccessController.doPrivileged(AccessController.java:319) [?:?]
2024-06-10 11:05:03 at org.opensearch.security.util.KeyUtils.createJwtParserBuilderFromSigningKey(KeyUtils.java:45) [opensearch-security-2.14.0.0.jar:2.14.0.0]
2024-06-10 11:05:03 at com.amazon.dlic.auth.http.jwt.HTTPJwtAuthenticator.<init>(HTTPJwtAuthenticator.java:88) [opensearch-security-2.14.0.0.jar:2.14.0.0]
2024-06-10 11:05:03 at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:62) ~[?:?]
2024-06-10 11:05:03 at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:502) ~[?:?]
2024-06-10 11:05:03 at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:486) ~[?:?]
2024-06-10 11:05:03 at org.opensearch.security.support.ReflectionHelper.instantiateAAA(ReflectionHelper.java:62) [opensearch-security-2.14.0.0.jar:2.14.0.0]
2024-06-10 11:05:03 at org.opensearch.security.securityconf.DynamicConfigModelV7.lambda$newInstance$1(DynamicConfigModelV7.java:432) [opensearch-security-2.14.0.0.jar:2.14.0.0]
2024-06-10 11:05:03 at java.base/java.security.AccessController.doPrivileged(AccessController.java:319) [?:?]
2024-06-10 11:05:03 at org.opensearch.security.securityconf.DynamicConfigModelV7.newInstance(DynamicConfigModelV7.java:430) [opensearch-security-2.14.0.0.jar:2.14.0.0]
2024-06-10 11:05:03 at org.opensearch.security.securityconf.DynamicConfigModelV7.buildAAA(DynamicConfigModelV7.java:329) [opensearch-security-2.14.0.0.jar:2.14.0.0]
2024-06-10 11:05:03 at org.opensearch.security.securityconf.DynamicConfigModelV7.<init>(DynamicConfigModelV7.java:102) [opensearch-security-2.14.0.0.jar:2.14.0.0]
2024-06-10 11:05:03 at org.opensearch.security.securityconf.DynamicConfigFactory.onChange(DynamicConfigFactory.java:285) [opensearch-security-2.14.0.0.jar:2.14.0.0]
2024-06-10 11:05:03 at org.opensearch.security.configuration.ConfigurationRepository.notifyAboutChanges(ConfigurationRepository.java:570) [opensearch-security-2.14.0.0.jar:2.14.0.0]
2024-06-10 11:05:03 at org.opensearch.security.configuration.ConfigurationRepository.notifyConfigurationListeners(ConfigurationRepository.java:559) [opensearch-security-2.14.0.0.jar:2.14.0.0]
2024-06-10 11:05:03 at org.opensearch.security.configuration.ConfigurationRepository.reloadConfiguration0(ConfigurationRepository.java:554) [opensearch-security-2.14.0.0.jar:2.14.0.0]
2024-06-10 11:05:03 at org.opensearch.security.configuration.ConfigurationRepository.loadConfigurationWithLock(ConfigurationRepository.java:538) [opensearch-security-2.14.0.0.jar:2.14.0.0]
2024-06-10 11:05:03 at org.opensearch.security.configuration.ConfigurationRepository.reloadConfiguration(ConfigurationRepository.java:531) [opensearch-security-2.14.0.0.jar:2.14.0.0]
2024-06-10 11:05:03 at org.opensearch.security.configuration.ConfigurationRepository.reloadConfiguration(ConfigurationRepository.java:522) [opensearch-security-2.14.0.0.jar:2.14.0.0]
2024-06-10 11:05:03 at org.opensearch.security.action.configupdate.TransportConfigUpdateAction.nodeOperation(TransportConfigUpdateAction.java:128) [opensearch-security-2.14.0.0.jar:2.14.0.0]
2024-06-10 11:05:03 at org.opensearch.security.action.configupdate.TransportConfigUpdateAction.nodeOperation(TransportConfigUpdateAction.java:52) [opensearch-security-2.14.0.0.jar:2.14.0.0]
2024-06-10 11:05:03 at org.opensearch.action.support.nodes.TransportNodesAction.nodeOperation(TransportNodesAction.java:200) [opensearch-2.14.0.jar:2.14.0]
2024-06-10 11:05:03 at org.opensearch.action.support.nodes.TransportNodesAction$NodeTransportHandler.messageReceived(TransportNodesAction.java:328) [opensearch-2.14.0.jar:2.14.0]
2024-06-10 11:05:03 at org.opensearch.action.support.nodes.TransportNodesAction$NodeTransportHandler.messageReceived(TransportNodesAction.java:324) [opensearch-2.14.0.jar:2.14.0]
2024-06-10 11:05:03 at org.opensearch.security.ssl.transport.SecuritySSLRequestHandler.messageReceivedDecorate(SecuritySSLRequestHandler.java:207) [opensearch-security-2.14.0.0.jar:2.14.0.0]
2024-06-10 11:05:03 at org.opensearch.security.transport.SecurityRequestHandler.messageReceivedDecorate(SecurityRequestHandler.java:211) [opensearch-security-2.14.0.0.jar:2.14.0.0]
2024-06-10 11:05:03 at org.opensearch.security.ssl.transport.SecuritySSLRequestHandler.messageReceived(SecuritySSLRequestHandler.java:106) [opensearch-security-2.14.0.0.jar:2.14.0.0]
2024-06-10 11:05:03 at org.opensearch.security.OpenSearchSecurityPlugin$6$1.messageReceived(OpenSearchSecurityPlugin.java:841) [opensearch-security-2.14.0.0.jar:2.14.0.0]
2024-06-10 11:05:03 at org.opensearch.indexmanagement.rollup.interceptor.RollupInterceptor$interceptHandler$1.messageReceived(RollupInterceptor.kt:114) [opensearch-index-management-2.14.0.0.jar:2.14.0.0]
2024-06-10 11:05:03 at org.opensearch.performanceanalyzer.transport.PerformanceAnalyzerTransportRequestHandler.messageReceived(PerformanceAnalyzerTransportRequestHandler.java:43) [opensearch-performance-analyzer-2.14.0.0.jar:2.14.0.0]
2024-06-10 11:05:03 at org.opensearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:108) [opensearch-2.14.0.jar:2.14.0]
2024-06-10 11:05:03 at org.opensearch.transport.TransportService$7.doRun(TransportService.java:1059) [opensearch-2.14.0.jar:2.14.0]
2024-06-10 11:05:03 at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:913) [opensearch-2.14.0.jar:2.14.0]
2024-06-10 11:05:03 at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52) [opensearch-2.14.0.jar:2.14.0]
2024-06-10 11:05:03 at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) [?:?]
2024-06-10 11:05:03 at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) [?:?]
2024-06-10 11:05:03 at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]
2024-06-10 11:05:03 [2024-06-10T17:05:03,922][WARN ][o.o.s.s.ReflectionHelper ] [opensearch-node1] Unable to enable 'com.amazon.dlic.auth.http.jwt.HTTPJwtAuthenticator' due to java.lang.reflect.InvocationTargetException
2024-06-10 11:05:03 [2024-06-10T17:05:03,926][ERROR][o.o.s.s.DynamicConfigModelV7] [opensearch-node1] Unable to initialize auth domain jwt_auth_domain=AuthcDomain [http_enabled=true, order=0, http_authenticator=HttpAuthenticator [challenge=false, type=jwt, config={signing_key=-----BEGIN PUBLIC KEY-----
2024-06-10 11:05:03 MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA64L6PDUm1ySNWIJiuxEv
2024-06-10 11:05:03 OA92MDmrevshZEWQsN98pkrp8pbCOVjA5OseddXOYVlXunbv1FMooGwVkOCAk6s3
2024-06-10 11:05:03 rxffHP2tN4NJRvvEDr/uoWKj22mV9cqyVar13HvdKkTm0MN1nDjo1fv1YBL5Xq7E
2024-06-10 11:05:03 2Ak6d0soKIg/RFzzv7JqLkUCsQ5krhm08A11cBbI8jqsebsiRCa7HkRyUalYLRVh
2024-06-10 11:05:03 NHf9Dxf3rmVoKtHs1jKRMm9NiAl1aa6U/BDzlM45uX9yNaOYiqNVbQIufwxiy4/a
2024-06-10 11:05:03 gbmkvuqTf/Pm6jzX09h0nb7CIiy+3AbbnncaZ8UjwKZ8iJJ8XTDtHuerM8eN728F
2024-06-10 11:05:03 NwIDAQAB
2024-06-10 11:05:03 -----END PUBLIC KEY-----
2024-06-10 11:05:03 , jwt_header=Authorization, jwt_url_parameter=token, jwt_clock_skew_tolerance_seconds=30, roles_key=roles, subject_key=sub, required_audience=vcp-opensearch, required_issuer=vcp-token-service}], authentication_backend=AuthcBackend [type=noop, config={}], description=Authenticate via Json Web Token] due to OpenSearchException[java.lang.reflect.InvocationTargetException]; nested: InvocationTargetException; nested: OpenSearchSecurityException[java.lang.IllegalArgumentException: Illegal base64 character a];
2024-06-10 11:05:03 org.opensearch.OpenSearchException: java.lang.reflect.InvocationTargetException
...

Error_log_2.txt

If it's expected that users should use | instead of |- then the documentation should probably be looked at.

scrawfor99 commented 3 weeks ago

Hi @trevorlyman,

You can use this:

Feel free to open a PR changing the public key parsing logic and/or the documentation.

opensearch-project / security

[BUG] JWT Authentication Backend Fails to parse public keys in security-config.yml #4406