Timeout happens to do Azure Authentication in Elastic Search 8.15.0 Custom Plugin

[BuperS]

Hello,

I'm trying to authenticate in a Elastic Search custom plugin using the Azure Java SDK, but I get a timeout at credential.getToken().block() If I use credential.getTokenSync(), it gets blocked on this function for an long time and doesn't unblock.

with sudo tcpdump -i br-982d5f33bf1d port not 9200, There is no egress http request captured such as 22:03:11.717573 IP 172.18.0.2.41458 > <proxyserver>.http-alt: Flags [P.], seq 1:88, ack 1, win 251, options [nop,nop,TS val 4269673029 ecr 2416468369], length 87: HTTP: CONNECT login.microsoftonline.com:443 HTTP/1.1

To validate my code, I created a simple Java application instead of an es plugin and executed it with gradlew run, which successfully authenticated.

Here is the code snippet that I used:

public class MyTokenFilter extends TokenFilter {
    @SuppressWarnings({"deprecation", "removal"})
    MyTokenFilter(TokenStream input) {
        super(input);

        AccessController.doPrivileged(
            (PrivilegedAction<Void>) () -> {
                try {
                    java.net.URL url = new java.net.URL("https://management.azure.com/.default");
                    java.net.HttpURLConnection conn = (java.net.HttpURLConnection)url.openConnection();
                    conn.setRequestMethod("GET");
                    int responseCode = conn.getResponseCode();
                    System.out.println(String.format("responseCode: %s", responseCode)); // 200 OK

                    String tenantId = System.getenv("AZURE_TENANT_ID");
                    String clientId = System.getenv("AZURE_CLIENT_ID");
                    String clientSecret = System.getenv("AZURE_CLIENT_SECRET");
                    TokenRequestContext tokenRequestContext = (new TokenRequestContext()).addScopes(new String[]{String.format("https://management.azure.com/.default")});

                    Configuration configuration = new Configuration()
                        .put("java.net.useSystemProxies", "true")
                        .put("http.proxyHost", "<proxyHost>")
                        .put("http.proxyPort", "<proxyPort>");

                    HttpClient httpClient = new NettyAsyncHttpClientBuilder()
                        .configuration(configuration)
                        // .proxy(new ProxyOptions(ProxyOptions.Type.HTTP, new InetSocketAddress(<proxyHost>, <proxyPort>)))
                        .build();

                    TokenCredential credential = new ClientSecretCredentialBuilder()
                        .tenantId(tenantId)
                        .clientId(clientId)
                        .clientSecret(clientSecret)
                        .httpClient(httpClient)
                        .build();

                    // AccessToken accessToken = credential.getTokenSync(tokenRequestContext); // blocked very long time
                    AccessToken accessToken = credential.getToken(tokenRequestContext).block(Duration.ofSeconds(10)); // timeout
                    System.out.println(String.format("token: %s", accessToken.getToken()));
                } catch (Exception e) {
                    e.printStackTrace(System.out);
                }
                return null;
            }
        );
    }
}

Since it works fine outside of Elastic Search, I suspect it's related to Elastic Search Security. I tried the following settings but didn't work.

• xpack.security.enabled: false
• Grant Permissions

  grant {
  permission java.net.SocketPermission "*", "connect,resolve";
  permission java.io.FilePermission "<<ALL FILES>>", "read";
  permission java.lang.RuntimePermission "getClassLoader";
  permission java.lang.reflect.ReflectPermission "suppressAccessChecks";
  permission java.lang.RuntimePermission "accessDeclaredMembers";
  permission java.net.NetPermission "getProxySelector";
  permission java.util.PropertyPermission "*", "write";
  permission java.net.URLPermission "scheme:*", "*";
  };

I tried to give permission AllPermision; but elasticsearch-plugin install prints "illegal permission ("java.security.AllPermission" "" "") in global grant"

If you have any other suggestions or if you've ever done Azure authentication with a Custom Plugin in ES version 8, please help me out.

For reference, we're using a proxy server, so I set up basic proxy server settings through environment variables and registered certificates using update-ca-certificates and java keytool.

### Tasks

[joshfree]

Hi @BuperS thanks for reaching out via this github issue. I'm not familiar with Elastic Search, and the best course of action for you may be to open an Azure support ticket to get routed to the right team - https://github.com/Azure/azure-sdk-for-java/blob/main/SUPPORT.md#azure-support-tickets; however @billwert from the Azure Identity SDK may have some other insights for you. @billwert could you take a look and share your thoughts?

[billwert]

Hi @BuperS - from an Identity standpoint what you have is correct. (As you'd expect as it works outside of ES.) Have you tried running with logging enabled? I am not particularly hopeful that will show anything interesting but it may! I agree with @joshfree that filing a support ticket is the correct next step.