ksqlDB java client giving error while executing pull/push queries at '/query-stream' api endpoint - "Received 400 response from server: This endpoint is only available when using HTTP2. Error code: 40004"

[kumar-subashp]

Describe the bug

ksqlDB java client giving error while executing pull/push queries at '/query-stream' api endpoint.

Error: Received 400 response from server: This endpoint is only available when using HTTP2. Error code: 40004

Also tried through CURL command, hitting with same error

Command:

curl -X "POST" "https://ksql.k8s.com:443/query-stream" --http2 -H "Content-Type: application/json; charset=utf-8" -d $'{ "sql": "SELECT * FROM EVENTS EMIT CHANGES;", "streamsProperties": {"ksql.streams.auto.offset.reset": "earliest"} }' | jq . 

CURL output :

{
  "@type": "generic_error",
  "error_code": 40004,
  "message": "This endpoint is only available when using HTTP2"
}  

To Reproduce

Steps to reproduce the behavior, include:

1. The version of KSQL: 0.12.0
2. JDK: Zulu 11
3. Sample source data:

public class KSQLClient {
    protected static final Logger log = LoggerFactory.getLogger(KSQLClient.class);

    public static void main(String[] args) throws ExecutionException, InterruptedException, MalformedURLException, URISyntaxException {
        URI uri = new URL("https://ksql-server:443/").toURI();
        ClientOptions options = ClientOptions.create()
                .setHost(uri.getHost())
                .setPort(uri.getPort())
                .setUseTls(true)
                .setUseAlpn(true)
                .setVerifyHost(false);

        Vertx vertx = Vertx.vertx().exceptionHandler((t) -> {
            log.error("Unhandled exception in Vert.x", t);
        });
        vertx.exceptionHandler(t -> System.out.println("Unhandled exception in Vert.x" + t));

        Client client = Client.create(options, vertx);
        Map<String, Object> properties = Collections.singletonMap("auto.offset.reset", "earliest");
        StreamedQueryResult streamedQueryResult = client.streamQuery(
                "SELECT * FROM EVENTS EMIT CHANGES;", properties).get();
        for (int i = 0; i < 10; i++) {
            Row row = streamedQueryResult.poll();
            if (row != null) {
                System.out.println("Received a row!");
                System.out.println("Row: " + row.values());
            } else {
                System.out.println("Query has ended.");
            }
        }
        vertx.close();
        client.close();
    }
}

Expected behavior

'/query-stream' api endpoint should return the query result stream data.

However push/pull queries are working as expected through ksqldb-cli client.

[apurvam]

The team couldn't reproduce this.

[kumar-subashp]

Sorry for delay response, here are more details:

As "/query-stream" endpoint is only available when HTTP 2 is used, and the Nginx which we are using for reverse proxy in-front of ksqlDB looks like does not support http2 to the upstream services, i.e. it terminates the http2 at nginx and upstream is then invoked as http 1.1 (https://trac.nginx.org/nginx/ticket/923), in that case, how can we have Nginx proxy in front of ksqldb server ?

Our ksqlDB server setup on k8s cluster, where we are facing problem:

• ksqlDB server container running inside a pod and listeners listen to https.
• a service created to communicating to pod.
• an nginx-ingress controller which is reverse proxy in front of ksqlDB server, and TLS enabled at ingress level.

even same problem will persist without using k8s cluster.

[kumar-subashp]

Team, please help to resolve this issue.

[IrvicRodriguez]

So I ran into the same issue.

I was following the tutorial https://docs.confluent.io/cloud/current/cp-component/ksqldb-ccloud-cli.html and when I run

curl -X "POST" "https://cluster info:443/query-stream" \
     --basic --user "key:private key" \
      -d $'{
   "sql": "SELECT * FROM 'table' EMIT CHANGES;",
   "streamsProperties": {}
 }'

I get: "@type":"generic_error","error_code":40004,"message":"This endpoint is only available when using HTTP2"} I am using the ccloud CLI from confluent. Don't know what else to do to make it work.

[pkgonan]

I have same issue.

[agavra]

hey folk - sorry this ticket slipped through the cracks, we'll take another look at it and see if we can reproduce on our end

[nateab]

Sorry for delay response, here are more details:

As "/query-stream" endpoint is only available when HTTP 2 is used, and the Nginx which we are using for reverse proxy in-front of ksqlDB looks like does not support http2 to the upstream services, i.e. it terminates the http2 at nginx and upstream is then invoked as http 1.1 (https://trac.nginx.org/nginx/ticket/923), in that case, how can we have Nginx proxy in front of ksqldb server ?

Our ksqlDB server setup on k8s cluster, where we are facing problem:

• ksqlDB server container running inside a pod and listeners listen to https.
• a service created to communicating to pod.
• an nginx-ingress controller which is reverse proxy in front of ksqlDB server, and TLS enabled at ingress level.

even same problem will persist without using k8s cluster.

Sorry @kumar-subashp but this is a problem from the Nginx side. The endpoint works as intended from our end, and only supports http2 currently at the moment unfortunately.

[nateab]

I have same issue.

Can you paste the commands you used? And perhaps some more context to help debug

[nateab]

So I ran into the same issue.

I was following the tutorial https://docs.confluent.io/cloud/current/cp-component/ksqldb-ccloud-cli.html and when I run

curl -X "POST" "https://cluster info:443/query-stream" \
     --basic --user "key:private key" \
      -d $'{
   "sql": "SELECT * FROM 'table' EMIT CHANGES;",
   "streamsProperties": {}
 }'

I get: "@type":"generic_error","error_code":40004,"message":"This endpoint is only available when using HTTP2"} I am using the ccloud CLI from confluent. Don't know what else to do to make it work.

@IrvicRodriguez I got this curl command to work by adding the --http2 at the end