search

RedPillAnalytics / gradle-confluent

Apache License 2.0
22 stars 10 forks source link

DESCRIBE stream/table throws error in Confluent Platform 5.5.0 #86

Closed terryf82 closed 4 years ago

terryf82 commented 4 years ago

Describe the bug DESCRIBE (and DESCRIBE EXTENDED) statements run against streams / tables created through the plugin throw an error similar to the following:

Failed to deserialise object
Caused by: Cannot construct instance of
    `io.confluent.ksql.rest.entity.RunningQuery`, problem: statusCount
 at [Source:
    (byte[])"[{"@type":"sourceDescription","statementText":"DESCRIBE
    KS_CAMPAIGN_SRC;","sourceDescription":{"name":"KS_CAMPAIGN_SRC","windowType":nul
    l,"readQueries":[{"queryString":"CREATE STREAM KS_CAMPAIGNS WITH
    (KAFKA_TOPIC='KS_CAMPAIGNS', PARTITIONS=3, REPLICAS=1) AS SELECT\n
    KS_CAMPAIGN_SRC.ID ID,\n  KS_CAMPAIGN_SRC.CONTEXT CONTEXT,\n  KS_CAMPAIGN_SRC.ORGANISATION
    ORGANISATION,\n  K"[truncated 3065 bytes]; line: 1, column: 919] (through
    reference chain:
    io.confluent.ksql.rest.entity.KsqlEntityList[0]->io.confluent.ksql.rest.entity.S
    ourceDescriptionEntity["sourceDescription"]->io.confluent.ksql.rest.entity.Sourc
    eDescription["readQueries"]->java.util.ArrayList[0])
Caused by: statusCount

To Reproduce Steps to reproduce the behavior:

  1. Create a stream or table via the plugin
  2. Run a DESCRIBE query against the new stream or table
  3. See error

Expected behavior A stream description should be returned

squ1d123 commented 4 years ago

I am seeing the same issue, Is this to do with ksql and kafka version compatibility? I am running ksqldb - cp-5.5.0 kafka - cp-5.3.1

stewartbryson commented 4 years ago

@terryf82 You had some additional insight into this, correct? I seem to recall it was a version issue... is that the case?

stewartbryson commented 4 years ago

I am unable to reproduce this, using CP 5.5.1:

--> gradle ksqlPipelineTest --tests "ExecuteTest.Execute :pipelineExecute task with default values"

> Task :ksqlPipelinesTest

ExecuteTest

  Test Execute :pipelineExecute task with default values PASSED (24.5s)
Results: SUCCESS (1 tests, 1 successes, 0 failures, 0 skipped)

SUCCESS: Executed 1 tests in 29s

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.5.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD SUCCESSFUL in 32s
7 actionable tasks: 7 executed

~/Source/gradle-confluent issues/86 33s
--> ksql
ksql> describe PAGES_PER_MIN;

Name                 : PAGES_PER_MIN
 Field   | Type
------------------------------------------------------------
 ROWTIME | BIGINT           (system)
 ROWKEY  | INTEGER          (system) (Window type: HOPPING)
 USERID  | INTEGER
 PAGES   | BIGINT
------------------------------------------------------------
For runtime statistics and query details run: DESCRIBE EXTENDED <Stream,Table>;
ksql>
terryf82 commented 4 years ago

@stewartbryson yep, I hit this bug while using the confluentinc/cp-ksqldb-server:5.5.0 image, which you thought was based on ksqldb 0.7.

It went away after I swapped to using confluentinc/ksqldb-server:0.9.0. Sounds like confluentinc/cp-ksqldb-server:5.5.1 is now using 0.9 or 0.10?

stewartbryson commented 4 years ago

@squ1d123 Is it possible for you to verify using a newer version of Kafka? I have verified now with 5.5.0 and 5.5.1 of CP, and I cannot reproduce this.