confluentinc / ksql

The database purpose-built for stream processing applications.
https://ksqldb.io
Other
127 stars 1.04k forks source link

Need better error messages when not able to connect to SchemaRegistry #5876

Open spena opened 4 years ago

spena commented 4 years ago

Describe the bug When attempting to create an AVRO topic, if KSQL cannot connect to SchemaRegistry, it displays a vague error message like below:

> create stream topic1(int i) with (kafka_topic='topic1', value_format='avro', replicas=1, partitions1);
Could not register schema for topic.

There were no errors in the logs either.

This happened when I configured KSQL with the following SR config:

KSQL_KSQL_SCHEMA_REGISTRY_URL: "schema-registry:8081"

The correct value was:

KSQL_KSQL_SCHEMA_REGISTRY_URL: "http://schema-registry:8081"

To Reproduce

Version: ksql-0.11

I used the docker-compose.yml that comes in the https://ksqldb.io/quickstart.html. I just added the following SchemaRegistry configs:

...
ksqldb-server:
    image: confluentinc/cp-ksqldb-server:5.5.1
    hostname: ksqldb-server
    container_name: ksqldb-server
    depends_on:
      - broker
      - connect
    ports:
      - "8088:8088"
    environment:
      KSQL_CONFIG_DIR: "/etc/ksql"
      KSQL_BOOTSTRAP_SERVERS: "broker:29092"
      KSQL_HOST_NAME: ksqldb-server
      KSQL_LISTENERS: "http://0.0.0.0:8088"
      KSQL_CACHE_MAX_BYTES_BUFFERING: 0
      KSQL_KSQL_SCHEMA_REGISTRY_URL: "http://schema-registry:8081"
      KSQL_PRODUCER_INTERCEPTOR_CLASSES: "io.confluent.monitoring.clients.interceptor.MonitoringProducerInterceptor"
      KSQL_CONSUMER_INTERCEPTOR_CLASSES: "io.confluent.monitoring.clients.interceptor.MonitoringConsumerInterceptor"
      KSQL_KSQL_CONNECT_URL: "http://connect:8083"

ksqldb-server:
    image: confluentinc/ksqldb-server:0.10.1
    hostname: ksqldb-server
    container_name: ksqldb-server
    depends_on:
      - broker
    ports:
      - "8088:8088"
    environment:
      KSQL_LISTENERS: http://0.0.0.0:8088
      KSQL_BOOTSTRAP_SERVERS: broker:9092
      KSQL_KSQL_LOGGING_PROCESSING_STREAM_AUTO_CREATE: "true"
      KSQL_KSQL_LOGGING_PROCESSING_TOPIC_AUTO_CREATE: "true"
      KSQL_KSQL_SCHEMA_REGISTRY_URL: "schema-registry:8081"
...

Expected behavior The error message should be clear about why it cannot register the schema. Failed connection? invalid SR URL?

It might be useful to do a connection check to SR when starting KSQL to verify it is well configured.

agavra commented 4 years ago

~Hey sergio - thanks for reporting this, this should no longer be a problem on master (see #5871) which was causing these issues. I also had a PR, but closed it because of the above pr (https://github.com/confluentinc/ksql/pull/5869) to improve the error.~

EDIT: nevermind, I misread the problem. Still a good idea to do this