Invalid schema type JSON; error code: 422

[barmaxence]

Version & Environment

Redpanda version: latest 23.2 community

version: "3.7"
name: redpanda-quickstart
networks:
  redpanda_network:
    driver: bridge
volumes:
  redpanda-0: null
services:
  redpanda-0:
    command:
      - redpanda
      - start
      - --kafka-addr internal://0.0.0.0:9092,external://0.0.0.0:19092
      # Address the broker advertises to clients that connect to the Kafka API.
      # Use the internal addresses to connect to the Redpanda brokers'
      # from inside the same Docker network.
      # Use the external addresses to connect to the Redpanda brokers'
      # from outside the Docker network.
      - --advertise-kafka-addr internal://redpanda-0:9092,external://localhost:19092
      - --pandaproxy-addr internal://0.0.0.0:8082,external://0.0.0.0:18082
      # Address the broker advertises to clients that connect to the HTTP Proxy.
      - --advertise-pandaproxy-addr internal://redpanda-0:8082,external://localhost:18082
      - --schema-registry-addr internal://0.0.0.0:8081,external://0.0.0.0:18081
      # Redpanda brokers use the RPC API to communicate with eachother internally.
      - --rpc-addr redpanda-0:33145
      - --advertise-rpc-addr redpanda-0:33145
      # Tells Seastar (the framework Redpanda uses under the hood) to use 1 core on the system.
      - --smp 1
      # The amount of memory to make available to Redpanda.
      - --memory 1G
      # Mode dev-container uses well-known configuration properties for development in containers.
      #- --mode dev-container
      # enable logs for debugging.
      - --default-log-level=debug
    image: docker.redpanda.com/redpandadata/redpanda:latest
    container_name: redpanda-0
    volumes:
      - redpanda-0:/var/lib/redpanda/data
    networks:
      - redpanda_network
    ports:
      - 18081:18081
      - 18082:18082
      - 19092:19092
      - 19644:9644
  console:
    container_name: redpanda-console
    image: docker.redpanda.com/redpandadata/console:latest
    networks:
      - redpanda_network
    entrypoint: /bin/sh
    command: -c 'echo "$$CONSOLE_CONFIG_FILE" > /tmp/config.yml; /app/console'
    environment:
      CONFIG_FILEPATH: /tmp/config.yml
      CONSOLE_CONFIG_FILE: |
        kafka:
          brokers: ["redpanda-0:9092"]
          schemaRegistry:
            enabled: true
            urls: ["http://redpanda-0:8081"]
        redpanda:
          adminApi:
            enabled: true
            urls: ["http://redpanda-0:9644"]
    ports:
      - 8080:8080
    depends_on:
      - redpanda-0

using latest (23.2) Redpanda community Docker install (docker-compose above) using latest (2.3.0) Confluent Kafka .NET client

What went wrong?

When I try to use the schema registry. In particular on creating schema with AutoRegisterSchemas producer option, I get a schema registry exception.

Confluent.Kafka.ProduceException2[System.String,MessageTest]: Local: Value serialization error ---> Confluent.SchemaRegistry.SchemaRegistryException: Invalid schema type JSON; error code: 422 at Confluent.SchemaRegistry.RestService.ExecuteOnOneInstanceAsync(Func1 createRequest) at Confluent.SchemaRegistry.RestService.RequestAsync[T](String endPoint, HttpMethod method, Object[] jsonBody) at Confluent.SchemaRegistry.RestService.RegisterSchemaAsync(String subject, Schema schema, Boolean normalize) at Confluent.SchemaRegistry.CachedSchemaRegistryClient.RegisterSchemaAsync(String subject, Schema schema, Boolean normalize) at Confluent.SchemaRegistry.Serdes.JsonSerializer1.SerializeAsync(T value, SerializationContext context) at Confluent.Kafka.Producer2.ProduceAsync(TopicPartition topicPartition, Message`2 message, CancellationToken cancellationToken)

What I can see on the internet is a cp-schema-registry version update could solve this issue. But Redpanda image is all in one, I don't have control on it.

How to reproduce the issue?

• Redpanda fresh install using the docker-compose above
• using .NET Confluent Kafka client, try to produce a message using JsonSerializer and AutoRegisterSchemas option enabled

JIRA Link: CORE-1533

[rockwotj]

Hi @barmaxence! Redpanda does not current support JSON for schema registry, you'll want to track this feature request: https://github.com/redpanda-data/redpanda/issues/6220

[barmaxence]

Thx for pointing this at me. So feature requested since a year... It probably easier to come back on cp-kafka and cp-kafka-schema-register for now

[rockwotj]

Understandable. It's also possible to use Redpanda with other schema registry implementations instead of the built in support 👍