provectus / kafka-ui

Open-Source Web UI for Apache Kafka Management
Apache License 2.0
9.78k stars 1.18k forks source link

SR: Expand schema versioning pattern to support Apicurio #3841

Open Haarolean opened 1 year ago

Haarolean commented 1 year ago
          I tested successfully an integration with Apicurio Registry using the Confluent compatibility API endpoint.

This is the values.yaml file used for my deployment using the Helm Chart:

yamlApplicationConfig:
  kafka:
    clusters:
      - name: my-cluster
        bootstrapServers: my-kafka-kafka-bootstrap.amq-streams-demo.svc:9092
        schemaRegistry: http://service-registry-service:8080/apis/ccompat/v6
  auth:
    type: disabled
  management:
    health:
      ldap:
        enabled: false

I found that it is working smoothly, only one detail about the schemas versioning. Confluent API only use integers (e.g: 1, 2, ...), however Apicurio Registry can use other versioning (e.g: 1.0, 1.1 ...). If your schemas in Apicurio Registry use that versioning mechanism, then Kafka-UI fails with an exception not well managed.

HTH

Originally posted by @rmarting in https://github.com/provectus/kafka-ui/issues/1786#issuecomment-1552857453

rmarting commented 1 year ago

If you follow the instructions of this repository[1] you can get the exception, as this example uses the 2.0 version for the Avro schema.

[1]https://github.com/rmarting/kafka-clients-quarkus-sample

Example of the exception stack trace:

�[30m2023-05-04 13:59:52,086�[0;39m �[1;31mERROR�[0;39m [�[34mreactor-http-epoll-6�[0;39m] �[33mo.s.b.a.w.r.e.AbstractErrorWebExceptionHandler�[0;39m: [cb544d04-102]  500 Server Error for HTTP GET "/api/clusters/my-cluster/schemas/messages/versions"
org.springframework.web.reactive.function.client.WebClientResponseException$InternalServerError: 500 Internal Server Error from GET http://service-registry-service:8080/apis/ccompat/v6/subjects/messages/versions
    at org.springframework.web.reactive.function.client.WebClientResponseException.create(WebClientResponseException.java:331)
    Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException: 
Error has been observed at the following site(s):
    *__checkpoint ⇢ 500 INTERNAL_SERVER_ERROR from GET http://service-registry-service:8080/apis/ccompat/v6/subjects/messages/versions [DefaultWebClient]
    *__checkpoint ⇢ Handler com.provectus.kafka.ui.controller.SchemasController#getAllVersionsBySubject(String, String, ServerWebExchange) [DispatcherHandler]
    *__checkpoint ⇢ com.provectus.kafka.ui.config.ReadOnlyModeFilter [DefaultWebFilterChain]
    *__checkpoint ⇢ com.provectus.kafka.ui.config.CustomWebFilter [DefaultWebFilterChain]
    *__checkpoint ⇢ AuthorizationWebFilter [DefaultWebFilterChain]
    *__checkpoint ⇢ ExceptionTranslationWebFilter [DefaultWebFilterChain]
    *__checkpoint ⇢ LogoutWebFilter [DefaultWebFilterChain]
    *__checkpoint ⇢ ServerRequestCacheWebFilter [DefaultWebFilterChain]
    *__checkpoint ⇢ SecurityContextServerWebExchangeWebFilter [DefaultWebFilterChain]
    *__checkpoint ⇢ ReactorContextWebFilter [DefaultWebFilterChain]
    *__checkpoint ⇢ HttpHeaderWriterWebFilter [DefaultWebFilterChain]
    *__checkpoint ⇢ ServerWebExchangeReactorContextWebFilter [DefaultWebFilterChain]
    *__checkpoint ⇢ org.springframework.security.web.server.WebFilterChainProxy [DefaultWebFilterChain]
    *__checkpoint ⇢ org.springframework.web.filter.reactive.ServerHttpObservationFilter [DefaultWebFilterChain]
    *__checkpoint ⇢ HTTP GET "/api/clusters/my-cluster/schemas/messages/versions" [ExceptionHandlingWebHandler]