Event sourcing not publishing any data

[derickmr]

Hi team.

Not sure whether this is a bug or just a misconfiguration on my end.

I have configured Apicurio Event Sourcing based on the docs. However, whenever I do an action in the registry (create/delete/update an artifact, or anything else), no data is published. I see no logs of event being fired on Apicurio side, and using kcat to tail the topic, no data is published on the topic I specified.

I am running the stack on my local machine via Docker. Apicurio version is 2.4.2.Final using the KafkaSQL distribution. This is my docker-compose configuration for the Apicurio Registry:

schema-registry:
    image: apicurio/apicurio-registry-kafkasql:2.4.2.Final
    ports:
      - "8087:8087"
    environment:
      HTTP_PORT: 8087
      registry.kafkasql.bootstrap.servers: kafka:9092
      registry.rules.global.compatibility: FULL_TRANSITIVE
      registry.rules.global.validity: SYNTAX_ONLY
      registry.rest.artifact.deletion.enabled: "true"
      registry.events.kafka.config.bootstrap.servers: kafka:9092
      registry.events.kafka.topic: registry-events

On startup, I can see the Sink subscribing to the Kafka Sink:

INFO <_> [io.apicurio.registry.events.EventsServiceImpl] (main) Subscribing sink Kafka Sink 

I also tried to do the HTTP configuration, and I got the same behavior: nothing happened.

With the DEBUG log level enabled, this is the output I get:

 2023-06-30 22:46:28 DEBUG <> [io.apicurio.registry.storage.impl.kafkasql.KafkaSqlRegistryStorage] (KSQL Kafka Consumer Thread) Consuming 1 journal records.
 2023-06-30 22:46:28 DEBUG <> [io.apicurio.registry.storage.impl.kafkasql.sql.KafkaSqlSink] (KSQL Kafka Consumer Thread) Processing Kafka message with UUID: 93ec9a26-a73d-4592-96c3-e0ae797bc9b7
 2023-06-30 22:46:28 DEBUG <_> [io.apicurio.registry.storage.impl.kafkasql.sql.KafkaSqlSink] (KSQL Kafka Consumer Thread) Clearing tenant id after message processed
 2023-06-30 22:46:28 DEBUG <> [io.apicurio.registry.storage.impl.kafkasql.sql.KafkaSqlSink] (KSQL Kafka Consumer Thread) Kafka message successfully processed. Notifying listeners of response.
 2023-06-30 22:46:28 DEBUG <> [io.apicurio.registry.storage.impl.kafkasql.KafkaSqlRegistryStorage] (KSQL Kafka Consumer Thread) Consuming 1 journal records.
 2023-06-30 22:46:28 DEBUG <> [io.apicurio.registry.storage.impl.kafkasql.sql.KafkaSqlSink] (KSQL Kafka Consumer Thread) Processing Kafka message with UUID: 7339073d-efd4-46f5-9a49-6bcb9c02cbdb
 2023-06-30 22:46:28 DEBUG <_> [io.apicurio.registry.storage.impl.sql.AbstractSqlRegistryStorage] (KSQL Kafka Consumer Thread) Inserting an artifact row for: null 7c8458c9-7a6c-4aaa-85ae-518b8166e685
 2023-06-30 22:46:28 DEBUG <_> [io.apicurio.registry.storage.impl.kafkasql.sql.KafkaSqlSink] (KSQL Kafka Consumer Thread) Clearing tenant id after message processed
 2023-06-30 22:46:28 DEBUG <> [io.apicurio.registry.storage.impl.kafkasql.sql.KafkaSqlSink] (KSQL Kafka Consumer Thread) Kafka message successfully processed. Notifying listeners of response.
 2023-06-30 22:46:28 INFO <_> [io.apicurio.common.apps.logging.audit.AuditLogService] (executor-thread-4) apicurio.audit action="createArtifact" result="success" src_ip="172.22.0.1" artifact_id="default" 
 2023-06-30 22:46:28 DEBUG <_> [io.apicurio.registry.storage.impl.sql.AbstractSqlRegistryStorage] (executor-thread-4) Getting a list of all artifact rules for: null 7c8458c9-7a6c-4aaa-85ae-518b8166e685
 2023-06-30 22:46:28 DEBUG <_> [io.apicurio.registry.storage.impl.sql.AbstractSqlRegistryStorage] (executor-thread-11) Selecting artifact (latest version) meta-data: null 7c8458c9-7a6c-4aaa-85ae-518b8166e685 (behavior = SKIP_DISABLED_LATEST)
 2023-06-30 22:46:28 DEBUG <_> [io.apicurio.registry.storage.impl.sql.AbstractSqlRegistryStorage] (executor-thread-10) Selecting artifact (latest version) meta-data: null 7c8458c9-7a6c-4aaa-85ae-518b8166e685 (behavior = SKIP_DISABLED_LATEST)
 2023-06-30 22:46:28 DEBUG <_> [io.apicurio.registry.storage.impl.sql.AbstractSqlRegistryStorage] (executor-thread-9) Searching for versions of artifact null 7c8458c9-7a6c-4aaa-85ae-518b8166e685

Steps to reproduce would be run Apicurio Registry with above configuration while running a Kafka cluster, do any relevant operation (create new artifact, for example), and see no data being published to the topic.

Please let me know if you need any extra information from my side, glad to assist.

[apicurio-bot[bot]]

Thank you for reporting an issue!

Pinging @jsenko to respond or triage.

[derickmr]

Hi @EricWittmann, @carlesarnal, @jsenko - have you had the chance to take a peak on this?

[andreaTP]

Hi @derickmr , I'll take a look in the next few days, sorry for the delay.

[andreaTP]

I'm sorry I got distracted by other tasks and this got delayed 😞 , still on my TODO, I'll eventually go through it!

[derickmr]

Thanks @andreaTP! One related question I have - do you know if the topic that will receive the registry events need to have only 1 partition set for ordering purposes? Or that's not the case here?

[andreaTP]

I started looking into this, finally ... Early findings:

do you know if the topic that will receive the registry events need to have only 1 partition set for ordering purposes?

no, this is not required as far as I understand it now.

There is an initialization ordering issue where:

• EventSourcedRegistryStorage.isEnabled() get's checked by RegistryStorageProducer before
• EventsServiceImpl.init event is fired and configuredSinks is not yet set

[andreaTP]

Thanks for reporting this issue @derickmr !

I found a bug in the singletons initialization order and, hopefully, is fixed with my PR. To let you test I uploaded a kafkasql container image with the fix at these coordinates: docker.io/andreatp/apicurio-registry-kafkasql:issue3482

Please let me know if this solves your problem!

[derickmr]

Thanks @andreaTP. I'll test it with the custom image and get back to you. Appreciate your help