KAFKA_PUBLISH task not working

[tarunya882]

I have a kafka task in my workflow, when execution reaches kafka task it is always stuck in SCHEDULED state. Here is the workflow, can anyone help if I am missing anything.

{
  "createTime": 1679668219943,
  "updateTime": 1679672604493,
  "accessPolicy": {},
  "name": "myWorkflow",
  "description": "My sample workflow",
  "version": 2,
  "tasks": [
    {
      "name": "call_kafka",
      "taskReferenceName": "call_kafka",
      "inputParameters": {
        "kafka_request": {
          "topic": "example-topic",
          "value": "Message to publish",
          "bootStrapServers": "localhost:9092",
          "headers": {},
          "key": "123",
          "keySerializer": "org.apache.kafka.common.serialization.IntegerSerializer"
        }
      },
      "type": "KAFKA_PUBLISH",
      "startDelay": 0,
      "optional": false,
      "asyncComplete": false
    }
  ],
  "inputParameters": [],
  "outputParameters": {},
  "schemaVersion": 2,
  "restartable": true,
  "workflowStatusListenerEnabled": false,
  "ownerEmail": "example@email.com",
  "timeoutPolicy": "ALERT_ONLY",
  "timeoutSeconds": 0,
  "variables": {},
  "inputTemplate": {}
}

[v1r3n]

Hi @tarunya882 do you have kafka publish module in your deployment? https://github.com/Netflix/conductor-community/tree/main/task/kafka

If you are just building from https://github.com/Netflix/conductor it does not have the module enabled.

[tarunya882]

Hi @v1r3n Yes, I have added Kafka module and task is picked up and status got changed to INPROGRESS. But task status is not updated even after publishing messages to kafka. I can see the message got published to topic but task is still in INPROGRESS state. I want to change it to completed after task execution is done. I think kafka task should handle all these updates automatically, but somehow it is not happening. How can I fix this? I have added this in build.gradle runtimeOnly group: 'com.netflix.conductor', name: 'conductor-kafka', version: '3.13.5'