Open rmoff opened 5 years ago
Seems to be an issue with Avro. See below.
Source data:
kafkacat -b localhost:9092 -t test -C
{"hello": "world"}
Config:
curl -i -X POST -H "Content-Type:application/json" http://localhost:38083/connectors/ \
-d '{
"name": "sink-telegram-test-json",
"config": {
"connector.class": "com.github.fbascheper.kafka.connect.telegram.TelegramSinkConnector",
"value.converter": "org.apache.kafka.connect.json.JsonConverter",
"value.converter.schemas.enable": "false",
"key.converter": "org.apache.kafka.connect.json.JsonConverter",
"key.converter.schemas.enable": "false",
"errors.tolerance":"all",
"errors.deadletterqueue.topic.name":"my_dlq",
"errors.deadletterqueue.topic.replication.factor": 1,
"topics": "test",
"telegram.bot.name":"Kafka Connect Bot",
"telegram.bot.username":"rmoff_connect_02_bot",
"telegram.bot.api.key":"xxx",
"telegram.bot.destination.chat.id":"-364377679"
}
}'
Message sent, and log file shows:
INFO Using text to send telegram text-message with contents = {hello=world} (com.github.fbascheper.kafka.connect.telegram.mapper.TelegramMessageMapper)
INFO Successfully sent message with assigned message-id ArrayBuffer(7) and text '{hello=world}' (com.github.fbascheper.kafka.connect.telegram.TelegramSinkTask)
Send the data
$ kafka-avro-console-producer \
--broker-list localhost:9092 --topic test2_avro \
--property value.schema='{"type":"record","name":"myrecord","fields":[{"name":"hello","type":"string"}]}'
{"hello":"world"}
Check the data:
$ kafka-avro-console-consumer \
--bootstrap-server localhost:9092 \
--property schema.registry.url=http://localhost:8081 \
--topic test2_avro --from-beginning|jq -c '.'
{"hello":"world"}
Config:
curl -i -X POST -H "Content-Type:application/json" http://localhost:38083/connectors/ \
-d '{
"name": "sink-telegram-avro2",
"config": {
"connector.class": "com.github.fbascheper.kafka.connect.telegram.TelegramSinkConnector",
"errors.tolerance":"all",
"errors.deadletterqueue.topic.name":"my_dlq",
"errors.deadletterqueue.topic.replication.factor": 1,
"topics": "test2_avro",
"telegram.bot.name":"Kafka Connect Bot",
"telegram.bot.username":"rmoff_connect_02_bot",
"telegram.bot.api.key":"xxx",
"telegram.bot.destination.chat.id":"-364377679"
}
}'
Log error -- Found TgMessage with type null
sounds like somethings not right
INFO Converting struct back to model (com.github.fbascheper.kafka.connect.telegram.mapper.TelegramMessageMapper)
INFO Found TgMessage with type null (com.github.fbascheper.kafka.connect.telegram.mapper.TelegramMessageMapper)
ERROR WorkerSinkTask{id=sink-telegram-avro2-0} Task threw an uncaught and unrecoverable exception. Task is being killed and will not recover until manually r
estarted. (org.apache.kafka.connect.runtime.WorkerSinkTask)
scala.MatchError: null
at com.github.fbascheper.kafka.connect.telegram.mapper.TelegramMessageMapper.mapTelegramMessage$1(TelegramMessageMapper.scala:140)
at com.github.fbascheper.kafka.connect.telegram.mapper.TelegramMessageMapper.mapAvroStructMessage$1(TelegramMessageMapper.scala:133)
[…]
Tried same config with adding "enhanced.avro.schema.support": "true",
(because I found it here) but it doesn't make any difference, I get the same error.
I get this error:
Config: