search

peterprib / node-red-contrib-kafka-manager

Implement
GNU General Public License v3.0
22 stars 12 forks source link

Snappy codec is not available #49

Closed elfinbird closed 1 year ago

elfinbird commented 1 year ago

While consuming messages from Kafka consumer group, I am getting 'Snappy codec is not available' error. I have already installed snappy in my application, yet I am getting this error every-time I receive a message, due to this my application exits ungracefully.

Is there way to ensure snappy compression works ?

peterprib commented 1 year ago

sorry about delay. Will take a look when I have a chance.

peterprib commented 1 year ago

snappy is an optional dependency so must have failed during install. Check install log for errors. Not sure why it would not work if y installed snappy via npm ensuring it was shared with nodered

elfinbird commented 1 year ago

snappy is installed & shared with node-red, still this library throws error!

I found the issue , your node-red library is dependent on kafkaJS lib & even if Snappy is installed, your library does not inject that dependency into KafkaJS. Hence these errors are shown for all snappy compression.

Fix: To support snappy compression you will have to propagate snappy dependency from your library to KafkaJS library explicitly, you can read more about it by looking for "Adding snappy support on Kafka JS".

It will be great if you can make required code changes & add support, anyway I will be publishing my library too since I have already fixed this for my use-case.

Thanks for help.

peterprib commented 1 year ago

Made big change to compression, Now more options should be available. Now manage compression of payloads and have added more options. Facilitates more optional features.