ClassCastException when using both spring-integration-kafka and spring-cloud-starter-bus-kafka

[LawrenceMouarkach]

When importing spring-cloud-starter-bus-kafka to a project using spring-integration-kafka when writing to kafka i receive the following exception:

I'm not sure what else I can do as theres no error within the code which I can trace back to our stack. It appears that the object is not being deserialized properly and ending up as a LinkedHashMap.

When I remove the spring-cloud-starter-bus-kafka lib everything works as intended.

Version of spring-integration-kafka: 3.1.5 Version of spring-cloud-starter-bus-kafka: 2.1.1

I've tried several versions and excluding certain groups and modules, I do know that when I exclude org.springframework.cloud:spring-cloud-bus everything is fine but then functionality within the spring-cloud-starter-bus-kafka doesn't work.

I think from investigating it a little further within spring-cloud-starter-bus-kafka the spring-cloud-bus dep within there the spring-cloud-stream dependency could be where the incompatibility lies.

I was unsure where to raise this but since the issue is traced back to the integration package in the stacktrace thought to raise it here.

Caused by: java.lang.ClassCastException: java.util.LinkedHashMap cannot be cast to org.apache.avro.specific.SpecificRecord
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:171)
    at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:120)
    at org.springframework.integration.handler.support.MessagingMethodInvokerHelper$HandlerMethod.invoke(MessagingMethodInvokerHelper.java:1115)
    at org.springframework.integration.handler.support.MessagingMethodInvokerHelper.invokeHandlerMethod(MessagingMethodInvokerHelper.java:624)
    at org.springframework.integration.handler.support.MessagingMethodInvokerHelper.processInternal(MessagingMethodInvokerHelper.java:491)
    at org.springframework.integration.handler.support.MessagingMethodInvokerHelper.process(MessagingMethodInvokerHelper.java:362)
    at org.springframework.integration.handler.MethodInvokingMessageProcessor.processMessage(MethodInvokingMessageProcessor.java:106)
    ... 39 common frames omitted

[artembilan]

How about to use the latest stable Spring Cloud Bus version: https://github.com/spring-cloud/spring-cloud-bus/releases ?

You don't need an explicit dependency for spring-integration-kafka: it comes transitively via Spring Cloud Stream Kafka Binder. So, just rely on the versions provided by Spring Cloud!

[LawrenceMouarkach]

Hi Artem, I removed the explicit dependency on spring-integration-kafka!

Thanks for the heads up!

As our project is at the moment we cannot upgrade to latest springBoot without a bit of work and time dedicated to doing so.

Therefore using the latest version of spring-cloud-bus isn't an immediate short term option.

So I tried the following as we are on springBootVersion: 2.1.8.RELEASE

        "org.springframework.cloud:spring-cloud-starter-zipkin:2.1.1.RELEASE",
        "org.springframework.cloud:spring-cloud-starter-config:2.1.1.RELEASE",
        "org.springframework.cloud:spring-cloud-starter-bus-kafka:2.1.1.RELEASE",
        "org.springframework.cloud:spring-cloud-stream-binder-kafka:2.1.1.RELEASE"

still the same issue.

Willing to upgrade springBoot and subsequently use the latest spring-cloud-bus version but obviously for the whole project might take a while, but if theres no other way of getting round this issue I am happy to do so.

Thanks

[artembilan]

Well, in fact you don't need that spring-cloud-stream-binder-kafka dependency as well. It comes from the spring-cloud-starter-bus-kafka.

I'm not sure what is going on with your application, so probably if you can share a simple one and already in that spring-cloud-bus project, it would be much easier to investigate.

It fully doesn't look like Spring Integration for Apache Kafka problem...

java.lang.ClassCastException: java.util.LinkedHashMap cannot be cast to org.apache.avro.specific.SpecificRecord

You may not have a proper deserialization on the consumer. Or spring-cloud-bus overrides your consumer config somehow...

[LawrenceMouarkach]

@artembilan I'm happy for u to close this for now as I won't get a chance to actually action the above any time soon! I'll try and get something and open the issue in the coming months

[artembilan]

Closed as "Can't Reporduce"