search

wso2 / product-micro-integrator

The cloud-native configuration driven runtime that helps developers implement composite microservices.
https://wso2.com/integration/
Apache License 2.0
204 stars 221 forks source link

The message processor is not consuming the messages once the failed back-end becomes available #3771

Open veneerac opened 1 week ago

veneerac commented 1 week ago

Description

If the backend fails while being called by RabbitMQ message processors and stays idle for a long time, message consumption fails even after the backend recovers. The logs show retries, but messages aren’t consumed until a restart .or reactive the message processor

Steps to Reproduce

[2024-11-12 17:07:41,946] ERROR {ForwardingService} - BlockingMessageSender of message processor [NewIncidentDCMessageProcessor] failed to forward message to endpoint NewIncidentDCcoreConsumerEP [2024-11-12 17:07:54,263] INFO {ForwardingService} - Message processor [NewIncidentDCMessageProcessor] received a response with HTTP_SC: from backend NewIncidentDCcoreConsumerEP. Message forwarding failed. [2024-11-12 17:07:54,263] ERROR {ForwardingService} - BlockingMessageSender of message processor [NewIncidentDCMessageProcessor] failed to forward message to endpoint NewIncidentDCcoreConsumerEP [2024-11-12 17:09:21,947] INFO {ForwardingService} - Message processor [NewIncidentDCMessageProcessor] received a response with HTTP_SC: from backend NewIncidentDCcoreConsumerEP. Message forwarding failed. [2024-11-12 17:09:21,947] ERROR {ForwardingService} - BlockingMessageSender of message processor [NewIncidentDCMessageProcessor] failed to forward message to endpoint NewIncidentDCcoreConsumerEP [2024-11-12 17:09:34,264] INFO {ForwardingService} - Message processor [NewIncidentDCMessageProcessor] received a response with HTTP_SC: from backend NewIncidentDCcoreConsumerEP. Message forwarding failed. [2024-11-12 17:09:34,264] ERROR {ForwardingService} - BlockingMessageSender of message processor [NewIncidentDCMessageProcessor] failed to forward message to endpoint NewIncidentDCcoreConsumerEP [2024-11-12 17:11:01,947] INFO {ForwardingService} - Message processor [NewIncidentDCMessageProcessor] received a response with HTTP_SC: from backend NewIncidentDCcoreConsumerEP. Message forwarding failed. [2024-11-12 17:11:01,947] ERROR {ForwardingService} - BlockingMessageSender of message processor [NewIncidentDCMessageProcessor] failed to forward message to endpoint NewIncidentDCcoreConsumerEP [2024-11-12 17:11:14,264] INFO {ForwardingService} - Message processor [NewIncidentDCMessageProcessor] received a response with HTTP_SC: from backend NewIncidentDCcoreConsumerEP. Message forwarding failed. [2024-11-12 17:11:14,264] ERROR {ForwardingService} - BlockingMessageSender of message processor [NewIncidentDCMessageProcessor] failed to forward message to endpoint NewIncidentDCcoreConsumerEP [2024-11-12 17:12:41,947] INFO {ForwardingService} - Message processor [NewIncidentDCMessageProcessor] received a response with HTTP_SC: from backend NewIncidentDCcoreConsumerEP. Message forwarding failed. [2024-11-12 17:12:41,948] ERROR {ForwardingService} - BlockingMessageSender of message processor [NewIncidentDCMessageProcessor] failed to forward message to endpoint NewIncidentDCcoreConsumerEP [2024-11-12 17:12:54,265] INFO {ForwardingService} - Message processor [NewIncidentDCMessageProcessor] received a response with HTTP_SC: from backend NewIncidentDCcoreConsumerEP. Message forwarding failed. [2024-11-12 17:12:54,265] ERROR {ForwardingService} - BlockingMessageSender of message processor [NewIncidentDCMessageProcessor] failed to forward message to endpoint NewIncidentDCcoreConsumerEP [2024-11-12 17:14:21,950] INFO {ForwardingService} - Message processor [NewIncidentDCMessageProcessor] received a response with HTTP_SC: from backend NewIncidentDCcoreConsumerEP. Message forwarding failed. [2024-11-12 17:14:21,950] ERROR {ForwardingService} - BlockingMessageSender of message processor [NewIncidentDCMessageProcessor] failed to forward message to endpoint NewIncidentDCcoreConsumerEP [2024-11-12 17:14:34,266] INFO {ForwardingService} - Message processor [NewIncidentDCMessageProcessor] received a response with HTTP_SC: from backend NewIncidentDCcoreConsumerEP. Message forwarding failed. [2024-11-12 17:14:34,266] ERROR {ForwardingService} - BlockingMessageSender of message processor [NewIncidentDCMessageProcessor] failed to forward message to endpoint NewIncidentDCcoreConsumerEP

Version

1.2.0.159

Environment Details (with versions)

No response