search

Foo-Foo-MQ / foo-foo-mq

Abstractions around RabbitMQ
MIT License
49 stars 24 forks source link

Certain `content_type` properties cause a message to disappear #38

Open GoFightNguyen opened 2 years ago

GoFightNguyen commented 2 years ago

RabbitMQ v3.8.9 foo-foo-mq v7.0.0

foo-foo-mq appears to be dropping messages depending on the value of the content_type property. Example values in which the message is dropped:

Example values in which the message is handled:

We have a pkg that wraps foo-foo-mq in order to provide some consumer processing logic. When foo-foo-mq drops the message, neither the message nor an error are surfaced to our pkg. In addition, the message is ack'd 😦

elizabeth-prairie commented 2 years ago

Specifically in RabbitMQ, the message is being ack'd as deliver(manual ack) when we were expecting it would be nack'd. Also, this error is only showing up by default when DEBUG=*.

We did discover that we can add a custom serializer for application/json; charset=UTF-8 and following this documentation would allow this error to be logged; however, we were expecting an error like this to be surfaced without configuration.

zlintz commented 1 year ago

Thanks @GoFightNguyen and @elizabeth-a-hobbs do you have a proposed change to support this functionality that you could add as a PR?

zlintz commented 1 year ago

Do you have a proposed solution to this problem?

zlintz commented 2 months ago

Hey @GoFightNguyen just following up also here... my phone hasn't change so call/text me if you like. If I recall I/we have encountered this before. There are default handlers based on contentType and if it can't determine the proper handler it fails to process it (dropped message, not determinable to be processable, no assumptions). Last time I had this issue the publisher wasn't providing what is translated to the content type. This is a recurring issue it seems, specifically around inconsistency on the publishing side as a result of inconsistent client libraries, which is bound to happen from a org with polyglot languages. I also understand this is a hard problem, getting large orgs to follow a best practice, particularly when an implementation "seems to work" out the box... If it was all one language (in this case node) and using this lib i.e. the internal "wrapper", this wouldn't be an issue.