rabbitmq / rabbitmq-server

Open source RabbitMQ: core server and tier 1 (built-in) plugins
https://www.rabbitmq.com/
Other
12.29k stars 3.91k forks source link

ActiveMQ to RabbitMQ shovel results in amqp header data being inserted in message data #12455

Open MrRoss256 opened 1 month ago

MrRoss256 commented 1 month ago

Describe the bug

As discussed on the Slack thread https://rabbitmq.slack.com/archives/C1EDN83PA/p1727472217316949, when a message is exchanged between an ActiveMQ broker and a RabbitMQ broker using a dynamic shovel some binary data appears at the beginning of the message. This looks like some AMQP control data. The shovel is configured as AMQP 1.0 -> AMQP 1.0, although the issue appears to happen with AMQP 1.0 -> AMQP 0.9.1.

The comment from the team was that the amqp-value data is being copied to the data section, and that the shovel has not been updated for the latest 4.0.2 release.

The issue existed in v3.

RabbitMQ: 4.0.2 amqp-client: 0.1.0 AWS ActiveMQ Version: 5.18.4

Reproduction steps

  1. Create a dynamic shovel between an Active MQ broker and RabbitMQ, shovel definition below (the same issue occurs with a 1.1 -> 0.9.1 shovel). ActiveMQ is the source broker at 192.168.56.45.
  {
    "node": "rabbit@host",
    "timestamp": "2024-10-05 12:02:22",
    "name": "ActiveMQ",
    "vhost": "/",
    "type": "dynamic",
    "state": "running",
    "src_uri": "amqp://guest:guest@192.168.56.45:5672?verify=verify_none",
    "src_protocol": "amqp10",
    "dest_protocol": "amqp10",
    "dest_uri": "amqp://guest:guest@localhost:5672",
    "src_address": "OUT_Q",
    "dest_address": "IN_Q",
    "blocked_status": "running"
  }
  1. Exchange a message between the ActiveMQ broker and consume it from the RabbitMQ broker (note the issue can be seen on the RabbitMQ management console where the message will be represented in Base64).
  2. The consumed message will have a few bytes of binary data appended to it. The message sent is "Message", what is received is "�SwMessage" (base 64 AFN3oQdNZXNzYWdl).

Expected behavior

The message published by ActiveMQ should be consumed by the client without the binary data.

Additional context

No response

michaelklishin commented 1 month ago

@MrRoss256 I'd like to see more information than "create a dynamic shovel". What exactly should be the parameters?

MrRoss256 commented 1 month ago

Thanks @michaelklishin, added additional details of the shovel configuration extracted from the /api/shovels/ API.

ansd commented 1 month ago

This is a bug in the shovel plugin. The shovel plugin must forward the bare message as is instead of converting from an amqp-value section to a data section.