RangeError [ERR_OUT_OF_RANGE] identified in messages with UTF8 bodies with certain length

[sh-developers]

Before reading the rest of this bug report it is best to read about the first time we hit this issue, back in 2018: #168

Background

We had some problems some time ago when sending certain messages over the messagebus, and have now identified how the bug might be replicable. It has to do with messages having a body being shorter than a uint8 (>=0 & <=255) in UTF8 characters, but longer than a uint8 in bytes.

Replication

Below is a sample of a test message in which this exact error occurs. Body:

{
  body: '{"id":null,"model":"product-metafield","data":{"key":"meta_description_fr","value":"Bestel de Testing nu voor €0.00 bij Smartphonehoesjes Zakelijk. ✓ Altijd gratis verzonden ✓ 30 dagen bedenktijd","productId":104185737},"shopUId":"shz","op":"create"}',
  message_annotations: undefined,
  application_properties: {
    delay: 0,
    failed: 0,
    'instance-identifier': 'Censoredush-0-4-17-01DS2W136EF6TVEQN4VZ5W9BEQ',
    'message-type': 'PushCensoredataQueueable'
  },
  priority: 0,
  creation_time: 1573128866466,
  content_type: 'application/json'
}

Error on send:

RangeError [ERR_OUT_OF_RANGE]: The value of "value" is out of range. It must be >= 0 and <= 255. Received 256
  at writeU_Int8 (internal/buffer.js:724:11)
  at Buffer.writeUInt8 (internal/buffer.js:734:10)
  at types.Writer.write_uint (/censored/node_modules/rhea/lib/types.js:726:28)
  at types.Writer.write_variable_width (/censored/node_modules/rhea/lib/types.js:752:10)
  at types.Writer.write_value (/censored/node_modules/rhea/lib/types.js:790:14)
  at types.Writer.write (/censored/node_modules/rhea/lib/types.js:780:14)
  at Object.message.encode (/censored/node_modules/rhea/lib/message.js:211:16)
  at Sender.send (/censored/node_modules/rhea/lib/link.js:335:50)
  at /censored/node_modules/censored/dist/bus/bus-server.js:93:26

Result / how to test

We could relatively easily replicate this error when the body of the message as it is placed on the message bus has a different bytelength, than it has a (unicode) length. Finding a(nother) message that has the above mentioned charisteristics, can be found for example using https://mothereff.in/byte-counter which reports for the above example a unicode/string length of 250 and a byte length of 256.

We have had this problem for a long time, so it would be great if this bug could finally be squashed.

[grs]

Thanks for the reproducer! I have checked in a fix for this now.

[grs]

Should be fixed in 1.0.12

[sh-developers]

That was a quick fix! We will test it in our pipeline, but I expect it to be fixed now, looking at the commits. Many thanks!