Problems in deserializing/decoding an amqp long type in the received message

[amarzavery]

At times, rhea correctly gives me the sequenceNumber of the message which happens to be an AMQP Long type. At times it gives me a Buffer. Not sure, why it cannot be consistent.

sequence-numbers is an array of number

{"correlation_id":"8ca0eb0c-2499-4962-8e1b-6f1f73667730","application_properties":{"statusCode":200,"errorCondition":null,"statusDescription":null,"com.microsoft:tracking-id":null},"body":{"sequence-numbers":[8162774324609037]}} +278ms

  rhea:io [connection-1] read 211 bytes +78ms
  rhea:io [connection-1] got frame of size 211 +0ms
  rhea:raw [connection-1] RECV: 211 000000d302000001005314c0120b520143a004443f19004340424040404042005373c0330d4040404040a12438636130656230632d323439392d343936322d386531622d36663166373336363737333040404040404040005374c15308a10a737461747573436f646571000000c8a10e6572726f72436f6e646974696f6e40a1117374617475734465736372697074696f6e40a119636f6d2e6d6963726f736f66743a747261636b696e672d696440005377c11f02a11073657175656e63652d6e756d62657273e00a0181001d00000000000d +78ms
  rhea:frames [connection-1]:1 <- transfer#14 {"handle":1,"delivery_tag":{"type":"Buffer","data":[68,63,25,0]}} <Buffer 00 53 73 c0 33 0d 40 40 40 40 40 a1 24 38 63 61 30 65 62 30 63 2d 32 34 39 39 2d 34 39 36 32 2d 38 65 31 62 2d 36 66 31 66 37 33 36 36 37 37 33 30 40 ... > +78ms
  rhea:message decoding section: Typed { type: TypeDesc { name: 'List8', typecode: 192, width: 1, category: 3, create: { [Function] typecode: 192 } }, value: [ Typed { type: [Object], value: null }, Typed { type: [Object], value: null }, Typed { type: [Object], value: null }, Typed { type: [Object], value: null }, Typed { type: [Object], value: null }, Typed { type: [Object], value: '8ca0eb0c-2499-4962-8e1b-6f1f73667730' }, Typed { type: [Object], value: null }, Typed { type: [Object], value: null }, Typed { type: [Object], value: null }, Typed { type: [Object], value: null }, Typed { type: [Object], value: null }, Typed { type: [Object], value: null }, Typed { type: [Object], value: null } ], descriptor: { [Number: 115] type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Object] }, value: 115 } } of type: { [Number: 115] type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: { [Function] typecode: 83 } }, value: 115 } +275ms
  rhea:message decoding section: Typed { type: TypeDesc { name: 'Map8', typecode: 193, width: 1, category: 3, create: { [Function] typecode: 193 } }, value: [ Typed { type: [Object], value: 'statusCode' }, { [Number: 200] type: [Object], value: 200 }, Typed { type: [Object], value: 'errorCondition' }, Typed { type: [Object], value: null }, Typed { type: [Object], value: 'statusDescription' }, Typed { type: [Object], value: null }, Typed { type: [Object], value: 'com.microsoft:tracking-id' }, Typed { type: [Object], value: null } ], descriptor: { [Number: 116] type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Object] }, value: 116 } } of type: { [Number: 116] type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: { [Function] typecode: 83 } }, value: 116 } +1ms
  rhea:message decoding section: Typed { type: TypeDesc { name: 'Map8', typecode: 193, width: 1, category: 3, create: { [Function] typecode: 193 } }, value: [ Typed { type: [Object], value: 'sequence-numbers' }, Typed { type: [Object], value: [Array], array_constructor: [Object] } ], descriptor: { [Number: 119] type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Object] }, value: 119 } } of type: { [Number: 119] type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: { [Function] typecode: 83 } }, value: 119 } +0ms
  rhea:events [connection-1] Link got event: message +79ms
  rhea-promise:receiver [connection-1] receiver got event: 'message'. Re-emitting the translated context. +280ms
  rhea-promise:translate [connection-1] Translating the context for event: 'message'. +79ms
  azure:amqp-common:reqreslink [connection-1] $management response:  {"correlation_id":"8ca0eb0c-2499-4962-8e1b-6f1f73667730","application_properties":{"statusCode":200,"errorCondition":null,"statusDescription":null,"com.microsoft:tracking-id":null},"body":{"sequence-numbers":[8162774324609037]}} +278ms

Sequence-numbers is an array of buffer

{"correlation_id":"6998fa8a-aa82-498b-be0a-770c599d9cb0","application_properties":{"statusCode":200,"errorCondition":null,"statusDescription":null,"com.microsoft:tracking-id":null},"body":{"sequence-numbers":[{"type":"Buffer","data":[0,137,0,0,0,0,0,9]}]}} 

rhea:io [connection-1] read 211 bytes +76ms
  rhea:io [connection-1] got frame of size 211 +0ms
  rhea:raw [connection-1] RECV: 211 000000d302000001005314c0120b520143a0044b1804004340424040404042005373c0330d4040404040a12436393938666138612d616138322d343938622d626530612d37373063353939643963623040404040404040005374c15308a10a737461747573436f646571000000c8a10e6572726f72436f6e646974696f6e40a1117374617475734465736372697074696f6e40a119636f6d2e6d6963726f736f66743a747261636b696e672d696440005377c11f02a11073657175656e63652d6e756d62657273e00a01810089000000000009 +76ms
  rhea:frames [connection-1]:1 <- transfer#14 {"handle":1,"delivery_tag":{"type":"Buffer","data":[75,24,4,0]}} <Buffer 00 53 73 c0 33 0d 40 40 40 40 40 a1 24 36 39 39 38 66 61 38 61 2d 61 61 38 32 2d 34 39 38 62 2d 62 65 30 61 2d 37 37 30 63 35 39 39 64 39 63 62 30 40 ... > +76ms
  rhea:message decoding section: Typed { type: TypeDesc { name: 'List8', typecode: 192, width: 1, category: 3, create: { [Function] typecode: 192 } }, value: [ Typed { type: [Object], value: null }, Typed { type: [Object], value: null }, Typed { type: [Object], value: null }, Typed { type: [Object], value: null }, Typed { type: [Object], value: null }, Typed { type: [Object], value: '6998fa8a-aa82-498b-be0a-770c599d9cb0' }, Typed { type: [Object], value: null }, Typed { type: [Object], value: null }, Typed { type: [Object], value: null }, Typed { type: [Object], value: null }, Typed { type: [Object], value: null }, Typed { type: [Object], value: null }, Typed { type: [Object], value: null } ], descriptor: { [Number: 115] type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Object] }, value: 115 } } of type: { [Number: 115] type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: { [Function] typecode: 83 } }, value: 115 } +270ms
  rhea:message decoding section: Typed { type: TypeDesc { name: 'Map8', typecode: 193, width: 1, category: 3, create: { [Function] typecode: 193 } }, value: [ Typed { type: [Object], value: 'statusCode' }, { [Number: 200] type: [Object], value: 200 }, Typed { type: [Object], value: 'errorCondition' }, Typed { type: [Object], value: null }, Typed { type: [Object], value: 'statusDescription' }, Typed { type: [Object], value: null }, Typed { type: [Object], value: 'com.microsoft:tracking-id' }, Typed { type: [Object], value: null } ], descriptor: { [Number: 116] type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Object] }, value: 116 } } of type: { [Number: 116] type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: { [Function] typecode: 83 } }, value: 116 } +1ms
  rhea:message decoding section: Typed { type: TypeDesc { name: 'Map8', typecode: 193, width: 1, category: 3, create: { [Function] typecode: 193 } }, value: [ Typed { type: [Object], value: 'sequence-numbers' }, Typed { type: [Object], value: [Array], array_constructor: [Object] } ], descriptor: { [Number: 119] type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: [Object] }, value: 119 } } of type: { [Number: 119] type: TypeDesc { name: 'SmallUlong', typecode: 83, width: 1, category: 1, read: [Function: read], write: [Function: write], create: { [Function] typecode: 83 } }, value: 119 } +0ms
  rhea:events [connection-1] Link got event: message +77ms
  rhea-promise:receiver [connection-1] receiver got event: 'message'. Re-emitting the translated context. +276ms
  rhea-promise:translate [connection-1] Translating the context for event: 'message'. +77ms
  azure:amqp-common:reqreslink [connection-1] $management response:  {"correlation_id":"6998fa8a-aa82-498b-be0a-770c599d9cb0","application_properties":{"statusCode":200,"errorCondition":null,"statusDescription":null,"com.microsoft:tracking-id":null},"body":{"sequence-numbers":[{"type":"Buffer","data":[0,137,0,0,0,0,0,9]}]}} +274ms

I tried using the int64-buffer library to convert the buffer [0,137,0,0,0,0,0,9] into a number (long). However, it has other issues where it is not able to convert the number into the same buffer accurately.

Is it possible for rhea to always convert the long type into a buffer or into a number consistently?

[grs]

Javscript only uses 53 bits for numbers which limits the range of longs that can be accurately represented. In decoding, rhea will only decode the long to a number if it is non-lossy to do so, otherwise it will return a buffer.

[0,137,0,0,0,0,0,9] is outside the range that can be expressed with 53 bytes so it is returned as a buffer.

If you pass the buffer to int64-buffer's Int64BE that should work, e.g.:

var value = require('int64-buffer').Int64BE(Buffer.from([0,137,0,0,0,0,0,9]));
console.log('value is %s', value);

[amarzavery]

I have a management operation in ServiceBus where the user can schedule a message to be active on the Queue at a later time. ServiceBus returns the sequenceNumber (of AMQP LONG type) of the message in response. If the user wishes to cancel the scheduled message, the sequenceNumber needs to be provided to ServiceBus for it to determine the right message to be cancelled.

Thus I need to be able to round trip the sequence number correctly. Problem with int64-buffer library is that the round trip cannot happen successfully as can be seen in this issue. Some bytes are lost in the transformation. When I specify the sequence number to ServiceBus, it returns an error stating that the sequenceNumber is not correct.

In decoding, rhea will only decode the long to a number if it is non-lossy to do so, otherwise it will return a buffer.

Is it possible to tell rhea to always send a buffer and let the customer handle the conversion?

In the issue filed in int64-buffer, you can see the Int64LE round trip number <---> buffer representation works fine. Once I saw it converting to a negative number, hence was thinking to use UInt64LE over Int64BE. Do you think that should be fine? Is there a reason to use the BE flavor over LE flavor?

[grs]

The precision is lost because you are converting to a javascript number which does not have sufficient precision. You need to use a different javascript type if externalizing the value. An AMQP long is a signed value (a ulong would be unsigned) and you want the bigendian version.

[amarzavery]

Have decided to take a dependency on long.js. It provides a nice Long type with methods for addition/substraction, etc. Commenting here so other folks can benefit from this.