search

hildjj / node-cbor

Encode and decode CBOR documents, with both easy mode, streaming mode, and SAX-style evented mode.
MIT License
357 stars 73 forks source link

json structure with multiple levels is not encoded properly. #110

Closed janvda closed 4 years ago

janvda commented 4 years ago

When encoding a json string with multiple levels using node-red which uses janvda/node-red-contrib-cbor (which is based on hildjj/node-cbor version v5.0.2) I don't get proper output when I test this as follows:

  1. manually copy paste the buffer output in string format from node-red debug node to https://codebeautify.org/string-hex-converter and convert it to hex.
  2. copy paste the hex string http://cbor.me/ in the right window and convert it back. In that case it reports an error.

An example of the problem

  1. The input json string:

    {"protected":{"ver":"v1","alg":"none","iat":1595339535},"payload":{"device_name":"ac:87:a3:0a:2d:1b","device_type":"DISCO-L475VG-IOT01A","interval_ms":10,"sensors":[{"name":"accX","units":"m/s2"},{"name":"accY","units":"m/s2"},{"name":"accZ","units":"m/s2"}]}}

  2. the json string converted to cbor (represented in string format)

    ¢iprotected£cverbv1calgdnoneciat_ógpayload¤kdevice_nameqac:87:a3:0a:2d:1bkdevice_typesDISCO-L475VG-IOT01Akinterval_ms
gsensorsƒ¢dnamedaccXeunitsdm/s2¢dnamedaccYeunitsdm/s2¢dnamedaccZeunitsdm/s2

  3. cbor this time converted to hex format (using https://codebeautify.org/string-hex-converter)

    a26970726f746563746564a36376657262763163616c67646e6f6e65636961741a5f16f3f677061796c6f6164a46b6465766963655f6e616d657161633a38373a61333a30613a32643a31626b6465766963655f7479706573444953434f2d4c34373556472d494f543031416b696e74657276616c5f6d73a6773656e736f727383a2646e616d65646163635865756e697473646d2f7332a2646e616d65646163635965756e697473646d2f7332a2646e616d65646163635a65756e697473646d2f7332

  4. cbor converted back gives (http://cbor.me/)

    {"protected": {"ver": "v1", "alg": "none", "iat": 1595339766}, "\u0006\u0017\x96\xC6\xF6\u0016JF\xB6FWf\x966U\xF6\xE6\u0016\xD6W\u0016\u00163": {["\xA6\u00133\xA3\u0006\u0013\xA3&C\xA3\u0016&\xB6FWf\x966U", undefined, h'97065734449534']: -21, simple(18): 20(-7558501), "\u0514\xF5C\u0003\u0014\u0016\xB6\x96\xE7FW'f\u0016\xC5\xF6\xD7": -1735615855}}

    and reports following error: 71 unused bytes after the end of the data item

janvda commented 4 years ago

The problem is that cbor in string format (see point 2 above) is not containing all relevant information (bits). So there is no issue with this repository.

I fixed it by directly converting the cbor buffer to hex format (using buffer.toString('hex')).

This resulted in following string:

a26970726f746563746564a36376657262763163616c67646e6f6e65636961741a5f1716f3677061796c6f6164a46b6465766963655f6e616d657161633a38373a61333a30613a32643a31626b6465766963655f7479706573444953434f2d4c34373556472d494f543031416b696e74657276616c5f6d730a6773656e736f727383a2646e616d65646163635865756e697473646d2f7332a2646e616d65646163635965756e697473646d2f7332a2646e616d65646163635a65756e697473646d2f7332

which is correctly converted back as specified in step 4 above.

image

hildjj commented 4 years ago

There's really no such thing as "CBOR in string format", as you found out. CBOR is bytes, not a valid string in any Unicode encoding.