Investigate data encoding efficiency

[tomtom5152]

Current encoding is to transmit raw JSON {"sensor_name":value} for each sensor present. Given that there are 4 10x16 matrix sensors on the device now this may be overly verbose.

It would be worth investigate how much bandwidth this uses and the limitations of it, as well as the potential CPU/network costs of alternatives. Developer readability is also hugely important should a less verbose format be used.

[tomtom5152]

Initial investigation with wireshark of the JSON object for one matrix sensor reveals that it requires 3 full packets of 1349 bytes + final 337 bytes packet on wire with DTLS/UDP. This gives a final on the wire size of 4384 bytes/frame for the left side.

Whilst this value is useful and easy to trace, it will not accurately scale to the full device.

Frame 419: 1349 bytes on wire (10792 bits), 1349 bytes captured (10792 bits) on interface 0
Null/Loopback
Internet Protocol Version 6, Src: fd00::1:a191:96ed:192f:93e2, Dst: fd00::1:a191:96ed:192f:93e2
    0110 .... = Version: 6
    .... 0000 0000 .... .... .... .... .... = Traffic class: 0x00 (DSCP: CS0, ECN: Not-ECT)
    .... .... .... 1101 1110 1110 0000 1110 = Flowlabel: 0x000dee0e
    Payload length: 1305
    Next header: UDP (17)
    Hop limit: 64
    Source: fd00::1:a191:96ed:192f:93e2
    Destination: fd00::1:a191:96ed:192f:93e2
    [Source GeoIP: Unknown]
    [Destination GeoIP: Unknown]
User Datagram Protocol, Src Port: 60291 (60291), Dst Port: 64758 (64758)
    Source Port: 60291
    Destination Port: 64758
    Length: 1305
    Checksum: 0xca4f [validation disabled]
    [Stream index: 0]
Datagram Transport Layer Security
    DTLSv1.2 Record Layer: Application Data Protocol: Application Data
        Content Type: Application Data (23)
        Version: DTLS 1.2 (0xfefd)
        Epoch: 1
        Sequence Number: 135
        Length: 1284
        Encrypted Application Data: 0001000000000087f721b65b3b6814efd720720ee5d91ecd...

Frame 422: 337 bytes on wire (2696 bits), 337 bytes captured (2696 bits) on interface 0
Null/Loopback
Internet Protocol Version 6, Src: fd00::1:a191:96ed:192f:93e2, Dst: fd00::1:a191:96ed:192f:93e2
    0110 .... = Version: 6
    .... 0000 0000 .... .... .... .... .... = Traffic class: 0x00 (DSCP: CS0, ECN: Not-ECT)
    .... .... .... 1101 1110 1110 0000 1110 = Flowlabel: 0x000dee0e
    Payload length: 293
    Next header: UDP (17)
    Hop limit: 64
    Source: fd00::1:a191:96ed:192f:93e2
    Destination: fd00::1:a191:96ed:192f:93e2
    [Source GeoIP: Unknown]
    [Destination GeoIP: Unknown]
User Datagram Protocol, Src Port: 60291 (60291), Dst Port: 64758 (64758)
    Source Port: 60291
    Destination Port: 64758
    Length: 293
    Checksum: 0xc65b [validation disabled]
    [Stream index: 0]
Datagram Transport Layer Security
    DTLSv1.2 Record Layer: Application Data Protocol: Application Data
        Content Type: Application Data (23)
        Version: DTLS 1.2 (0xfefd)
        Epoch: 1
        Sequence Number: 138
        Length: 272
        Encrypted Application Data: 000100000000008af133cec6a43b3b9d46317f09a05ca939...

[tomtom5152]

A small change to use native JSON arrays rather than an object for each sensor and using a fast writer has dramatically reduced this to the point where it's almost not an issue. New readings are coming in at 1 frame of 1349 and a second of 209 on the wire. The new total is 1588 bytes/frame OTW DTLS/UDP, just over 1/3 what it was previously for twice the data!

Frame 74: 1349 bytes on wire (10792 bits), 1349 bytes captured (10792 bits) on interface 0
Null/Loopback
Internet Protocol Version 6, Src: fd00::1:ddf2:ba06:3978:8e9, Dst: fd00::1:ddf2:ba06:3978:8e9
User Datagram Protocol, Src Port: 55578 (55578), Dst Port: 59657 (59657)
    Source Port: 55578
    Destination Port: 59657
    Length: 1305
    Checksum: 0xb3e3 [validation disabled]
    [Stream index: 0]
Datagram Transport Layer Security
    DTLSv1.2 Record Layer: Application Data Protocol: Application Data
        Content Type: Application Data (23)
        Version: DTLS 1.2 (0xfefd)
        Epoch: 1
        Sequence Number: 9
        Length: 1284
        Encrypted Application Data: 00010000000000097736c041996dc0b3b7a7d98af216189a...

Frame 74: 1349 bytes on wire (10792 bits), 1349 bytes captured (10792 bits) on interface 0
Null/Loopback
Internet Protocol Version 6, Src: fd00::1:ddf2:ba06:3978:8e9, Dst: fd00::1:ddf2:ba06:3978:8e9
User Datagram Protocol, Src Port: 55578 (55578), Dst Port: 59657 (59657)
    Source Port: 55578
    Destination Port: 59657
    Length: 1305
    Checksum: 0xb3e3 [validation disabled]
    [Stream index: 0]
Datagram Transport Layer Security
    DTLSv1.2 Record Layer: Application Data Protocol: Application Data
        Content Type: Application Data (23)
        Version: DTLS 1.2 (0xfefd)
        Epoch: 1
        Sequence Number: 9
        Length: 1284
        Encrypted Application Data: 00010000000000097736c041996dc0b3b7a7d98af216189a...