binary measurement data

[ameinhardt]

As a maintenance expert, I want to have pictures alongside my machine messages / measurements / processes, in order to get more insight about this very machine state.

It should be possible to encode digital data in PPMP. Currently, there are multiple approaches on that:

• create a 'context' field for measurements / messages that indicates that ther measurement is encoded in base64 instead of being a number
• add another json key to hold base64 encoded binary data
• create another message type, specifically for sending binary data

This has been asked for by Balluff & Trumpf. Here's the former discussion on the Forum: https://www.eclipse.org/forums/index.php/t/1084828/

[kartben]

Would it be an option to have the image data send out-of-band? Would it make sense to have machine that already captures image data and make those images available e.g on an FTP server or an S3 bucket, and therefore the PPMP message could only contain the URL to an image (that, arguably, the PPMP server would probably need to download to keep a copy?)

[ameinhardt]

Hi @kartben, that works in IoT, but could be a problem in the industry, since not all machines can have internet access. I would prefer PPMP messages to remain one-way transport protocol neutral communication without the dependency to other systems.

[bgusach]

Hi @ameinhardt, here just some thoughts from my side...

Option 1: allow more types in the series.

I think this is good. At the end, data in the series should be data regardless of its type, right?

Just as thought food: if we allow the standard JSON types (number, string, bool) maybe it is not even necessary to describe the types in a "context". We could let the consumer of the data decide what to do with it, (unless we want the bindings to automatically convert from base64 to a number). This may be harder to implement in a statically typed language like Java, however.

Option 2: add another json key to hold base64 encoded binary data

Does this mean to have two keys under series, one for numeric and one for binary data? something like:

"series": {
        "$_time": [189, 677],
        "binary": {
           "iolProcessData": ["ASNFZ8KJwqvDjcOv", "wonCq8ONw68BI0Vn"]
        },
        "numeric": {
            "temperature": [12, 94]
        }

That would work too, and maybe it makes the implementation for static languages easier. Still, I prefer the first approach. With this one it is more likely to have invalid messages, like having the same dimension (e.g. temperature) in both keys.

Option 3: another type of message

Creating another message type for binary data looks complicated to me. If your machine is producing floats, binary data and strings at the same time, you would have to create diferent messages with the same timestamp and offsets but different values, and when receiving them, probably conflate them again into a record so that you know the state of your machine at a given time. That feels clunky. And if more types are needed, you would need further message types.

[fpatz]

I'd prefer option 1.

[bf-bryants]

Option 1 seems to be covered by issue #35 when the optional context section is used. Example snippet:

{
    "ts": "2002-05-30T09:30:10.123+02:00",
    "series": {
        "$_time": [ 0, 1013, 2026 ],
        "scanData": [ "Zm9vCg==", "YmFyCg==", "Y2hlZXNlCg==" ],
        "otherValue": [ 4, 5, 6 ]
    },
    "context": {
        "scanData": {
            "dataType": "BASE64"
        }
    }
}

This works for small amounts of data. For large amounts, I would suggest that PPMP is not the correct container type - putting lots of binary data into JSON is best avoided if a more appropriate alternative is available.

[ameinhardt]

I have also included the data type "REF" and "OTHER". REF could be used, e.g. in a multipart/form-data -ish telegram to reference binary data in another (non-standardized) part of the telegram.