eclipse / tahu

Eclipse Tahu addresses the existence of legacy SCADA/DCS/ICS protocols and infrastructures and provides a much-needed definition of how best to apply MQTT into these existing industrial operational environments.
https://eclipse.org/tahu
Eclipse Public License 2.0
216 stars 123 forks source link

key multiPart missing #389

Open marcoratto opened 2 weeks ago

marcoratto commented 2 weeks ago

I have a problem using the key "MultiPart" for the key "Metadata". I wrote a java sample code: IssueMetaDataMultiPart.java.txt

This is the output written on the console: ` MetaData [isMultiPart=true, contentType=text/plain, size=12, seq=0, fileName=fileName, fileType=fileType, md5=md5, description=config]

Metric [name=file, alias=1234, timestamp=1718715733989, dataType=String, isHistorical=null, isTransient=null, metaData=MetaData [isMultiPart=true, contentType=text/plain, size=12, seq=0, fileName=fileName, fileType=fileType, md5=md5, description=config], properties=null, value=Hello world!, isNull=false] `

This is the message send to the Broker MQTT:

{ "timestamp": 1718715733989, "metrics": [ { "name": "file", "alias": 1234, "timestamp": 1718715733989, "dataType": "String", "metaData": { "contentType": "text/plain", "size": 12, "seq": 0, "fileName": "fileName", "fileType": "fileType", "md5": "md5", "description": "config" }, "value": "Hello world!" } ], "seq": 0 }

the key isMultiPart is missing. I tried the same code with the Tahu Libraries 1.0.0 and 1.0.7 but it doesn't work.

Someone could help me ?

thanks in advance Marco

wes-johnson commented 2 weeks ago

What tool are you using to view the resulting message?

marcoratto commented 2 weeks ago

I'm using mqttfx but I also wrote a Java class for encode and decode sparkplug messages and I have always the same issue.

wes-johnson commented 2 weeks ago

I know the last version of MQTT.fx I used didn't properly display all portions of the Metric including the MetaData. Based on your code, I think you've set it up properly. What does your decoder code look like? What if you use the Tahu decoder?

marcoratto commented 2 weeks ago

I tried with a python script:

./DecodeSparkplugB.py` ../../../cmdFileUpload.bin

timestamp: 1718803102 metrics { name: "cmdFileUpload" alias: 10000 timestamp: 1718803102 datatype: 12 is_null: false metadata { content_type: "application/json" size: 46 seq: 0 file_name: "12" file_type: "base64" md5: "bd694094acf029e4db780a322fcd7611" description: "config" } string_value: "ewogICJJRUMtMTA0LVNsYXZlIjogewogICAgInZlcnNpb24iOiAxCiAgfQp9Cg==" } seq: 18446744073709551615

and with a java tahu example.

{

"timestamp": 1718887068, "metrics": [ { "name": "cmdFileUpload", "alias": 10000, "timestamp": 1718887068, "dataType": "String", "metaData": { "contentType": "application/json", "multiPart": true, "size": 46, "seq": 0, "fileName": "12", "fileType": "base64", "md5": "bd694094acf029e4db780a322fcd7611", "description": "config" }, "value": "ewogICJJRUMtMTA0LVNsYXZlIjogewogICAgInZlcnNpb24iOiAxCiAgfQp9Cg==" } ] }

Metric [name=cmdFileUpload, alias=10000, timestamp=Tue Jan 20 22:28:07 CET 1970, dataType=String, isHistorical=null, isTransient=null, isNull=false, metaData=MetaData [isMultiPart=true, contentType=application/json, size=46, seq=0, fileName=12, fileType=base64, md5=bd694094acf029e4db780a322fcd7611, description=config], propertySet=null, value=ewogICJJRUMtMTA0LVNsYXZlIjogewogICAgInZlcnNpb24iOiAxCiAgfQp9Cg==]

java -jar ./java/examples/listener/target/example_listener-1.0.7.jar Message Arrived on Sparkplug topic SparkplugListener/Sparkplug B Devices/NCMD/Python Edge Node 1 Message Arrived on Sparkplug topic SparkplugListener/Sparkplug B Devices/NCMD/Python Edge Node 1 14:37:48.733 [MQTT Call: SparkplugBListenerEdgeNode] TRACE o.e.t.m.SparkplugBPayloadDecoder - For metricName=cmdFileUpload and alias=10000 - handling metric type in decoder: 12 { "timestamp" : 1718887068, "metrics" : [ { "name" : "cmdFileUpload", "alias" : 10000, "timestamp" : 1718887068, "dataType" : "String", "metaData" : { "contentType" : "application/json", "size" : 46, "seq" : 0, "fileName" : "12", "fileType" : "base64", "md5" : "bd694094acf029e4db780a322fcd7611", "description" : "config", "multiPart" : false }, "value" : "ewogICJJRUMtMTA0LVNsYXZlIjogewogICAgInZlcnNpb24iOiAxCiAgfQp9Cg==" } ], "seq" : -1 }

Json sample: { "timestamp": 1718803102, "metrics": [ { "name": "cmdFileUpload", "alias": 10000, "timestamp": 1718803102, "dataType": "String", "metaData": { "contentType": "application/json", "multiPart": true, "size": 46, "seq": 0, "fileName": "12", "fileType": "base64", "md5": "bd694094acf029e4db780a322fcd7611", "description": "config" }, "value": "ewogICJJRUMtMTA0LVNsYXZlIjogewogICAgInZlcnNpb24iOiAxCiAgfQp9Cg==" } ] }

hex dump for the binary message:

hexdump -C cmdFileUpload.bin

00000000 08 9e b5 cb b3 06 12 ac 01 0a 0d 63 6d 64 46 69 |...........cmdFi| 00000010 6c 65 55 70 6c 6f 61 64 10 90 4e 18 9e b5 cb b3 |leUpload..N.....| 00000020 06 20 0c 38 00 42 4c 12 10 61 70 70 6c 69 63 61 |. .8.BL..applica| 00000030 74 69 6f 6e 2f 6a 73 6f 6e 18 2e 20 00 2a 02 31 |tion/json.. .*.1| 00000040 32 32 06 62 61 73 65 36 34 3a 20 62 64 36 39 34 |22.base64: bd694| 00000050 30 39 34 61 63 66 30 32 39 65 34 64 62 37 38 30 |094acf029e4db780| 00000060 61 33 32 32 66 63 64 37 36 31 31 42 06 63 6f 6e |a322fcd7611B.con| 00000070 66 69 67 7a 40 65 77 6f 67 49 43 4a 4a 52 55 4d |figz@ewogICJJRUM| 00000080 74 4d 54 41 30 4c 56 4e 73 59 58 5a 6c 49 6a 6f |tMTA0LVNsYXZlIjo| 00000090 67 65 77 6f 67 49 43 41 67 49 6e 5a 6c 63 6e 4e |gewogICAgInZlcnN| 000000a0 70 62 32 34 69 4f 69 41 78 43 69 41 67 66 51 70 |pb24iOiAxCiAgfQp| 000000b0 39 43 67 3d 3d 18 ff ff ff ff ff ff ff ff ff 01 |9Cg==...........| 000000c0 `

I tried also on this website : https://protobuf-decoder.netlify.app/

to decode the Protobuf message and I don't find the definition of the key "multiPart":

image