Open jdsika opened 1 month ago
@timmruppert I have invited you to the official team.
Here is a current draft from Pierre for meta data deifnitions for osi tracefiles: https://github.com/GAIA-X4PLC-AAD/ontology-management-base/pull/80
I explored the possibilities of using MCAP, conducted a quick proof of concept (POC), and discussed it with the Persival team.
mcap
is available, which can be used to display information, among other things. Example:$ mcap info 20240830T101515Z_sv_370_244_10000_mcap.mcap
library: python mcap-protobuf-support 0.5.1; mcap 1.1.1
profile:
messages: 10000
duration: 16m39.9s
start: 0.000000000
end: 999.900000000
compression:
lz4: [6/6 chunks] [5.43 MiB/520.69 KiB (90.64%)] [533.00 B/sec]
channels:
(1) sensor_view 10000 msgs (10.00 Hz) : osi3.SensorView [protobuf]
attachments: 0
metadata: 2
In the POC, I created an .mcap file consisting of OSI3::SensorView
messages using the Python package. I then read this .mcap file using an extended version of the OpenMSL trace-file-player, which fed the data into a co-simulation using openmcx. This combination of Python and C++ worked smoothly.
Now, let's dive into the details:
Each message must be stored under a topic.
We propose using the name of the top-level message class, e.g., SensorView
.
5,2M 20240830T100943Z_sv_370_244_10000_mcap.osi # corresponding trace file
5,7M 20240830T100943Z_sv_370_244_10000_mcap.mcap # no compression
509K 20240830T101032Z_sv_370_244_10000_mcap.mcap # zstd (default)
737K 20240830T101032Z_sv_370_244_10000_mcap.mcap # lz4
- name: time_of_first_message
- key: timestamp value: string format 20210818T150542Z
- name: versions
- key: osi value: string format x.y.z
- key: protobuf value: string format x.y.z
- name: creation_tool
- key: name value: arbitrary string
cration_tool
metadata entry would be a new addition.While specifying the required metadata and formats would be sufficient, we strongly recommend providing implementations of the MCAP writer and reader, similar to the existing osi3trace/osi_trace.py
script.
The benefits of providing such implementations, which are essentially thin wrappers, include:
I will be out of the office for the next three weeks, but Clemens will be available to discuss this further. Upon my return, I plan to create a branch to provide such an implementation.
@jdsika let's discuss this next week.
Describe the feature
One or multiple OSI tracefiles should be packaged within an MCAP container MCAP repository. MCAP is now used for ros2 and is MIT licensed. Recently the requirement for signing a CLA with Foxglove Inc. was lifted. Therefore the broad usage is now unproblematic.
Describe the solution you would like
We analyse the meta data used and proposed in the MCAP project and map it to the meta data information used in the OSI project. The osi tracefile writer python script is extended to also wirte these MCAP traces. The documentation of osi regarding traces is extended with the example and usage guidelines for MCAP. A good usecase are e.g. traffic participant models which have multiple osi trace outputs which are time synchronized and could profit from being stored in a single container.
Describe alternatives you have considered
Consider mentioning the possibility of using ASAM MDF as container. Remark: MCAP is used broadly in simulation context already.
Describe the backwards compatibility
No issues. We should name the tracefiles contained within the MCAP container according to the existing naming conventions.