# Python version
('python=3.11.5 (tags/v3.11.5:cce6ba9, Aug 24 2023, 14:38:34) [MSC v.1936 64 '
'bit (AMD64)]')
'os=Windows-10-10.0.19044-SP0'
'numpy=1.26.1'
Code
MDF version
I used an MDF4 file
Code snippet
# https://asammdf.readthedocs.io/en/latest/api.html#asammdf.mdf.MDF.export
mdf_object_to_export.export(
fmt="parquet", keep_arrays=True, raw=True, filename=export_file_name_path, **other_kwargs
)
# (tried different combinations of raw and keep_arrays arguments)
Traceback
no traceback/error just missing column in parquet file.
Description
I am trying to export a CAN trace as parquet and just want to export the whole mdf_object via .export()
Everything works fine but the collumn/channel DataBytes is missing in the parquet file.
In the asammdf GUI program I can open the CAN trace view and the column shows the DataBytes data correctly as byte array in hex.
When I export as .csv file the column is export like this (byte array in decimal numbers):
# one example of one row of the DataBytes column
[240 160 0 10 20 30 40 50]
It seems that the parquet export just ignores byte arraycolumns.
I already tried the argument raw=True and keep_arrays=True but I did not find the column in the result either.
Some other issues which describe similar problems:
Code
MDF version
I used an MDF4 file
Code snippet
Traceback
no traceback/error just missing column in parquet file.
Description
I am trying to export a CAN trace as parquet and just want to export the whole
mdf_object
via.export()
Everything works fine but the collumn/channel
DataBytes
is missing in the parquet file.In the asammdf GUI program I can open the CAN trace view and the column shows the DataBytes data correctly as byte array in hex.
When I export as
.csv
file the column is export like this (byte array in decimal numbers):It seems that the parquet export just ignores byte arraycolumns.
I already tried the argument
raw=True
andkeep_arrays=True
but I did not find the column in the result either.Some other issues which describe similar problems:
https://github.com/danielhrisca/asammdf/issues/918
https://github.com/danielhrisca/asammdf/issues/186
https://github.com/danielhrisca/asammdf/issues/299
Does somebody have a solution? I really would like to export to parquet instead of csv because then I can query the data faster.