asammdf hierarchy in python

[StefanLaube96]

Python version

3.6

MDF version

5.14.5

Description

Hello,

I want to write a python-script that takes data from iFile and write it to mdf4.

My question is about the block hierarchical order in asammdf. On 'https://www.asam.net/standards/detail/mdf/wiki/' I find out that there is a order with data group, channel group and channels. Can you tell me how to write a python-script that takes a pandas.DataFrame or dictionary and save it to mdf4 in this special hierarchy? It will be good if you have an example for me.

Regards, Stefan

[danielhrisca]

from asammdf import MDF

mdf = MDF()
mdf.append(pandas_data_frame)
mdf.save('output.mf4')

[StefanLaube96]

Hello,

I already tried it like this but I only get the order up to channel group. How can I get the data group (one stage higher)?

That means I want to save more than one channnel group in one data group.

When I´m looking in mdfviewer I can see the channel groups only.

[StefanLaube96]

For example, I want to order channel group 0 and channel group 1 in data group 1.

[danielhrisca]

Download MDFValidator from Vector https://www.vector.com/int/en/search/?tx_solr[filter][0]=categoryContent%3A7&tx_solr[filter][1]=categoryStandard%3A54 too see exactly all the blocks from the file

Our GUI doesn't display the data groups, but for each Channel group that you see there is also a data group.

[StefanLaube96]

Okay I installed the MDFValidator.

Now, every channel group has an own data group. Is there the possibility that more than one channel group is in one data group?

How can I change the names of data/channel group? ('Channel group 0' is not really a meaningful name)

Can I edit the 'Meta Data Block'?

[danielhrisca]

Please have a look at the specification for MDF v 3.3 available with the mdfvaldiator and check out the difference between sorted and unsorted files (pages 27 and 28)

The main ideas are very similar for MDF v4.

[StefanLaube96]

I think you misunderstand me. I want to know if there is a comand in python to built these data groups.

The automatic creation is not my aim.

And can I change the names of data/channel group? ('Channel group 0' is not really a meaningful name)

[danielhrisca]

I want to know if there is a comand in python to built these data groups.

there is no direct way to do this, you will need to look into the append method and see how to add new group object to the mdf.groups (see https://github.com/danielhrisca/asammdf/blob/master/asammdf/blocks/mdf_v4.py#L2903) and properly populate it

And can I change the names of data/channel group? ('Channel group 0' is not really a meaningful name)

just set the channel group comment

mdf.groups[gp_index].channel_group.comment = 'meaningul description'

[danielhrisca]

The automatic creation is not my aim.

What is it that you want to do?

[StefanLaube96]

1.) Does it means I need to change your code for my problem? I just want to write some channels in one datagroup and maybe show it with MDFValidator.

I found this lines:

ch = Channel(**kwargs)
ch.unit = time_unit
ch.name = time_name
ch.source = source_block
ch.dtype_fmt = t.dtype
name = time_name

gp_channels.append(ch)

(https://github.com/danielhrisca/asammdf/blob/master/asammdf/blocks/mdf_v4.py#L2986) This command appends channles to the datagroup. Is it right? That means I have to manipulate 'ch'.

2.) Unfortunaly I don't get what I want when I use the comment command: My code:

for key, channels in data_dic.items():                          
    mdf4.append(channels, 'Created by python')             
    mdf4.groups[i].channel_group.comment = key
    mdf4.save(filename, overwrite=True)                         
    i += 1

I want the text in brackets only. Without 'channel group 0'.

[danielhrisca]

1.) Does it means I need to change your code for my problem? I just want to write some channels in one datagroup and maybe show it with MDFValidator.

I really don't understand why you would need this. The code snippet that you have used for the item 2 is all that is necessary.

I want the text in brackets only. Without 'channel group 0'.

It is just the way the groups are displayed in the GUI, and in parenthesis you see the channel group comment if it is available. Open your file with MDFValidator and you will see something similar.

[danielhrisca]

for i, (key, channels) in enumerate(data_dic.items()):                          
    mdf4.append(channels, 'Created by python')             
    mdf4.groups[i].channel_group.comment = key
mdf4.save(filename, overwrite=True)  

[StefanLaube96]

My output on MDFValidator:

I want 'datagroup 1' and 'datagroup 2' as one 'datagroup CA'. Now, all the channel names are 'Channel Group 1' even though I change my code as you wrote.

[danielhrisca]

I want 'datagroup 1' and 'datagroup 2' as one 'datagroup CA'. Now, all the channel names are 'Channel Group 1' even though I change my code as you wrote.

You are a bit confused: "Channel Group 1" is not an actual text in the file, it is how the GUI tool decides to name/enumerate the channel groups. As you see, additionaly compared to MDFValdiator, in the asammdf GUI you can also see the channel group comment as a friendly way to identify the channel groups.

When you call append with a list of channels, then a new data group and channel groups will be created and all the channels will be added to the channel group. If you want to have the channel from channel groups 1 and 2 together, then you have to call append with a list that contains all those channels.

[StefanLaube96]

Okay thats means I cannot give names to the groups because the 'names' only exist in the viewer. I can only comment it for a better clarity.

And I cannot append two datagroups after creating the mf4-file. I need to sort them before.

Am I right?

[danielhrisca]

And I cannot append two datagroups after creating the mf4-file. I need to sort them before.

If you meant to write And I cannot append TO datagroups after creating the mf4-file. I need to sort them before. then no, you cannot append new channels to an existing channel group.

[StefanLaube96]

Oh yes sorry.

I think I understand how to solve my problem now. I will try it again.

[StefanLaube96]

Is there a possibility to append a list of pandas.DataFrames to mdf4?

data_dic = {key:{name_dict1: pd.DataFrame({base_name: base, name_df: data, ...}),name_dict2: pd.DataFrame({base_name: base, name_df: data, ...}) ...}, ...}

with MDF(version='4.10') as mdf4:
for i, (key, value) in enumerate(data_dic.items()):
    lst = []
    for k, val in value.items():
        lst.append(val)
    mdf4.append(lst, 'Created by python')
    mdf4.groups[i].channel_group.comment = key
mdf4.save(filename, overwrite=True)

I got the error: AttributeError: 'DataFrame' object has no attribute 'timestamps'

[danielhrisca]

Hello Stefan,

you can't append a list of DataFrames (see the documentation for the append method https://asammdf.readthedocs.io/en/latest/api.html#mdf4)

[StefanLaube96]

Hello,

I don´t really solve the problem, but I am using an other method. I give every column in the DataFrames names in form of 'datakey_channelname'.

With the 'comment'-command I give the channel groups a comment. Is there a way to get this comments while reading the mf4-file? I want to read different channel groups as DataFrames.

I am not sure if I have to use "to_dataframe" or "get_group". While using "to_dataframe" I get all channels in one DataFrame. Thats problematic beacuse not all columns have the same length (thats why they are saved in different DataFrames). While using "get_groups" I need to know how much groups are saved in the file. (Or maybe the index only) Can I read all groups seperated with one of these commands?

[danielhrisca]

dfs =[df for df in mdf.iter_groups()]

[danielhrisca]

@StefanLaube96 can we close this issue?

[StefanLaube96]

Yes. Thank you for the fast and professionel help!