I-PDU-Multiplexing within CONTAINER-I-PDU not working as expected

stefan325 commented 5 years ago

Hi @ebroecker ,

thanks for your support earlier. I faced some issues with multiplexed PDUs within one CAN FD Frame. To be more precise, the issue is with "7.3 Multiple-PDU-to-Container handling", as described in[ Specification of I-PDU Multiplexer] https://www.autosar.org/fileadmin/user_upload/standards/classic/4-3/AUTOSAR_SWS_IPDUMultiplexer.pdf) on page 27 of 87. I am describing the issue as precisely as I can in this ticket, sorry that the post is a bit long.

I have two I-PDUs, Pdu1 and Pdu2 which are transmitted on the same frame (CAN ID) and occupy the same bit-positions within that frame. They belong to a different multiplexed I-PDU. The AUTOSAR concept behind this is a CONTAINER-I-PDU which contains several I-PDUs. I am looking at this in the JSON format first, as it offers the easiest way to understand the internal data representation.

The issue is: In JSON export, I see that several signal of the same frame have the same bit position / start Bit (that same issue applies to the DBC export, too):

"signals": [
    {
        "bit_length": 1,
        "factor": "1",
        "is_big_endian": false,
        "is_float": false,
        "is_signed": true,
        "name": "Pdu1_Signal1",
        "offset": "0",
        "start_bit": 0
    },
    {
        "bit_length": 1,
        "factor": "1",
        "is_big_endian": false,
        "is_float": false,
        "is_signed": true,
        "name": "Pdu2_Signal1",
        "offset": "0",
        "start_bit": 0
    },

The desired behavior would be that the parser recognizes that those signals are within the same frame, but belong to a different multiplexed I-PDU / different I-PDU Header. I guess I would want something like this:

"signals": [
    {
        "bit_length": 1,
        "factor": "1",
        "is_big_endian": false,
        "is_float": false,
        "is_signed": true,
        "name": "Pdu1_Signal1",
        "offset": "0",
        "start_bit": 0,
        "header_id" : 0x310000
    },
    {
        "bit_length": 1,
        "factor": "1",
        "is_big_endian": false,
        "is_float": false,
        "is_signed": true,
        "name": "Pdu2_Signal1",
        "offset": "0",
        "start_bit": 0,
        "header_id" : 0x320000
    },

That can also be reflected in the DBC file, but I guess we would need to solve the parser first and then think about how to potentially reflect that multiplexing correctly in DBC format (or other formats). I am aware that DBC multiplexor/multiplexed signal feature has limitations (just one multiplexor per frame allowed).

The AUTOSAR mechanism in my ARXMl file is the following:

There is a ContainerIPdu. The ContainerIPdu defines which IPdus may be collected inside that ContainerIPdu (ContainerIPdu.containedPduTriggering).
Each IPdu which shall be collected inside a ContainerIPdu shall provide the IPdu.containedIPduProps. This ContainedIPduProps defines a unique header Id (page 241 of AUTOSAR_TPS_SystemTemplate.pdf)
The way the resulting frame looks like is nicely described in 7.3.1 Dynamic Container Layout of Specification of I-PDU Multiplexer. Bepoke unique header is somewhat similar to the DBC multiplexor of multiplexed messages. The final Frame shall contain -- 24 Bit I-PDU Header ID -- 8 Bit DLC -- The I-PDU

What do you think? Could such a feature be added to the parser?

I see that the code is already doing something with CONTAINER-I-PDUs (branch iss382): arxml.py, def get_frame, line 1174ff

    if pdu.tag == ns + "CONTAINER-I-PDU":
        pdus = get_children(pdu, "CONTAINED-PDU-TRIGGERING", root_or_cache, ns)

Called CONTAINED-PDU-TRIGGERING-REFS in AUTOSAR 4.3, not CONTAINED-PDU-TRIGGERING. Does that matter?

        signal_group_id = 1
        singnals_grouped = []
        for pdu in pdus:
            ipdu = get_child(pdu, "I-PDU", root_or_cache, ns)
            # pdu_sig_mapping = get_children(ipdu, "I-SIGNAL-IN-I-PDU", root_or_cache, ns)
            pdu_sig_mapping = get_children(ipdu, "I-SIGNAL-TO-I-PDU-MAPPING", root_or_cache, ns)

            # TODO
            if pdu_sig_mapping:
                get_signals(pdu_sig_mapping, new_frame, root_or_cache, ns, None, float_factory)
                new_signals = []
                for signal in new_frame:

I guess here, we'd need to add I-PDU's attribute HEADER-ID-SHORT-HEADER

                    if signal.name not in singnals_grouped:
                        new_signals.append(signal.name)
                new_frame.add_signal_group(get_element_name(pdu, ns), signal_group_id, new_signals)
                singnals_grouped += new_signals
                signal_group_id += 1

Example ARXML

xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://autosar.org/schema/r4.0" xsi:schemaLocation="http://autosar.org/schema/r4.0 AUTOSAR_4-3-0.xsd"

<!-- /Pdus -->
        <AR-PACKAGE UUID="11">
            <SHORT-NAME>Pdus</SHORT-NAME>
            <AR-PACKAGES>
                <!-- /Pdus/xxx -->
                <AR-PACKAGE UUID="22">
                    <SHORT-NAME>xxx</SHORT-NAME>
                    <AR-PACKAGES>
                        <!-- /Pdus/xxx/yyy -->
                        <AR-PACKAGE UUID="22">
                            <SHORT-NAME>yyy</SHORT-NAME>
                            <ELEMENTS>
                                <!-- "Normal" I PDUs-->
                                <I-SIGNAL-I-PDU ... />
                                <I-SIGNAL-I-PDU ... />
                                <I-SIGNAL-I-PDU ... />
                                <SECURED-I-PDU ... />
                            </ELEMENTS>
                            <!-- multiplexed IPDUs come in a AR-PACKAGE -->
                            <AR-PACKAGES>
                                <!-- /Pdus/xxx/yyy/container1 -->
                                <AR-PACKAGE UUID="333">
                                    <SHORT-NAME>container1</SHORT-NAME>
                                    <ELEMENTS>
                                        <!-- The first multiplexed I PDU -->
                                        <I-SIGNAL-I-PDU T="2019-07-05T07:11:01+01:00" UUID="AAAE">
                                            <SHORT-NAME>Container1_Pdu1</SHORT-NAME>    
                                            <LENGTH>8</LENGTH>                                          
                                            <CONTAINED-I-PDU-PROPS>
                                                <COLLECTION-SEMANTICS>LAST-IS-BEST</COLLECTION-SEMANTICS>
                                                <HEADER-ID-SHORT-HEADER>0x310000</HEADER-ID-SHORT-HEADER>
                                                <TRIGGER>NEVER</TRIGGER>
                                            </CONTAINED-I-PDU-PROPS>                                            
                                            ...
                                            <I-SIGNAL-TO-PDU-MAPPINGS>
                                                <I-SIGNAL-TO-I-PDU-MAPPING UUID="1233">
                                                    <SHORT-NAME>Pdu1_Signal1</SHORT-NAME>
                                            ...
                                        </I-SIGNAL-I-PDU>
                                        <!-- The second multiplexed I PDU -->
                                        <I-SIGNAL-I-PDU T="2019-01-05T07:12:01+01:00" UUID="AAAF">
                                            <SHORT-NAME>Container1_Pdu2</SHORT-NAME>    
                                            <LENGTH>24</LENGTH>
                                            <CONTAINED-I-PDU-PROPS>
                                                <COLLECTION-SEMANTICS>LAST-IS-BEST</COLLECTION-SEMANTICS>
                                                <HEADER-ID-SHORT-HEADER>0x320000</HEADER-ID-SHORT-HEADER>
                                                <TRIGGER>NEVER</TRIGGER>
                                            </CONTAINED-I-PDU-PROPS>                                            
                                            ...
                                            <I-SIGNAL-TO-PDU-MAPPINGS>
                                                <I-SIGNAL-TO-I-PDU-MAPPING UUID="1233">
                                                    <SHORT-NAME>Pdu2_Signal1</SHORT-NAME>
                                            ...
                                        </I-SIGNAL-I-PDU>       
                                        <!-- The container I PDU which contains the above 2 in a multiplexed fashion -->
                                        <CONTAINER-I-PDU T="2019-03-11T01:05:31+01:00" UUID="1111">
                                            <SHORT-NAME>Container1_IPdu</SHORT-NAME>
                                            <LENGTH>64</LENGTH>
                                            <HEADER-TYPE>SHORT-HEADER</HEADER-TYPE>
                                            <CONTAINED-PDU-TRIGGERING-REFS>
                                                <CONTAINED-PDU-TRIGGERING-REF DEST="PDU-TRIGGERING">/CommunicationClusters/xxx/container1pdu1triggering</CONTAINED-PDU-TRIGGERING-REF>
                                                <CONTAINED-PDU-TRIGGERING-REF DEST="PDU-TRIGGERING">/CommunicationClusters/xxx/container1pdu2triggering</CONTAINED-PDU-TRIGGERING-REF>
                                            </CONTAINED-PDU-TRIGGERING-REFS>
                                        </CONTAINER-I-PDU>
                                    </ELEMENTS>

<!-- /CommunicationClusters/xxx -->
                                <!-- Links the container I PDU with PDU 1 -->
                                <PDU-TRIGGERING UUID="111">
                                    <SHORT-NAME>container1pdu1triggering</SHORT-NAME>
                                    <I-PDU-PORT-REFS>
                                            ...
                                    </I-PDU-PORT-REFS>
                                    <I-PDU-REF DEST="I-SIGNAL-I-PDU">/Pdus/Container1_Pdu1</I-PDU-REF>
                                    <I-SIGNAL-TRIGGERINGS>
                                    </I-SIGNAL-TRIGGERINGS>
                                </PDU-TRIGGERING>
                                <!-- Links the container I PDU with PDU 2 -->
                                <PDU-TRIGGERING UUID="111">
                                    <SHORT-NAME>container1pdu2triggering</SHORT-NAME>
                                    <I-PDU-PORT-REFS>
                                            ...
                                    </I-PDU-PORT-REFS>
                                    <I-PDU-REF DEST="I-SIGNAL-I-PDU">/Pdus/Container1_Pdu2</I-PDU-REF>
                                    <I-SIGNAL-TRIGGERINGS>
                                    </I-SIGNAL-TRIGGERINGS>
                                </PDU-TRIGGERING>

References: Specification of I-PDU Multiplexer https://www.autosar.org/fileadmin/user_upload/standards/classic/4-3/AUTOSAR_SWS_IPDUMultiplexer.pdf AUTOSAR_TPS_SystemTemplate.pdf https://www.autosar.org/fileadmin/user_upload/standards/classic/4-3/AUTOSAR_TPS_SystemTemplate.pdf page 241ff, section 6.3.1

ebroecker commented 5 years ago

@stefan325 Thanks for your detailed description. These answers the open questions from #363

I did a first hack implementing these header-information and transposing to multiplex-info. It's in branch iss385.

the generated dbc does not look to bad

stefan325 commented 5 years ago

Hello @ebroecker ,

thanks again for the quick answer. I tried your branch and got it to work nicely. I did one minor correction, since the header ID is given in format 0x0000, it can't be directly casted to integer. Script aborts throwing an error in line 1345 in arxml.py, since it can't execute int(header_id), since headerid is a hex string. This following fix works: # need to convert headerid from hex-format to int

get_signals(pdu_sig_mapping, new_frame, root_or_cache, ns, int(str(header_id), 16), float_factory, bit_offset=header_length)

If you think it could also be given in decimal (I think it is always hex string), we can do

if str(header_id).find("0x") > -1:
...
else:
...

The generated DBC looks nice, indeed. I am using CANdb++ - the issue with that is that it does not allow any CAN Frame to have a DLC > 8 Byte, since it is assuming Standard CAN Frames, as opposed to Standard CAN FD Frames. So when I open a message, the tool complains that the DLC is invalid.

I fixed this by manually changing the DBC file (left is the new version, right the original). I did this by just creating a new empty CANFD dbc from Template in CANdb++ and looked for differences. This way I got it to work and display the multiplexed messages:

However, that is not quite right yet, since the extended ID CAN Frames should be CAN FD Extended. Do you have any knowledge about the CAN Frame Type in DBC format? I guess we must define the Message Type for each message individually somehow.

Alright, I will go ahead and test the generated DBC file againt CAN Traces and verify the correct CONTAINER PDU Handling next.

Thanks

ebroecker commented 5 years ago

@stefan325 I have the same issue with candb++ I hacked some special support for candb++ in 305fba3

LONG-HEADER is supported also, only NONE-TYPE header not supported yet

EDIT: frames should be FD now in dbc

stefan325 commented 5 years ago

@ebroecker Thanks, great stuff.

ebroecker commented 5 years ago

@stefan325 I close this issue. branch is merged to development now.

stefan325 commented 5 years ago

Great, thank you.

ebroecker / canmatrix

I-PDU-Multiplexing within CONTAINER-I-PDU not working as expected #385