Open-CMSIS-Pack / Open-CMSIS-Pack-Spec

Common Microcontroller Software Interface Standard - Pack(age) based distribution system
https://open-cmsis-pack.github.io/Open-CMSIS-Pack-Spec/
Apache License 2.0
54 stars 22 forks source link

[GPDSC Spec clarification]: Do "csolution" uses the GPDSC file content to override the PDSC definition of components ? #317

Open BadreddineBJA opened 6 months ago

BadreddineBJA commented 6 months ago

GPDSC Spec clarification

1. Assumptions:

2. Scenario example description:

a. Use a packA PDSC definition like the following:

<?xml version="1.0" encoding="UTF-8" ?>
<package xmlns:xs="http://www.w3.org/2001/XMLSchema-instance" schemaVersion="1.7.2" xs:noNamespaceSchemaLocation="https://github.com/Open-CMSIS-Pack/Open-CMSIS-Pack-Spec/blob/main/schema/PACK.xsd">
    <vendor>STMicro</vendor>
    <name>packA</name>
    <description>Pack A description</description>
    <generators>
        <generator id="GEN-1">
            <description>GEN1 generator</description>
            <exe>
                <command host="win">gen.bat</command>
                <command host="linux">gen.sh</command>
                <!-- Generator Id -->
                <argument>GEN-1</argument>
                <!-- Generator Input File absolute path -->
                <argument>$G</argument>
                <!-- GPDSC Template File Name -->
                <argument>template_model.gpdsc.hbs</argument>
            </exe>
            <gpdsc name="$P/template_model.gpdsc"/>
        </generator>
    </generators>
    <components>
        <component Cclass="Device" Cgroup="generated" Csub="compA" Cversion="0.0.1" generator="GEN-1">
            <description>compA description</description>
            <RTE_Components_h>
                <!-- the following content goes into file 'RTE_Components.h' -->
                #define _RTE_COMPA_FROM_PDSC
            </RTE_Components_h>
            <Pre_Include_Global_h>
                <!-- the following content goes into file 'Pre_Include_Global.h' -->
                #define PRE_INCLUDE_GLOBAL_COMPA_FROM_PDSC
            </Pre_Include_Global_h>
            <Pre_Include_Local_Component_h>
                <!-- the following content goes into file 'Pre_Include_Local_<ComponentId>.h' -->
                #define PRE_INCLUDE_LOCAL_COMPA_FROM_PDSC
            </Pre_Include_Local_Component_h>
            <files>
                <file category="genAsset" name="templates/compA/stm32_compA_template.c"/>
                <file category="genAsset" name="templates/compA/stm32_compA_template.h"/>
            </files>
        </component>
    </components>
</package>

b. Generate a GPDSC file with the following content:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!-- ******************************************************************************
 * File Name   : GPDSC_template.gpdsc
 * Description : Generator PDSC File generated (DO NOT EDIT!)
 ****************************************************************************** -->

<package xmlns:xs="http://www.w3.org/2001/XMLSchema-instance" schemaVersion="1.0"
    xs:noNamespaceSchemaLocation="https://github.com/Open-CMSIS-Pack/Open-CMSIS-Pack-Spec/blob/main/schema/PACK.xsd">
    <vendor>STMicro</vendor>
    <name>packA</name>
    <description>Pack A description</description>
    <url>path</url>
    <generators>
        <generator id="GEN-1">
            <description>GEN1 generator</description>
            <exe>
                <command host="win">gen.bat</command>
                <command host="linux">gen.sh</command>
                <!-- Generator Id -->
                <argument>GEN-1</argument>
                <!-- Generator Input File absolute path -->
                <argument>$G</argument>
                <!-- GPDSC Template File Name -->
                <argument>template_model.gpdsc.hbs</argument>
            </exe>
            <gpdsc name="$P/template_model.gpdsc" />
        </generator>
    </generators>
    <components>
        <component Cclass="Device" Cgroup="generated" Csub="compA" Cversion="1.0.0" generator="GEN-1">
            <description>compA description</description>
            <RTE_Components_h>
                <!-- the following content goes into file 'RTE_Components.h' -->
                #define USING_CompA_GPDSC
            </RTE_Components_h>
            <Pre_Include_Global_h>
                <!-- the following content goes into file 'Pre_Include_Global.h' -->
                #define PRE_INCLUDE_GLOBAL_compA_GPDSC
            </Pre_Include_Global_h>
            <Pre_Include_Local_Component_h>
                <!-- the following content goes into file 'Pre_Include_Local_<ComponentId>.h' -->
                #define PRE_INCLUDE_LOCAL_compA_GPDSC
            </Pre_Include_Local_Component_h>
            <files>
                <file category="source" name="stm32_compA.c" />
                <file category="header" name="stm32_compA.h" />
            </files>
        </component>
    </components>    
</package>

3. Observed Result: The content of the RTE_ Generated files is the PDSC list of defines, the GPDSC definition of , , for the same component is not taken into consideration by csolution.

_RTEComponents.h generated file:

/*
 * CSOLUTION generated file: DO NOT EDIT!
 * Generated by: csolution version 2.0.0
 * Target:  'debug+main' 
 */
#ifndef RTE_COMPONENTS_H
#define RTE_COMPONENTS_H
/*
 * Define the Device Header File: 
 */
#define CMSIS_device_header "stm32u5xx.h"

/* STMicro::Device:generated:compA:0.0.1 */
#define _RTE_COMPA_FROM_PDSC**

#endif /* RTE_COMPONENTS_H */

_Pre_Include_Global.h_

/*
 * CSOLUTION generated file: DO NOT EDIT!
 * Generated by: csolution version 2.0.0
 * Target:  'debug+main' 
 */
#ifndef PRE_INCLUDE_GLOBAL_H
#define PRE_INCLUDE_GLOBAL_H
/* STMicro::Device:generated:compA:0.0.1 */
#define PRE_INCLUDE_GLOBAL_COMPA_FROM_PDSC**
#endif /* PRE_INCLUDE_GLOBAL_H */

__Pre_Include_Local_compId.h__

/*
 * CSOLUTION generated file: DO NOT EDIT!
 * Generated by: csolution version 2.0.0
 * Target:  'debug+main' 
 */

#ifndef PRE_INCLUDE_DEVICE_GENERATED_COMPA_H
#define PRE_INCLUDE_DEVICE_GENERATED_COMPA_H

/* STMicro::Device:generated:compA:0.0.1 */
#define PRE_INCLUDE_LOCAL_COMPA_FROM_PDSC

#endif /* PRE_INCLUDE_DEVICE_GENERATED_COMPA_H */

4. What is expected ?

BadreddineBJA commented 5 months ago

Tests re-executed using csolution v2.3.0, I get the same result as csolution v2.0.0, only the PDSC defines are taken into consideration, GPDSC declared defines are not taken into account at the convert step

_RTEComponents.h generated file:

/*
 * CSOLUTION generated file: DO NOT EDIT!
 * Generated by: csolution version 2.3.0
 *
 * Project: 'app6_SW.debug+main' 
 * Target:  'debug+main' 
 */

#ifndef RTE_COMPONENTS_H
#define RTE_COMPONENTS_H

/*
 * Define the Device Header File: 
 */
#define CMSIS_device_header "stm32u5xx.h"

/* STMicro::Device:generated:compA@0.0.1 */
#define _RTE_COMPA_FROM_PDSC

#endif /* RTE_COMPONENTS_H */

_Pre_Include_Global.h_

/*
 * CSOLUTION generated file: DO NOT EDIT!
 * Generated by: csolution version 2.3.0
 *
 * Project: 'app6_SW.debug+main' 
 * Target:  'debug+main' 
 */

#ifndef PRE_INCLUDE_GLOBAL_H
#define PRE_INCLUDE_GLOBAL_H

/* STMicro::Device:generated:compA@0.0.1 */
#define PRE_INCLUDE_GLOBAL_COMPA_FROM_PDSC

#endif /* PRE_INCLUDE_GLOBAL_H */

__Pre_Include_Local_compId.h__

/*
 * CSOLUTION generated file: DO NOT EDIT!
 * Generated by: csolution version 2.3.0
 *
 * Project: 'app6_SW.debug+main' 
 * Target:  'debug+main' 
 */

#ifndef PRE_INCLUDE_DEVICE_GENERATED_COMPA_H
#define PRE_INCLUDE_DEVICE_GENERATED_COMPA_H

/* STMicro::Device:generated:compA@0.0.1 */
#define PRE_INCLUDE_LOCAL_COMPA_FROM_PDSC

#endif /* PRE_INCLUDE_DEVICE_GENERATED_COMPA_H */

Observed behavior:

GPDSC definition of the same component is not overriding the PDSC component element.

Extra tests:

When updating the pack vendor name of the GPDSC template, or the component cgroup like displayed in the snapshot:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!-- ******************************************************************************
 * File Name   : GPDSC_template.gpdsc
 * Description : Generator PDSC File generated by CodeGen (DO NOT EDIT!)
 ****************************************************************************** -->

<package xmlns:xs="http://www.w3.org/2001/XMLSchema-instance" schemaVersion="1.0"
    xs:noNamespaceSchemaLocation="https://github.com/Open-CMSIS-Pack/Open-CMSIS-Pack-Spec/blob/main/schema/PACK.xsd">
    <vendor>STMicro</vendor>
    <name>packA</name>
    <description>Pack A description</description>
    <url>path</url>
    <releases>
        <release version="0.0.1">
            {{lowercase "- First GPDSC template Model Release"}}
        </release>
    </releases>
    <conditions>
        <condition id="conditionC">
            <description>condition C</description>
            <require Cclass="fake" Cgroup="generated" Csub="compF" />
        </condition>
        <condition id="filecondition1">
            <description>Condition 1</description>
            <require Cclass="fake" Cgroup="generated" Csub="compB" />
        </condition>
        <condition id="filecondition2">
            <description>Condition 2</description>
            <require Cclass="fake" Cgroup="generated" Csub="compC" />
        </condition>
    </conditions>
    <generators>
        <generator id="GEN-1">
            <description>GEN1 generator</description>
            <exe>
                <command host="win">gen.bat</command>
                <command host="linux">gen.sh</command>
                <!-- Generator Id -->
                <argument>GEN-1</argument>
                <!-- Generator Input File absolute path -->
                <argument>$G</argument>
                <!-- GPDSC Template File Name -->
                <argument>template_model.gpdsc.hbs</argument>
                <!-- Flag to indicate that this Generator is dry-run capable  -->
                <argument mode="dry-run">--dry-run</argument>
            </exe>
            <gpdsc name="$P/template_model.gpdsc" />
        </generator>
    </generators>
    {{!-- SELECTED COMPONENTS LIST --}}

    <components>
        <component Cclass="Device" Cgroup="generated-updated" Csub="compA" Cversion="1.0.0" generator="GEN-1">
            <description>TO BE DEFINED</description>
            <RTE_Components_h>
                <!-- the following content goes into file 'RTE_Components.h' -->
                #define USING_CompA_GPDSC
            </RTE_Components_h>
            <Pre_Include_Global_h>
                <!-- the following content goes into file 'Pre_Include_Global.h' -->
                #define PRE_INCLUDE_GLOBAL_compA_GPDSC
            </Pre_Include_Global_h>
            <Pre_Include_Local_Component_h>
                <!-- the following content goes into file 'Pre_Include_Local_<ComponentId>.h' -->
                #define PRE_INCLUDE_LOCAL_compA_GPDSC
            </Pre_Include_Local_Component_h>
            <files>
                <file category="source" name="stm32_compA.c" />
                <file category="header" name="stm32_compA.h" />
            </files>
        </component>
    </components>    
</package>

the new updated component description is taken into consideration this is what we get in the RTE_Components.h file

/*
 * CSOLUTION generated file: DO NOT EDIT!
 * Generated by: csolution version 2.3.0
 *
 * Project: 'app6_SW.debug+main' 
 * Target:  'debug+main' 
 */

#ifndef RTE_COMPONENTS_H
#define RTE_COMPONENTS_H

/*
 * Define the Device Header File: 
 */
#define CMSIS_device_header "stm32u5xx.h"

/* STMicro::Device:generated-updated:compA@1.0.0 */
#define USING_CompA_GPDSC
/* STMicro::Device:generated:compA@0.0.1 */
#define _RTE_COMPA_FROM_PDSC

#endif /* RTE_COMPONENTS_H */

Main Question

jkrech commented 5 months ago

I reproduced a similar scenario in MDK uVision 5.40 and do see the expected behavior, that the Cversion attribute of the component definition in the GPDSC file needs to be larger or equal to the component version in the PDSC file to override it. This includes all aspects of the component including RTE_Components.h, Global and Local Pre-Include definitions.

I converted the uvprojx to csolution and repeated the test and I can see the defines from GPDSC if the component defined in the gpdsc is greater or equal to the Cversion of the component specified in the PDSC file. Otherwise I see the defines coming from the PDSC I used CMSIS-Toolbox 2.4.0 and 2.3.0 but was unable to reproduce the behavior you are describing above.

BadreddineBJA commented 5 months ago

Thanks Joachim for the tests, I will re-check the tests following your described scenario about the component version in the GPDSC that should be greater or equal to the PDSC version and confirm the behavior. i used the csolution "v2.3.0" from the toolbox on "windows" for previous described scenarios,

BadreddineBJA commented 5 months ago

Following the above recommendations from @jkrech (The Cversion attribute of the component definition in the GPDSC file needs to be larger or equal to the component version in the PDSC file to override it), at the convert step, i can see in the RTE folder that the defines from GPDSC are taken into consideration if the component defined in the gpdsc is greater or equal to the Cversion of the component specified in the PDSC file. Otherwise I see the defines coming from the PDSC.

. I think that this behavior should be documented in the GPDSC spec

For the rest of the applied tests, the GPDSC template manual editing is very error prone since it can cause GPDSC .xsd schema inconsistancies which seems implicitly not detected by csolution (maybe warnings messages to the end user should be added to ease the debug step)

BadreddineBJA commented 5 months ago

Hi @jkrech,

Thank you for your feedback Regarding your question, Do you make use of config files in your component descriptions for the generator component and the overwrite in the generated gpdsc?

Following our discussion about the strange behaviors observed and described in this ticket, I re-adjusted the GPDSC template following your recommendation:

After following this scenario:

  1. Compose a project using 2 components: compA, compB .
  2. Generate a GPDSC file with the same PDSC definition of this components (Cclass, Cgroup, Cversion,..) (Same exact version).
  3. Apply a conversion step: ‘csolution convert -s .csolution.yml)

Observed results:

this would confirm our assumption that every PDSC element described in the spec (https://open-cmsis-pack.github.io/[Open-CMSIS-Pack](https://open-cmsis-pack.github.io/Open-CMSIS-Pack-Spec/main/html/index.html)-Spec/main/html/index.html) is supported via the GPDSC.

jkrech commented 5 months ago

Summary: