Maintenance planner downloads: Deselected files are still downloaded

[MatthiasWinzeler]

Hi folks

First of all, many thanks for providing this library! It is an incredibly valuable tool when it comes to automating our SAP Basis deployment pipeline.

I think we found a minor bug in community.sap_launchpad.maintenance_planner_files - it returns files for a download that were deselected in the maintenance planner.

How to reproduce

1. Access the SAP maintenance planner

2. Click on "Plan a new system"

3. Click on "Plan"

4. Choose a SID and "Install an SAP S/4HANA system"

5. Select some release, i.e. "SAP S/4HANA 2022" with FPS 02, confirm selection

6. Go to "Select Files", select Linux X64, confirm selection

7. If you expand "Select Stack Independent Files" & "NON-ABAP", you will notice that all language stacks other than German (DE) and English (EN) are unselected:

   

8. When going further to "Download Files", you will see that only German (DE) and English (EN) are selected for download:

   

9. We finalize the MP part by pushing the downloads to the download basket (not sure this part is necessary)

Now, when listing the files to download with this module, it will not only tell us to download the language packs DE & EN, but all available language packs, even though they were not selected in the MP. I think this is a bug:

...
    - name: Execute Ansible Module 'maintenance_planner_files' to get files from MP
      community.sap_launchpad.maintenance_planner_files:
        suser_id: "{{ suser_id }}"
        suser_password: "{{ suser_password }}"
        transaction_name: "{{ transaction_name }}"
      register: sap_maintenance_planner_basket_register

    - name: Debug
      ansible.builtin.debug:
        var: sap_maintenance_planner_basket_register
        verbosity: 2
...

This will output all language files, even though deselected:

...
    "sap_maintenance_planner_basket_register": {
        "changed": true,
        "download_basket": [
...
            {
                "DirectLink": "https://softwaredownloads.sap.com/file/0030000001312922022",
                "Filename": "S4HANAOP107_ERP_LANG_SL.SAR"
            },
            {
                "DirectLink": "https://softwaredownloads.sap.com/file/0030000001312932022",
                "Filename": "S4HANAOP107_ERP_LANG_SV.SAR"
            },
            {
                "DirectLink": "https://softwaredownloads.sap.com/file/0030000001312942022",
                "Filename": "S4HANAOP107_ERP_LANG_TH.SAR"
            },
            {
                "DirectLink": "https://softwaredownloads.sap.com/file/0030000001312962022",
                "Filename": "S4HANAOP107_ERP_LANG_TR.SAR"
            },
            {
                "DirectLink": "https://softwaredownloads.sap.com/file/0030000001312972022",
                "Filename": "S4HANAOP107_ERP_LANG_UK.SAR"
            },
            {
                "DirectLink": "https://softwaredownloads.sap.com/file/0030000001312982022",
                "Filename": "S4HANAOP107_ERP_LANG_VI.SAR"
            },
            {
                "DirectLink": "https://softwaredownloads.sap.com/file/0030000001312992022",
                "Filename": "S4HANAOP107_ERP_LANG_ZF.SAR"
            },
            {
                "DirectLink": "https://softwaredownloads.sap.com/file/0030000001313002022",
                "Filename": "S4HANAOP107_ERP_LANG_ZH.SAR"
...

To summarize, it appears that the module returns all software packages from the stack, even deselected ones.

I dug a bit deeper into the code and saw that the API call that return the files above returns an XML which also contains the deselected files:

<?xml version="1.0" encoding="utf-16"?>
<mnp:response xmlns:mnp="http://xml.sap.com/2012/01/mnp">
...
 <mnp:entities>
        <mnp:entity id="selectFiles" step="downloadFiles" label="Download Files">
            <mnp:entity id="downloadStackFiles" label="Download Stack Files">
                <mnp:entity label="List of files selected for download" id="stack_files"
                            type="Download Stack Independent Files" enable="false">
                    <mnp:entity id="0010000000033172015" label="GBX01HR5605.SAR" type="REG" size="5"
                                selection_type="multiple" selected="true"
                                description="Attribute Change Package 28 for GBX01HR5 605" enable="false"/>
....
                    <mnp:entity id="0030000001312572022" label="S4HANAOP107_ERP_LANG_CS.SAR" type="STL" size="687978"
                                selection_type="multiple" selected="true" description="File on DVD" enable="false"/>
                    <mnp:entity id="0030000001312582022" label="S4HANAOP107_ERP_LANG_DA.SAR" type="STL" size="475621"
                                selection_type="multiple" selected="true" description="File on DVD" enable="false"/>
                    <mnp:entity id="0030000001312472022" label="S4HANAOP107_ERP_LANG_DE.SAR" type="STL" size="292015"
                                selection_type="multiple" selected="true" description="File on DVD" enable="false"/>
...

As you can see, both S4HANAOP107_ERP_LANG_DE (which was selected in the MP GUI) as well as S4HANAOP107_ERP_LANG_DA.SAR (which was unselected in the MP planner) have selected=true. If the clients are supposed to respect this field, then it might even be a bug in the SAP backend. I am not sure this can easily be fixed, but maybe you folks have a good internal contact working on this part that can take a look?

FWIW, I also checked out the SAP Download Manager (which I assume is the official client for the MP) and it correctly returns only the selected files, but I think it uses a slightly different API than the XML above.

[sean-freeman]

@MatthiasWinzeler is this sample XML from after you have also clicked [Save] ?

As you debugged, the XML element mnp:entity has the XML attribute selected="true". However, when this XML is returned to the Ansible Module maintenance_planner_files there is currently no handling in the underlying Python Module function get_transaction_filename_url to only parse those with the XML attribute selected="true". A code change would be required.

[MatthiasWinzeler]

@sean-freeman Thanks for your answer!

Yes, I just did went through the steps again and made sure I both clicked Save as well as Push to Download Basket (not sure the latter is necessary).

The issue is, that - even though the files are deselected - the files still show up in the XML with selected=true. So I suspect that this is actually a bug in the backend system (and adding code in this module to filter them out won't help). Do you by any chance have a contact to the guys on the backend system so that you could reach out to them to confirm this/get it fixed?

Another option would be to use a different API. From what I can tell, the SAP Download Manager - which has the correct behaviour and only lists the selected files - seems to use a different API than the XML above. Maybe we could switch to the other API? Happy to support if I can!

[sean-freeman]

@MatthiasWinzeler Addressing this in reverse order.....

The MP Web GUI code filters the downloads selected and automatically appends to the SAP Software Center "Download Basket" API, which is then queried by the SAP Download Manager binary file (JAR).

Within the codebase here you will see leftovers of this approach:

• def get_download_basket_files(trans_id)
• def add_stack_download_files_to_basket(trans_id)
• def get_download_basket_json()
• def clear_download_basket()
• def delete_item_in_download_basket(object_id)

If the Response Body payload (XML file) cannot be trusted with the correct data payload (i.e. selected="true" and selected="false"), then as you rightly state it's an error beyond our scope.

Even if the above Python Functions were re-added into the code execution flow, we would still have to use the XML with the same incorrect selection data and POST to the Download Basket. An extra few steps that do not have any benefit to the code flow and in my experience, can lead to clashes due to throttling of the APIs for the Download Basket (i.e. potentially downloading files that were left in the Download Basket from days/weeks/months ago).

The error would need to be raised in an SAP Support Incident using the Component BC-UPG-MP.

I think we should note this as a Known Issue, it is unlikely this will be resolved.

[MatthiasWinzeler]

I think we should note this as a Known Issue, it is unlikely this will be resolved.

@sean-freeman agree! thanks for your input, I'll close this.