SAP / python-pyodata

Enterprise-ready Python OData client
Apache License 2.0
219 stars 93 forks source link

Incorrect metadata parsing for attributes with sap prefix #155

Open msardana94 opened 3 years ago

msardana94 commented 3 years ago

I am using this to extract metadata when I realized the sap prefixed attributes are not coming through accurately.

Pyodata version: 1.7.0

XML metadata from API for Entity Type JobApplicationSnapshot_FuncExperience Api call: https://api4.successfactors.com/odata/v2/JobApplicationSnapshot_FuncExperience/$metadata

<?xml version="1.0" encoding="utf-8"?>
<edmx:Edmx Version="1.0" xmlns:edmx="http://schemas.microsoft.com/ado/2007/06/edmx" xmlns:atom="http://www.w3.org/2005/Atom">
    <edmx:DataServices m:DataServiceVersion="2.0" xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata">
        <Schema Namespace="SFODataSet" xmlns="http://schemas.microsoft.com/ado/2008/09/edm" xmlns:sf="http://www.successfactors.com/edm/sf" xmlns:sap="http://www.successfactors.com/edm/sap">
            <EntityContainer Name="EntityContainer" m:IsDefaultEntityContainer="true">
                <EntitySet Name="JobApplicationSnapshot_FuncExperience" EntityType="SFOData.JobApplicationSnapshot_FuncExperience" sap:label="JobApplicationSnapshot_FuncExperience" sap:creatable="false" sap:updatable="false" sap:upsertable="false" sap:deletable="false">
                    <Documentation>
                        <Summary>Candidate Background</Summary>
                        <LongDescription>These entities represent the background elements as defined in the candidate template. For example:CandidateBackground_Certificate, CandidateBackground_Education, etc</LongDescription>
                        <sap:tagcollection>
                            <sap:tag>Recruiting (RCM)</sap:tag>
                            <sap:tag>RCM - Candidate</sap:tag>
                        </sap:tagcollection>
                    </Documentation>
                </EntitySet>
            </EntityContainer>
        </Schema>
        <Schema Namespace="SFOData" xmlns="http://schemas.microsoft.com/ado/2008/09/edm" xmlns:sf="http://www.successfactors.com/edm/sf" xmlns:sap="http://www.successfactors.com/edm/sap">
            <EntityType Name="JobApplicationSnapshot_FuncExperience">
                <Key>
                    <PropertyRef Name="backgroundElementId"></PropertyRef>
                </Key>
                <Property Name="applicationId" Type="Edm.Int64" Nullable="false" sap:required="false" sap:creatable="false" sap:updatable="false" sap:upsertable="false" sap:visible="true" sap:sortable="false" sap:filterable="true" sap:label="Application Id"></Property>
                <Property Name="backgroundElementId" Type="Edm.Int64" Nullable="false" sap:required="false" sap:creatable="false" sap:updatable="false" sap:upsertable="false" sap:visible="true" sap:sortable="false" sap:filterable="true" sap:label="Background Element Id"></Property>
                <Property Name="bgOrderPos" Type="Edm.Int64" Nullable="false" sap:required="false" sap:creatable="false" sap:updatable="false" sap:upsertable="false" sap:visible="true" sap:sortable="true" sap:filterable="false" sap:label="Background Order Position"></Property>
                <Property Name="function1" Type="Edm.String" Nullable="false" sap:required="true" sap:creatable="false" sap:updatable="false" sap:upsertable="false" sap:visible="true" sap:sortable="false" sap:filterable="false" sap:picklist="func_exp" sap:label="Experience Type"></Property>
                <Property Name="lastModifiedDateTime" Type="Edm.DateTimeOffset" Nullable="false" sap:required="false" sap:creatable="false" sap:updatable="false" sap:upsertable="false" sap:visible="true" sap:sortable="false" sap:filterable="true" sap:label="Last Modified Date"></Property>
                <Property Name="startDate" Type="Edm.DateTime" Nullable="true" sap:required="false" sap:creatable="false" sap:updatable="false" sap:upsertable="false" sap:visible="true" sap:sortable="false" sap:filterable="false" sap:display-format="Date" sap:label="As of Date"></Property>
                <Property Name="yearsExp" Type="Edm.Double" Nullable="true" sap:required="false" sap:creatable="false" sap:updatable="false" sap:upsertable="false" sap:visible="true" sap:sortable="false" sap:filterable="false" sap:label="Years of Experience (Number)"></Property>
            </EntityType>
        </Schema>
    </edmx:DataServices>
</edmx:Edmx>

Here's the code to reproduce the error for above entity:

In [29]: import requests, pyodata

In [30]: session = requests.Session()

In [31]: session.auth = ("username", "password")

In [32]: SERVICE_URL = "https://api4.successfactors.com/odata/v2"

In [33]: service = pyodata.Client(SERVICE_URL, session)

In [34]: p = service.schema.entity_type("JobApplicationSnapshot_FuncExperience").proprty("applicationId")

In [35]: print(f"label={p.label}, updatable={p.updatable}, creatable={p.creatable}, visible={p.visible}, sortable={p.sortable}, filterable={p.filterable}")
label=None, updatable=True, creatable=True, visible=True, sortable=True, filterable=True

Expected Output:

label=Application Id, updatable=False, creatable=False, visible=True, sortable=False, filterable=True

After digging into the code, I believe the issue is here. The uri for prefix sap that I see in the raw XML data (as shown above) is http://www.successfactors.com/edm/sap.

Apologies if I am missing something obvious here. I am still fairly new to XML and successfactors api.

filak-sap commented 3 years ago

You are absolutely right. The current code reads "ABAP Odata annotations" and your service is from SuccessFactors. Honestly, adding support for the SAP attributes to pyodata directly was a bad idea and now we have to find a way how to support different "SAP" namespace URL. It would be nice to have that magically resolved but manual configuration by a programmer would be safer. What do you think?

msardana94 commented 3 years ago

Thanks for the quick response. Yes I agree that having the ability to pass information such as namespace urls via manual configuration will definitely solve this issue and make the package extensible.

phanak-sap commented 3 years ago

@filak-sap What about having this problem solved same way as our sibling library, odata-library? E.g. sap attributes , namespaces etc. not hardcoded in a model.py but as extension mechanism, with SAP as one possible "metadata-specification lingo" - see https://github.com/SAP/odata-library/tree/master/lib/model/nw/extensions/sap

jfilak commented 3 years ago

@phanak-sap Go for it ;)

jfilak commented 3 years ago

@phanak-sap but if you have time, I would first finish support for v4 becaus that brings a major redisign of the entire library.