OData V2 CSDL error with TextTransform.exe

[adrien-constant]

Describe the bug

When using TextTransform.exe with a SAP OData V2 CSDL, there is an error. However, it works with the same XML file when the Connected Service is added within Visual Studio. The TextTransform.exe command also works with another OData V4 CSDL file.

Version of the Project affected

ODataConnectedService v1.2.0 VisualStudio Version v16.8.2 Microsoft.Odata.Client v7.6.3

To Reproduce

Steps to reproduce the behavior:

1. Copy the files from C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\Common7\IDE\Extensions\mstwd1cc.mrs\Templates to c:\temp\templates
2. Save the XML file to C:\temp\test.xml (xml file is specified at the end of this issue)
3. Customize the ODataT4CodeGenerator.tt file with the following paramerters :
4. Set MetadataDocumentUri and MetadataFilePath to c:\temp\test.xml
5. Set MetadataFileRelativePath to test.xml
6. Set TargetLanguage to CSharp
7. Execute "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\Common7\IDE\TextTransform.exe" -P "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\Common7\IDE\Extensions\Microsoft\ConnectedServices" -P "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\Common7\IDE\Extensions\mstwd1cc.mrs" -P "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\Common7\IDE\PublicAssemblies" "c:\temp\templates\ODataT4CodeGenerator.tt"

Expected behavior

Code is generated.

Actual behavior

Got an error :

error : Running transformation: System.ArgumentNullException: Value cannot be null.
Parameter name: key
   at System.Collections.Generic.Dictionary`2.FindEntry(TKey key)
   at System.Collections.Generic.Dictionary`2.TryGetValue(TKey key, TValue& value)
   at Microsoft.OData.Edm.Csdl.CsdlReader.TryParse(IEnumerable`1 referencedModels, Boolean includeDefaultVocabularies, IEdmModel& model, IEnumerable`1& parsingErrors)
   at Microsoft.OData.Edm.Csdl.CsdlReader.TryParse(XmlReader reader, IEnumerable`1 references, CsdlReaderSettings settings, IEdmModel& model, IEnumerable`1& errors)
   at Microsoft.VisualStudio.TextTemplating32cae004e60c432f8141566248f683c1.GeneratedTextTransformation.CodeGenerationContext.get_EdmModel() in :line 984
   at Microsoft.VisualStudio.TextTemplating32cae004e60c432f8141566248f683c1.GeneratedTextTransformation.CodeGenerationContext.get_NamespacesInModel() in :line 1088
   at Microsoft.VisualStudio.TextTemplating32cae004e60c432f8141566248f683c1.GeneratedTextTransformation.ODataClientTemplate.WriteNamespaces() in :line 1740
   at Microsoft.VisualStudio.TextTemplating32cae004e60c432f8141566248f683c1.GeneratedTextTransformation.ODataClientTemplate.TransformText() in :line 1733
   at Microsoft.VisualStudio.TextTemplating32cae004e60c432f8141566248f683c1.GeneratedTextTransformation.TransformText() in :line 148

CSDL File (test.xml)

<edmx:Edmx Version="1.0" xmlns:edmx="http://schemas.microsoft.com/ado/2007/06/edmx" xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata">
  <edmx:DataServices m:DataServiceVersion="2.0">
    <Schema Namespace="ZC_APIARTICLE2_CDS" xml:lang="en" xmlns="http://schemas.microsoft.com/ado/2008/09/edm">
      <EntityType Name="ZC_ApiArticle2Type" >
        <Key>
          <PropertyRef Name="matnr"/>
        </Key>
        <Property Name="matnr" Type="Edm.String" Nullable="false" MaxLength="40" />
        <Property Name="matkl" Type="Edm.String" MaxLength="9" />
      </EntityType>
      <EntityContainer Name="ZC_APIARTICLE2_CDS_Entities" m:IsDefaultEntityContainer="true" >
        <EntitySet Name="ZC_ApiArticle2" EntityType="ZC_APIARTICLE2_CDS.ZC_ApiArticle2Type" />
      </EntityContainer>
      <Annotations Target="ZC_APIARTICLE2_CDS.ZC_APIARTICLE2_CDS_Entities" xmlns="http://docs.oasis-open.org/odata/ns/edm">
        <Annotation Term="Aggregation.ApplySupported">
          <Record>
            <PropertyValue Property="Transformations">
              <Collection>
                <String>aggregate</String>
                <String>groupby</String>
                <String>filter</String>
              </Collection>
            </PropertyValue>
            <PropertyValue Property="Rollup" EnumMember="None"/>
          </Record>
        </Annotation>
      </Annotations>
      <atom:link rel="self" href="http://sapserver/sap/opu/odata/sap/ZC_APIARTICLE2_CDS/$metadata" xmlns:atom="http://www.w3.org/2005/Atom"/>
      <atom:link rel="latest-version" href="http://sapserver/sap/opu/odata/sap/ZC_APIARTICLE2_CDS/$metadata" xmlns:atom="http://www.w3.org/2005/Atom"/>
    </Schema>
  </edmx:DataServices>
</edmx:Edmx>

[unchase]

Hi, @adrien-constant Probably this issue is occured because the T4 template works only with OData v4 metadata.

[adrien-constant]

Hi, @adrien-constant Probably this issue is occured because the T4 template works only with OData v4 metadata.

Thank you @unchase . What is the recommended way to automate the code generation for V2 metadata then ? I am using TextTransform.exe inside a Azure DevOps pipeline to generate C# files. If it's not possible to use T4 templates, is it possible to call directly the librairies with parameters to have the same behavior as using the extension inside VS ?

[habbes]

Hi @adrien-constant as @unchase has mentioned, the T4 templates only work with V4. Code generation for OData V1 to V3 are handled by a separate closed-source library. You can find documentation for it, but it was not meant for public use: EntityClassGenerator of the System.Data.Services.Design namespace.

To use this class you need to add references to the following assemblies to your project:

• System.Data.Services.Design
• System.Data.Entity

Note: I believe these are not available in .NET Core or .NET 5.

Here's an example of how you'd use it to generate code from a schema file:

// using System.Data.Services.Design;

var generator = new EntityClassGenerator(LanguageOption.GenerateCSharpCode)
{
    Version = DataServiceCodeVersion.V2, // if you also want to support V3, check the notes below
    UseDataServiceCollection = true // this adds supports for property binding by implementing INotifyPropertyChanged
};

using (var reader = XmlReader.Create("path\to\schema.xml"))
{
   var errors = generator.GenerateCode(reader, "path\to\output\GeneratedCode.cs");
}

Notes: If for some reason you do not have access to the required assemblies, you could download the v3 assembly binaries that are located in the ODataConnectedService repo: https://github.com/OData/ODataConnectedService/tree/master/external/Binaries/V3

If you want to support V3, then you should definitely use these binaries as the built-in references seem to only support V1 and V2 (DataServiceCodeVersion.V3 is not available in the built-in assemblies).

[unchase]

@habbes Or we can modify the solution so that it has a separated command-line project (with all needed options) to run code generation for both OData V4 and OData V3 with using standart functionality in the Connected Service.

We can use, for example: commandline, or write your own command line parameter parser.

[habbes]

Hi @unchase, actually creating a cli tool is something we have considered https://github.com/OData/ODataConnectedService/issues/169 We don't yet have an ETA for it. @marabooy could you chime in.

[marabooy]

@unchase I think creating the Cli version could be a great thing to have, Initially my thoughts for this would be to have a project that extracts the core codegeneration from the current connected service and have the new core be extensible/configurable by different frontends with the Vs extension being one of them and CLI being another and any other future projects.

[unchase]

@marabooy @habbes Do you think it is better to add the CLI project to the current solution or to move it out into a separate one? In the second case, it is necessary to move the common functionality into a separate library (for example, in nuget), and this can take a lot of time.

[marabooy]

This is just my opinion, I do think we could have a new project in the solution the same way OData.net has 4 projects Edm, Spatial, Core & Client but with time we could extract the core library into a project that can be published to nuget if there are people who want to write their own frontends with the same. @habbes @gathogojr @ElizabethOkerio & @KenitoInc thoughts?

[habbes]

@unchase I think we can have a CLI project in the current solution. I don't think that contradicts the proposal made by @marabooy since the common functionality could also be a project in the same solution.

I'd suggest that the first path be to do the minimum refactoring required to have a standalone CLI project. Maybe we could even omit the file-splitting feature from this version of the CLI. Then in subsequent versions we also surface the core library that has no dependency on VS Shell.

But I think publishing the core library on NuGet should be done if there's demand for it or once we're sure its public API will be stable, because putting it on NuGet also means maintaining it, being conscious of breaking changes, etc.