Microsoft.OpenApi.Kiota.ApiDescription.Client

[kimbell]

Why has Microsoft.OpenApi.Kiota.ApiDescription.Client been abandoned? It's unlisted from Nuget. We currently use this infrastructure with NSwag and are looking into using Kiota.

Today we can just update the json file, and everything happens automatically. With Kiota, we would have to update the file, then run the kiota tool.

Since this is relatively new infrastructure from Microsoft, an explanation for your decision in the docs would be helpful.

[baywet]

Thanks for your interest in kiota and for reaching out. There are multiple reasons why we deprecated this package:

• extremely low usage (264 overall downloads at the time of writing, and the vast majority of those is probably from Microsoft)
• to provide a complete experience, visual studio would have needed to install kiota, and to update the "add open api reference" experience. Which was not prioritized by the Visual Studio team.
• we shipped a Visual Studio Code extension which offers a much better experience. (and we're considering extensions to other IDEs like JetBrain intelliJ or VS)

To be clear, this decision has no impact on the NSwag experience itself and was only related to the Kiota integration with the "add an OpenAPI reference" experience.

Additionally @christianhelle integrated Kiota in REST client code generator, you should give it a try. ( Christian, maybe you should PR the docs similarly to what Andrea has done https://github.com/MicrosoftDocs/openapi-docs/pull/4)

Don't hesitate to provide more feedback and context about your scenario if you feel we're still missing something at this point.

[christianhelle]

Additionally @christianhelle integrated Kiota in REST client code generator, you should give it a try. ( Christian, maybe you should PR the docs similarly to what Andrea has done MicrosoftDocs/openapi-docs#4)

I did think about that a couple of times but it felt a bit too pushy to promote my tools via the official MS documentation channels. But since @baywet mentioned it then I would really love to link to my tools and provide usage instructions as well

[baywet]

@christianhelle it only feels fair to promote your work :) Just make sure you base your PR off Andrea's one please.

[kimbell]

I have trouble seeing how Visual Studio comes into play. If something requires an IDE specific feature, it's a big red flag. Over the last 20 years I've been through enough crap that was bound to VS that didn't get updated as newer versions became available.

The way NSwag is set up isn't very complicated, and something Kiota also should be able to mirror. NSwag provides a nuget package NSwag.ApiDescription.Client that contains the console application used to generate code. This package also contains some MSBuild magic that enables it to hook into the ApiDescription.Client infrastructure.

The generated code contains tons of references to Newtonsoft, but doesn't install Newtonsoft. The assumption is that the package already has been installed. If your system requires some packages to be present before running the tool, document it.

Currently you expect users to call a command line tool to generate code. I'm guessing most people will add some custom MSBuild to call this automatically. If you provided the MSBuild glue, it would give developers another option if they wanted to use it and it fits into what other Microsoft Teams are doing. I'm guessing they didn't do it if nobody wanted it.

You are quoting a low usage count as reason for cutting this; a package with 2 msbuild files and a nuspec. Since the package has been unlisted, no new consumers will find it. The latest version I can find is 0.5.0-preview2 and based on other packages still available, I'm guessing that makes it around September 2022. Version 1 of your tooling was released March 2023. I wouldn't even know about Kiota unless a colleague had heard something at a conference. I can't remember seeing anything related to Kiota in any of the channels I follow.

I had a look at the Rest Client Generator that you mentioned. Visual Studio 2022 isn't listed as a supported version. When I see dependencies as NPM and JAVA, then it becomes a showstopper for me. It will really need some stellar functionality for me to voluntarily suffer through NPM hell.

When I clone a .NET project and build it, I just want it to work. I'm already there now with NSwag, but System.Text.Json support is experimental. I was hoping Kiota could be an alternative. So far I've only done basic Kiota testing, so I have no idea if more advanced scenarios such as polymorphism works; supported by System.Text.Json in .NET7. When the initial setup is a step back from what I already have, doesn't encourage me for further exploration.

[christianhelle]

@kimbell There is a separate VSIX package for REST API Client Code Generator for VS 2022

[christianhelle]

@kimbell NPM and Java are only required for using AutoRest, Swagger Codegen, and OpenAPI Generator.

NSwag, Kiota, and Refit code generators don't require anything else than Visual Studio

[baywet]

Thanks for the additional context. Thanks Christian for clarifying some of the dependencies aspect of your Visual Studio extension.

Open API references

We initially released that package in public preview on 2022-05-30 and sunset it a month ago (see #2752), this is over a year of preview for that package rather than two months. And during that time we only saw 264 downloads. The reference package lifecycle is independent from the generator itself for which we're seeing healthy growth (> 100k downloads) and has GAed back in March as you outlined.

When we talk about OpenAPI references this has multiple layers to it:

1. the generator itself (NSwag/Kiota/....)
2. a reference package (what we pulled a month ago)
3. the OpenAPIReference msbuild template (ships with MSbuild, which ships with dotnet and visual studio)
4. the add Open API reference experience in Visual Studio

While it is true that somebody could get OpenAPI reference generate a kiota client with only the first 3 parts, it is complex, lacks of documentation, requires manual editing of the csproj to work. Additionally the OpenAPI reference part was impacted by a path resolution bug we asked the team under ASP.NET organization to solve but didn't get a fix for at the time.

To provide an end to end experience, we would have needed the "Add OpenAPI reference" experience to be updated to allow selection of the generator, and to request a different set of parameters to the user, so it can generate the appropriate entry in the csproj. This also never happened as the team had other priorities. Which is why we pursued other options.

Lastly, the generator (part 1) needs to be installed, either manually by the user, or packaged with Visual Studio, or through a dependencies chain. But we didn't even explore that problem space given we were not able to make progress on the other aspects.

Adding the "Add OpenAPI reference" screenshot so we all know what we're talking about.

Kiota

Kiota doesn't have any dependencies on msbuild (other than it's built in dotnet) or any other tooling (other than people will need to add the right dependencies to their project). It's a standalone tool distributed through various ways.

Kiota does support composed types (anyOf, oneOf), inheritance (special case of allOf), discriminated inheritance, and polymorphism (other cases of allOf). Although the end support depends on each language and its maturity, stable languages all support those patterns today.

We know we have a gap in terms of documentation and in terms of advocacy. We're working on the former, and the latter is partly due to an internal organization aspect which you, the customer, should not have to care for. In the meantime we're more than happy to support any community member who'd want to speak about kiota at conferences, write blog posts about it, livestream, record videos, build integrations etc... and some of you already have: 1 2 3 4 ...

I hope this lengthy reply clarifies things a bit further, let us know if you have additional questions or comments.

[kimbell]

Thanks for the additional information.

I know the dialog has existed, but never actually used it. I started playing around with the 'Add Service Reference' back in VS2017. Then you needed an extra plugin to get it to generate from WSDL; it was not part of the standard VS install.

My initial work with OpenApi involved NSwag Studio to generate the file; not a smooth workflow. Then using the MSBuild task they provide and finally the newer ApiDescription.Client stuff. I've always set things up manually since that is what I've been used to. Took some time figuring out all the NSwag parameter stuff.

I tried adding using the dialog now, and was really surprised that the code was set up to use NSWag; I had not expected VS to have such a dependency.

Since the OpenApiReference is part of the ecosystem, it would be very helpful with an explanation in the documentation on how it relates to Kiota. For a casual observer, it looks like Kiota doesn't play nice with the ecosystem. The fact that it depends on VS changes would give people some understanding on why you made your choices.

[baywet]

The documentation gap on how to integrate with ms build is fair feedback. At this point what benefits would you see of using an open API reference vs an exec task? https://learn.microsoft.com/en-us/visualstudio/msbuild/exec-task?view=vs-2022

[kimbell]

One thing I have noticed is that whenever I change the OpenApi json file, the client is automatically re-generated. If that for some reason doesn't happen, then I can just delete the generated file and a new file pops up within a few seconds. This happens in the VS IDE without having to trigger a build. Looks like there is some change tracking going on in the background.

One downside to using the API reference is if your project is multi-targeted. Then the MSBuild magic doesn't trigger at all. One workaround is adding some more MSBuild magic.

<Target Name="GenerateMultiTarget" BeforeTargets="BeforeCompile" Condition=" '$(OpenApiGenerateCodeOnBuild)' == 'true' and '$(TargetFramework)' == 'net6.0'">
     <!-- this is required when running in multi target builds-->
     <CallTarget Targets="GenerateOpenApiCode" />
 </Target>

[baywet]

I believe this refresh behavior is done because their targets (internal to visual studio/msbuild) define an item group (similar to the binaries example in the docs). But as I don't have access to the sources, I can't confirm that. It's functionally the equivalent to the refresh command of the dotnet tool (but it triggers automatically).

And yes multi-targeting is another limitation of the OpenAPI references we identified at the time. It'd be nice if their target/props files could be open sourced so we could PR it, like we had started doing to the dotnet tool.

@christianhelle does your extension inject anything to the csproj at this time?

@kimbell it'd be nice if we could come up with a targets/props example file that people could include in their repository and reference in their csproj. Ideally this would:

• Rely on msbuild infrastructure to trigger refreshes when a local OpenAPI file gets changed
• Trigger the generation command with the right parameters
• Warn if kiota is missing
• Optionally. Add the package references.

What do you think?

[christianhelle]

@baywet My extension installs the required NuGet packages to build the generated code.

In the case of Kiota, it installs the following packages:

• Microsoft.Kiota.Abstractions
• Microsoft.Kiota.Http.HttpClientLibrary
• Microsoft.Kiota.Serialization.Form
• Microsoft.Kiota.Serialization.Text
• Microsoft.Kiota.Serialization.Json
• Microsoft.Kiota.Authentication.Azure
• Azure.Identity

I do this to improve the first experience of switching to (or trying out) a new code generator.

All this happens immediately after clicking the "Generate with Kiota" context menu from the solution explorer

Here's an old screenshot that I took from my docs. The context menu shows exactly what version of the code generator will be used

[baywet]

Thanks for the context. But your extension doesn't add any msbuild task to refresh the generation on build or on change of the open api description, correct?

[christianhelle]

Thanks for the context. But your extension doesn't add any msbuild task to refresh the generation on build or on change of the open api description, correct?

@baywet No, it does not

[kimbell]

@baywet I have done some experimentation on msbuild integration and this is what I have come up with. It looks like VS automatically runs some of the tasks; if I remove the package references, they are automatically added back in.

Kiota.props

<Project>

    <!-- Properties that control tooling setup-->
    <PropertyGroup>
        <KiotaAutoInstallTool>true</KiotaAutoInstallTool>
        <KiotaAutoInstallPackages>true</KiotaAutoInstallPackages>
    </PropertyGroup>

    <!-- Propertis that control the actual CLI-->
    <PropertyGroup>
        <!-- These are the only required arguments required by the CLI -->
        <KiotaDescriptionFile></KiotaDescriptionFile>
        <KiotaLanguage></KiotaLanguage>

        <!-- If these are not specified, the CLI will use default values -->
        <KiotaOutputPath></KiotaOutputPath>
        <KiotaClassName></KiotaClassName>
        <KiotaNamespaceName></KiotaNamespaceName>
        <KiotaLogLevel></KiotaLogLevel>
        <KiotaBackingStore></KiotaBackingStore>
        <KiotaAdditionlData></KiotaAdditionlData>
        <KiotaSerializer></KiotaSerializer>
        <KiotaDeserializer></KiotaDeserializer>
        <KiotaMimeTypes></KiotaMimeTypes>
        <KiotaIncludePath></KiotaIncludePath>
        <KiotaExcludePath></KiotaExcludePath>
        <KiotaDisableValidationRules></KiotaDisableValidationRules>
    </PropertyGroup>

    <ItemGroup>
        <KiotaDefaultPackages Include="Microsoft.Kiota.Abstractions"/>
        <KiotaDefaultPackages Include="Microsoft.Kiota.Http.HttpClientLibrary"/>
        <KiotaDefaultPackages Include="Microsoft.Kiota.Serialization.Form"/>
        <KiotaDefaultPackages Include="Microsoft.Kiota.Serialization.Json"/>
        <KiotaDefaultPackages Include="Microsoft.Kiota.Serialization.Text"/>
    </ItemGroup>

</Project>

Kiota.targets

<Project>

    <!-- Install Kiota packages-->
    <Target
        Name="_KiotaInstallPackages"
        Condition="'$(KiotaAutoInstallPackages)' == 'true'">

        <ItemGroup>
            <KiotaMissingReferences Include="@(KiotaDefaultPackages)" Exclude="@(PackageReference)" />
        </ItemGroup>

        <Exec
            Condition="'@(KiotaMissingReferences)' != ''"
            Command="dotnet add package %(KiotaMissingReferences.Identity)" />
    </Target>

    <!-- Install the Kiota tool -->
    <Target
        Name="_KiotaInstallTool"
        Condition="'$(KiotaAutoInstallTool)' == 'true'">

        <PropertyGroup>
            <KiotaToolLocation>$(USERPROFILE)\.dotnet\tools\.store\microsoft.openapi.kiota</KiotaToolLocation>
        </PropertyGroup>

        <Exec Condition="exists('$(KiotaToolLocation)') == 'false'" Command="dotnet tool install --global Microsoft.OpenApi.Kiota"/>
    </Target>

    <!-- Run Kiota tool -->
    <Target 
        Name="_KiotaRunTool"
        DependsOnTargets="_KiotaInstallTool;_KiotaInstallPackages"
        BeforeTargets="BeforeCompile">

        <!-- Make sure that required properties have been specified -->
        <Error Condition="'$(KiotaDescriptionFile)' == ''" Text="KiotaDescriptionFile property must have a value" />
        <Error Condition="'$(KiotaLanguage)' == ''" Text="KiotaLanguage property must have a value" />

        <!-- Build up the CLI arguments -->
        <PropertyGroup>
            <KiotaArguments></KiotaArguments>
            <KiotaArguments>$(KiotaArguments) --openapi $(KiotaDescriptionFile)</KiotaArguments>
            <KiotaArguments>$(KiotaArguments) --language $(KiotaLanguage)</KiotaArguments>

            <KiotaArguments Condition="'$(KiotaClassName)' != ''">$(KiotaArguments) --class-name $(KiotaClassName)</KiotaArguments>
            <KiotaArguments Condition="'$(KiotaNamespaceName)' != ''">$(KiotaArguments) --namespace-name $(KiotaNamespaceName)</KiotaArguments>
            <KiotaArguments Condition="'$(KiotaOutputPath)' != ''">$(KiotaArguments) --output $(KiotaOutputPath)</KiotaArguments>
            <KiotaArguments Condition="'$(KiotaLogLevel)' != ''">$(KiotaArguments) --log-level $(KiotaLogLevel)</KiotaArguments>
            <KiotaArguments Condition="'$(KiotaBackingStore)' != ''">$(KiotaArguments) --backing-store $(KiotaBackingStore)</KiotaArguments>
            <KiotaArguments Condition="'$(KiotaAdditionlData)' != ''">$(KiotaArguments) --additional-data $(KiotaAdditionlData)</KiotaArguments>
            <KiotaArguments Condition="'$(KiotaSerializer)' != ''">$(KiotaArguments) --serializer $(KiotaSerializer)</KiotaArguments>
            <KiotaArguments Condition="'$(KiotaDeserializer)' != ''">$(KiotaArguments) --deserializer $(KiotaDeserializer)</KiotaArguments>
            <KiotaArguments Condition="'$(KiotaMimeTypes)' != ''">$(KiotaArguments) --structured-mime-types $(KiotaMimeTypes)</KiotaArguments>
            <KiotaArguments Condition="'$(KiotaIncludePath)' != ''">$(KiotaArguments) --include-path $(KiotaIncludePath)</KiotaArguments>
            <KiotaArguments Condition="'$(KiotaExcludePath)' != ''">$(KiotaArguments) --exclude-path $(KiotaExcludePath)</KiotaArguments>
            <KiotaArguments Condition="'$(KiotaDisableValidationRules)' != ''">$(KiotaArguments) --disable-validation-rules $(KiotaDisableValidationRules)</KiotaArguments>
        </PropertyGroup>

        <!-- Run the generate command-->
        <Exec
            Command="kiota generate $(KiotaArguments)"/>
    </Target>

</Project>

project file

<Import Project="Kiota.props" />
<Import Project="Kiota.targets" />

<PropertyGroup>
    <KiotaDescriptionFile>./OpenApi.json</KiotaDescriptionFile>
    <KiotaOutputPath>./Generated</KiotaOutputPath>
    <KiotaLanguage>CSharp</KiotaLanguage>
    <KiotaClassName>TypeClient</KiotaClassName>
    <KiotaNamespaceName>Bob.Kiota.Client</KiotaNamespaceName>
</PropertyGroup>

[baywet]

This is great work! Thanks for looking into this! I think we should move this to a pull request to make sure we have those as part of the repository, even as part of the distributed assets.

Couple of remarks:

• Any project could have multiple OpenAPI descriptions, we should probably introduce a parent node so people can have multiple
• What do you think of including that as an asset (Copy) in the kiota.csproj? this way people could install the tool and reference the props/targets from the tool installation.
• I'm guessing it doesn't refresh on description change? but that's probably not an issue since it does on rebuild
• I'm guessing you had additional lines to reference the targets and props files in your csproj?

@christianhelle would it make sense for your extension to automatically insert the entries in the csproj?

[christianhelle]

@christianhelle would it make sense for your extension to automatically insert the entries in the csproj?

@baywet No, I don't think that it would really make much sense.

The core features of my extension are Visual Studio custom tools and the suggested MSBuild properties above are things you already have provided by Visual Studio when implementing the IVsSingleFileGenerator interface.

When Visual Studio executes the Generate() method in the Custom Tool (The IVsSingleFileGenerator implementation), you already know the following:

• Path to the OpenAPI spec file
• Contents of the OpenAPI spec file
• The namespace to use. This is based on the location of the file within the project, prefixed by the default root namespace

The custom tool is run every time there are changes to the OpenAPI spec file, including moving it around to different folders, or even just saving the file without any actual changes

The output of my generated is treated as a "code-behind file" to the OpenAPI spec in the project. It looks something like this in Visual Studio

[christianhelle]

@baywet the MSBuild properties that @kimbell uses would make sense for a rosyln source generator.

I have a similar project called Refitter that generates a Refit interface from an OpenAPI spec file (or URL).

In this project, I recently implemented a source generator that uses a .refitter file in the project to generate code on compile time. I guess this idea is similar to what @kimbell does

[christianhelle]

@kimbell That was some really great stuff! I remember doing something similar in the past and distributed the .props file in a NuGet package. This basically had the same effect as a rosyln source generator

[kimbell]

I've updated my previous comment with the missing import statements.

One of the things I do for my client is working on one of their product platform teams. There we build a number of packages that is used by 150+ in-house applications. Recently we have starter bundling props and targets files so that we can easily impact all applications. So far this has mainly been to enable analyzers and suppress 'approved' warnings, but can be extended to other things in the future.

If the idea is to bundle these files with a nuget package, figuring out support for multiple OpenApi documents quickly becomes a must. I have no idea how to approach something like that. The easy approach is for people to copy the files, add some name-prefix and edit the files depending on their needs.

[baywet]

@christianhelle: understood, thanks for the additional context. I hadn't realized the experience of your extension was so advanced.

@kimbell what do you think of something like that (if possible)

<PropertyGroup>
    <KiotaReference>
    <KiotaDescriptionFile>./OpenApi.json</KiotaDescriptionFile>
    <KiotaOutputPath>./Generated</KiotaOutputPath>
    <KiotaLanguage>CSharp</KiotaLanguage>
    <KiotaClassName>TypeClient</KiotaClassName>
    <KiotaNamespaceName>Bob.Kiota.Client</KiotaNamespaceName>
    </KiotaReference>
</PropertyGroup>

And then your targets could loop?

Or alternatively something like this if it's easier

<PropertyGroup>
    <KiotaReference DescriptionFile="./OpenApi.json" OutputPath="./Generated" Language="CSharp" ClassName="TypeClient" NamespaceName="Bob.Kiota.Client" />
</PropertyGroup>

[kimbell]

I managed to get a multi-reference project to work. I uploaded my testing project: https://github.com/kimbell/Kiota.Testing

[baywet]

This is great news! Don't forget to open a pull request here whenever your testing is completed.

[microsoft-github-policy-service[bot]]

This issue has been automatically marked as stale because it has been marked as requiring author feedback but has not had any activity for 4 days. It will be closed if no further activity occurs within 3 days of this comment.

[johnwc]

@kimbell @baywet whatever happened with this? We were looking to test out Kiota and were directed here from our searching. We use NSWag right now using the NSwag.ApiDescription.Client package using sample reference below. Was hoping to be able to do something similar for Kiota by installing a Kiota package and changing CodeGenerator with correct Options needed for Kiota.

<ItemGroup>
  <OpenApiReference Include="API\ExampleOpenAPI.json" CodeGenerator="NSwagCSharp" ClassName="ExampleSDK" Options="/OperationGenerationMode:SingleClientFromOperationId" Namespace="Example.API">
  </OpenApiReference>
</ItemGroup>

If not already done, can this same functionality be wrapped up in a package similar to how nswag makes it happen so effortlessly for us developers? https://github.com/RicoSuter/NSwag/tree/master/src/NSwag.ApiDescription.Client

[kimbell]

For my client, we have about 50 nuget packages that wrap WebApi clients; some auto-generated, some manually written. These package are then consumed by around 200 different applications. The dependencies these packages take becomes very important in order to avoid dependency hell.

Our current dependencies are on MS System packages and Newtonsoft since these have been stable for a long period of time. The good thing with NSwag was that it doesn't depend on any NSwag packages to run; only Newtonsoft and HttpClient. One thing I don't like is having to catch exceptions for non-200 codes, and it seems to be still be based around Newtonsoft instead of System.Text.Json. I know there is a 14.* release available, but when I tried updating to that, the level of errors made we roll back to the previous version.

Kiota requires a package to be included in every client; a no-go for us. Last time I checked, it also does it's own Json parsing since it supports multiple mime types; some abstraction layer.

For our client, we know everything is Json. We can also make certain other assumptions a general purpose tool cannot make. We spent a couple of weeks rolling out our C# generator; this uses STJ, nullable reference types, and other C# .NET8 features we know everyone will have. This is basically a console application with some MSBuild files we copied from NSwag. We have been testing this on a couple of clients based on OpenApi from the latest Swashbuckle release. Wider adoption is planned after we get it running on output from the new Microsoft OpenApi work coming in .NET9.

How did we avoid exception usage like NSwag? Each generated method returns a wrapper object containing status code, headers and a typed property for each possible status. If the method can return 200, we add in an implicit operator. The happy path still works, but code can do other things on specific status codes.