Support OpenAPI v3

[patricksadowski]

DocFX Version Used: 2.35.4

Template used: default

Steps to Reproduce:

1. Clone docfx-seed
2. Add petstore.yaml to cloned repo inside restapi/
3. Add yamlmime ### YamlMime:ManagedReference in the first line of restapi/petstore.yaml
4. Build with docfx

Expected Behavior:

• Successful build
• Working output with documented REST API
• Use of yaml files as is (without yamlmime)
• Support of OpenAPI v3

Actual Behavior:

Build failed.
[18-05-08 12:55:54.098]Error:System.AggregateException: Mindestens ein Fehler ist aufgetreten. ---> Microsoft.DocAsCode.Build.Engine.Incrementals.BuildCacheException: Full build hasn't loaded model restapi/petstore.yaml
   bei Microsoft.DocAsCode.Build.Engine.HostService.<>c__DisplayClass94_1.<SaveIntermediateModel>b__1()
   bei Microsoft.DocAsCode.Build.Engine.Incrementals.IncrementalUtility.RetryIO(Action action)
   bei System.Threading.Tasks.Parallel.<>c__DisplayClass42_0`2.<PartitionerForEachWorker>b__1()
   bei System.Threading.Tasks.Task.InnerInvokeWithArg(Task childTask)
   bei System.Threading.Tasks.Task.<>c__DisplayClass176_0.<ExecuteSelfReplicating>b__0(Object )
   --- Ende der internen Ausnahmestapelüberwachung ---
   bei System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   bei System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
   bei System.Threading.Tasks.Parallel.PartitionerForEachWorker[TSource,TLocal](Partitioner`1 source, ParallelOptions parallelOptions, Action`1 simpleBody, Action`2 bodyWithState, Action`3 bodyWithStateAndIndex, Func`4 bodyWithStateAndLocal, Func`5 bodyWithEverything, Func`1 localInit, Action`1 localFinally)
   bei System.Threading.Tasks.Parallel.ForEachWorker[TSource,TLocal](IEnumerable`1 source, ParallelOptions parallelOptions, Action`1 body, Action`2 bodyWithState, Action`3 bodyWithStateAndIndex, Func`4 bodyWithStateAndLocal, Func`5 bodyWithEverything, Func`1 localInit, Action`1 localFinally)
   bei System.Threading.Tasks.Parallel.ForEach[TSource](IEnumerable`1 source, ParallelOptions parallelOptions, Action`1 body)
   bei Microsoft.DocAsCode.Build.Engine.HostService.SaveIntermediateModel(IncrementalBuildContext incrementalContext)
   bei Microsoft.DocAsCode.Build.Engine.CompilePhaseHandlerWithIncremental.PostHandle(List`1 hostServices)
   bei Microsoft.DocAsCode.Build.Engine.PhaseProcessor.Process(List`1 hostServices, Int32 maxParallelism)
   bei Microsoft.DocAsCode.Build.Engine.SingleDocumentBuilder.BuildCore(PhaseProcessor phaseProcessor, List`1 hostServices, DocumentBuildContext context)
   bei Microsoft.DocAsCode.Build.Engine.SingleDocumentBuilder.BuildCore(DocumentBuildParameters parameters)
   bei Microsoft.DocAsCode.Build.Engine.SingleDocumentBuilder.Build(DocumentBuildParameters parameters)
   bei Microsoft.DocAsCode.Build.Engine.DocumentBuilder.BuildCore(DocumentBuildParameters parameter, IMarkdownServiceProvider markdownServiceProvider, BuildInfo currentBuildInfo, BuildInfo lastBuildInfo)
   bei Microsoft.DocAsCode.Build.Engine.DocumentBuilder.Build(IList`1 parameters, String outputDirectory)
   bei Microsoft.DocAsCode.SubCommands.DocumentBuilderWrapper.BuildDocument(BuildJsonConfig config, TemplateManager templateManager, String baseDirectory, String outputDirectory, String pluginDirectory, String templateDirectory)
   bei Microsoft.DocAsCode.SubCommands.BuildCommand.BuildDocument(String baseDirectory, String outputDirectory)
   bei Microsoft.DocAsCode.SubCommands.BuildCommand.Exec(SubCommandRunningContext context)
   bei Microsoft.DocAsCode.SubCommands.CompositeCommand.Exec(SubCommandRunningContext context)
   bei Microsoft.DocAsCode.Program.ExecSubCommand(String[] args)
---> (Interne Ausnahme #0) Microsoft.DocAsCode.Build.Engine.Incrementals.BuildCacheException: Full build hasn't loaded model restapi/petstore.yaml
   bei Microsoft.DocAsCode.Build.Engine.HostService.<>c__DisplayClass94_1.<SaveIntermediateModel>b__1()
   bei Microsoft.DocAsCode.Build.Engine.Incrementals.IncrementalUtility.RetryIO(Action action)
   bei System.Threading.Tasks.Parallel.<>c__DisplayClass42_0`2.<PartitionerForEachWorker>b__1()
   bei System.Threading.Tasks.Task.InnerInvokeWithArg(Task childTask)
   bei System.Threading.Tasks.Task.<>c__DisplayClass176_0.<ExecuteSelfReplicating>b__0(Object )<---

[vwxyzh]

@patricksadowski for Open API 3.0, we use schema driven not manage reference in docs.microsoft.com, but in docfx, we still missing following item:

• integrate CI
• document schema for Open API 3.0
• template for Open API 3.0

Manage reference is only for dotnet API, and we will move to schema driven in v3.

[patricksadowski]

Ok, thank you. I'll be patient and watch https://github.com/dotnet/docfx/projects/1 for updates.

[adanb007]

Does DocFx support OpenAPI 3.0.1 definition by adding the swagger.json. I am getting an error while building my project.

[pryabov]

Good day. Could you clarify, pls. Do you have any plans to support OpenAPI 3.0.1? And when you are planning to implement this

[adanb007]

If anything, we should know if we need to re-open this issue. The methodology to "downgrade" to the swagger 2.0 definition is not a good solution. Is anyone working on this? maybe on v3?

[superyyrrzz]

OpenAPI v3 is nearly ready on docs.microsoft.com, but there is still no page served.

@adanb007 You are right. v3 uses docs.microsoft.com template, and we need to wait for v3 to ship this feature.

/cc @yishengjin1413

[adanb007]

@superyyrrzz Thanks!

[pryabov]

thank you

[LinasPix]

Any changes on this topic lately? maybe release date is known or some preview version is available? Thank you!

[allscripts-niravbhatt]

Is Open Api 3.0 support added on any of docfx v3 preview versions?

Any updates would be great.

[Zavarzza]

Good afternoon! Please tell me if there is any progress on working with Open Api 3.0 in docfx. This is a very important issue for our company using your product. Thanks!

[ManuelRauber]

Is there any news regarding this?

[Sergi0Martin]

I had same problem. Whith this tool I solved it DocFxOpenApi

[filzrev]

I thought it's needs Microsoft.OpenAPI library integration to support OpenAPI 2.0/3.0. And map OpenAPI models to existing FileModel at RestApiDocumentProcessor.

[dougclutter]

This has become a critical issue for us because NSwag v14 has dropped support for generating Swagger 2.0 files; their command line tools have dropped the aspnetcore2swagger option.

Any ETA on when can we expect docfx to support OpenApi 3.0 files?

2024/1/26 - I was incorrect in asserting that NSwag v14 dropped support for Swagger 2.0. Sorry.

[davesmits]

@dougclutter we are all excited for your contribution

[dougclutter]

I've started looking into adding support for OpenApi 3 but I have some questions for the Dev Team:

• It appears the current code is very forgiving of JSON that doesn't follow the Swagger 2.0 specifications. Was this an intentional decision? Can we be a little stricter with OpenApi 3?
• The current code appears to load the JSON into models defined within docfx. While this approach gives us a lot of control, it also adds a lot of technical debt. As the OpenApi spec evolves, the docfx models have to be updated. As an alternative, I was looking into using Microsoft.OpenApi.Readers to parse the JSON. Any thoughts on this approach?
• The current code is heavily dependent on Newtonsoft. Should we transition to STJ?
• Currently, SwaggerModelConverter creates a RestApiRootItemViewModel. Initially, I thought RestApiRootItemViewModel would be a good target for OpenApi 3 docs, so the rest of the code could remain unchanged. However, it looks like RestApiRootItemViewModel may be too closely tied to Swagger 2.0. Any thoughts?

I forked the repo and created a branch named addSupportForOpenApi3. This is a very rough cut, but I was hoping for feedback before spending any additional time on this approach. Thanks!

[yufeih]

Hi @dougclutter,

Thank you so much for contributing the OpenApi 3 support!

OpenAPI.NET is definitely the library we would use for OpenApi 3 support and I agree that redefining the models was not a great technical choice, same could be said about the .NET MREF approach.

I would suggest start fresh by leveraging the new API Page that was designed as a base to support multiple languages. You can refer to how .NET generates API Pages as a reference, but basically, create a standalone component that reads OpenApi 3 specs using OpenAPI.NET and translate the models to TOC files and API Page YAML files. We could use a separate docfx openapi command and config sections for generating REST API YAML files, similar to what docfx metadata is for generating .NET YAML files. In case you need anything that the current API Page schema cannot represent for OpenAPI 3, let me know and we can figure out how to enrich the API page schema.

[dougclutter]

Hi @yufeih,

OpenAPI.NET is definitely the library we would use for OpenApi 3 support and I agree that redefining the models was not a great technical choice, same could be said about the .NET MREF approach.

Please understand that I meant no critisism of your code. docfx has been tremendously useful to our Team and we can't thank you enough for all you've done to get it to where it is.

I read over the links you provided, and it looks like you're recommending a radically different approach for OpenApi 3 support. Currently, Swagger 2.0 documentation is generated as a single page. If I'm understanding you, you're recommending that OpenApi 3 generation works more like the code API generation: a toc.yml file with separate topic files. While I agree this would be a better approach, I'm worried about what this will do for folks that are currently generating REST API docs. So, I have a few new questions:

• Should we leave the current code intact, so there is no change in how Swagger 2.0 files are generated?
• If we do leave the current code intact, it might be really confusing because the 2.0 vs 3.0 generation will work so differently. As I've worked on the 3.0 code so far, I've left all the 2.0 as is. Once I got it all working, I planned to go back and delete the 2.0 code so that 2.0 and 3.0 specs were both handled by the new code. Of course, this assumed we were generating a single file.
• If we generate separate files for OpenApi 3.0, how should they be broken into separate pages? Should each API method get its own page (my preference) or should we break on controller/tags (e.g. create a separate file for each H1 section of the current Swagger 2.0 page)?
• If we generate separate files, what do we do for the models? One page for all models or separate pages for each model (my preference)?

Also, you didn't answer all the questions I posted earlier, so I'm kinda blocked until I get your feedback.

Thanks!

[filzrev]

There is another way to support OpenAPI integration. By using Redoc to generate api document page.

How to use ReDoc (CDN version)

---
_layout: landing
---

<redoc spec-url="https://petstore3.swagger.io/api/v3/openapi.json"></redoc>
<script src="https://cdn.redoc.ly/redoc/latest/bundles/redoc.standalone.js"></script>

Output sample

https://filzrev.github.io/docfx.samples.redoc/restapi/OpenAPI/index.html

Limitations

• API document search is not integrated to docfx search.
• Dark mode is not supported.

---

If docfx openapi command are expected to be added. It might be better to supports Redoc-based Static HTML generation by using Redocly CLI.

[dougclutter]

@filzrev - thanks for the suggestion. The link you provided is really nice looking, but the limitations you mentioned really kill it for most corporate/gov't use:

• CDNs are frequently forbidden within the firewall
• Integration with docfx Search is critical (at least for our use)
• I'm not familiar with Redocly CLI, but it certainly looks like something we should look into

@davesmits / @yufeih - any thoughts on Redocly?

[yufeih]

@dougclutter

Should we leave the current code intact, so there is no change in how Swagger 2.0 files are generated?

Yes.

If we do leave the current code intact, it might be really confusing because the 2.0 vs 3.0 generation will work so differently.

We may want to use a different config option for the new OpenAPI pipeline. The old pipeline is configured as:

{
  "build": {
    "content": [{
      "files": [ "**/*.swagger.json" ] // <-- Include swagger JSON files
    }]
  }
}

In the new OpenAPI pipeline, we may want to include a top openapi config section (same shape as metadata config) and potentially a standalone docfx openapi command to trigger OpenAPI doc generation:

{
  "openapi": {
    "src": [{
        "files": ["**/*.swagger.json"]
    }],
    "dest": "api"
  }
}

As I've worked on the 3.0 code so far, I've left all the 2.0 as is. Once I got it all working, I planned to go back and delete the 2.0 code so that 2.0 and 3.0 specs were both handled by the new code. Of course, this assumed we were generating a single file. If we generate separate files for OpenApi 3.0, how should they be broken into separate pages? Should each API method get its own page (my preference) or should we break on controller/tags (e.g. create a separate file for each H1 section of the current Swagger 2.0 page)? If we generate separate files, what do we do for the models? One page for all models or separate pages for each model (my preference)?

Today 2.0 spec supports generating a single page, as well as multiple pages grouped by tag or operation using plugins.

For the new pipeline, we can start with generating operation level pages similar to MS Learn. MS Learn duplicates the models and embeds them in the corresponding operation level page, there is no dedicated models page, we can follow the same pattern.

---

It appears the current code is very forgiving of JSON that doesn't follow the Swagger 2.0 specifications. Was this an intentional decision? Can we be a little stricter with OpenApi 3?

Yes we could be strict with OpenAPI 3, if that is needed by OpenAPI.NET, I suppose we aren't doing additional checks that are not related to loading the models and generating docs.

The current code is heavily dependent on Newtonsoft. Should we transition to STJ? Currently, SwaggerModelConverter creates a RestApiRootItemViewModel. Initially, I thought RestApiRootItemViewModel would be a good target for OpenApi 3 docs, so the rest of the code could remain unchanged. However, it looks like RestApiRootItemViewModel may be too closely tied to Swagger 2.0. Any thoughts?

With the API Page approach, we should be able to move to STJ and decouple from RestApiRootItemViewModel.

---

any thoughts on Redocly?

Looks like a great alternative. Docfx probably won't provide built-in support for Redocly that integrates with various aspects of docfx (theme/search/PDF, etc.) due to the maintenance cost of keeping up with a big dependency.

[ipanin]

Support for OpenAPI 3.0 is a "must have" feature for documentation in 2024...