[Feature] Split / de-bundle OpenAPI spec to directory structure

[mosheeshel]

First thank you for this tool, the multi file notions conforms very well with our API documentation methodology, and the tool seems very well done!

Wondering if there is a thought to add a capability to split an existing spec (single yaml/json file) into this structure.

Two use cases I can see for this:

1. Existing project looking to move from a single file to multi file structure - this would be a great migration path.
2. Many teams work with one of the many editors out there which support only the single file method - this would allow a path back from editor work (i.e. bundle->edit->debundle->review->commit)

One thing is that I didn't see in the openapi spec any definition on how a bundle looks like - the spec does say that splitting is OK, but not how. (for example we chose originally to split, and every file had to have the required openapi header info to make the file parsable on it's own.

[RomanHotsiy]

Hey @mosheeshel!

Thanks for reaching out!

Have you seen https://github.com/Redocly/create-openapi-repo?

It covers the use case 1

As for the use case 2 there is a major bottleneck: there is no reliable and fast yaml AST parser out there so splitting loses formatting and comments which is not acceptable for bundle->edit->debundle->review->commit flow.

[mosheeshel]

Thanks @RomanHotsiy for the quick response. I saw the repo previously, but missed the split functionality, I thought it was more like a template repo. Since it uses the CLI for the other things, I wonder why the split isn't in the CLI as well.

Regarding your answer on 2 , isn't the loss of data/format on de-bundle issue relevant for 1 as well? I would assume the existing split functionality works in a somewhat deterministic way (i.e. two runs will produce the same output - so it might be a cool functionality to test - sacrificing some data - or is it really slow (how slow)?

[adamaltman]

@mosheeshel on issue 1, you generally split once and review it. Because the split is typically a one-time activity, one-time review is acceptable. It is true we lose comments but also single file formats are often machine generated and often have few (if any) comments.

The splitting typically leads to further preference-based restructuring.

Regarding number 2, we’re looking to integrate this tool into various authoring tools (eg VS Code), to offer a consistent authoring experience. Even to make this practical we had to improve the performance (we have an alpha which we hope to release in 1-2 weeks). Of course, none of this is obvious with a. “Pet store” type of API, but with large enterprise APIs

Also, peer review is much easier to do in a split repo format in a tool like GitHub. A diff on a 30k line YAML or JSON file is more painful to understand the level of nesting it may be at, etc... Therefore, we are advocates of keeping the source of truth in a split format in a GitHub repository.

[mosheeshel]

@adamaltman thanks for clarifying.

We also use a split file solution (we wrote our own bundler I think before yours was available) - with a ~200 R&D group it's likely that several teams are working on multiple APIs at the same time - this reduces friction considerably, also they can own their sections. However we do have an issue of speed in previewing changes, our tool is not stand alone and also verifies compliance with our API Gateway configuration (ensuring the paths of both docs and gateway match, and certain requirements are filled in the docs).

[adamaltman]

@mosheeshel you will want to look for our next major version. 1-2 weeks. You'll be able to write rules to ensure compliance across both docs and gateway (and it will be fast to execute).

In progress PR: https://github.com/Redocly/openapi-cli/pull/146

[mosheeshel]

A 337 files PR, goodness gracious!

[adamaltman]

Now, I think both use cases are valid, even with the caveat that the split->bundle will be lossy with respect to YAML comments/format. I've thought about it more and think, "so what?" If it enables teams to be more productive, great. It means it is split to an opinionated split structure. That is one possible drawback -- eventually if there is demand, perhaps that split algo and structure could be configurable. However, we might never need that level of configuration?

So effectively, I will morph this proposal into bring the split command into openapi-cli. Maybe we can morph create-openapi-repo into more of a template repo?

[mosheeshel]

I think it's a good direction, I think an opinionated solution is not a bad idea to begin with, it helps form opinions 🙂 especially that there is a willingness to support configuration of there is demand. The lossiness can also be fixed at least for the comment loss, again if there ever is demand. The back & forth capability makes it a more useful tool IMO

[andriyl]

Closed by #206