search

swagger-api / swagger-codegen

swagger-codegen contains a template-driven engine to generate documentation, API clients and server stubs in different languages by parsing your OpenAPI / Swagger definition.
http://swagger.io
Apache License 2.0
16.89k stars 6.03k forks source link

[Discussion] Migrate the template system from Mustache to other alternatives #3950

Open wing328 opened 7 years ago

wing328 commented 7 years ago

Currently, Swagger Codegen mainly uses mustache as the template system for code generation.

Mustache, a logic-less template system, may not be the best template system for this project and we would like to gather feedback from the community to see if anyone has a better suggestion.

This is not to say we hit some showstoppers with mustache and it must be replaced. If there are better alternatives and tools to ease the migration, we would definitely consider as our goal has always been making developers' (specially contributors) life easier.

jimschubert commented 7 years ago

I would suggest we consider using Liquid. (see Liqp). We may be able to reuse quite a bit of the current templates.

Another thing I'd like to consider is how we define the templating structure. I think it would be interesting to maintain the target directory structure within the language's resources directory. This would work for C# and ASP.NET generators, but I don't know how well it would work for generators supporting libraries.

fehguy commented 7 years ago

Thanks for starting this conversation, @wing328. If you look in the way-back machine, we chose STL for the first version, but it's lack of traction and unfamiliar syntax was a big barrier to using this project and contributing. I'm going to guess that the web world has some really good templates now that we should consider. Mustache was a great choice when we selected it, because it was easy to use and fast to iterate.

If the web experts could chime in with what they're using now, that'd be great input. We can always keep with Mustache but it's definitely worth considering alternatives, even it's sibling handlebars because we're really having to bake a lot of logic into them.

Lastly, it's very important that whatever template syntax we use has good tool support. I know that Golang has a nice template framework but we'd be seriously constrained by the runtime (for now at least).

cbornet commented 7 years ago

If we want to stay on a JVM runtime, Thymeleaf is pretty popular in the Spring MVC community for server rendering and it has a non-html mode

pelepelin commented 7 years ago

IMHO, in web world the most popular syntax style is Django/Jinja/Twig, but Mustache/Handlebars is popular enough, so unless you have a pressing matter, the style change does not worth it.

cbornet commented 7 years ago

To continue the discussion (that can lead us very far...), I think most modern web sites do client-side rendering with logic-less templating and an intermediate presentation layer (MVVM way). I'm not sure this fits well with our (current) architecture where we have language libraries that share java code and have distinct templates. But maybe we could use a subclass or interface impl for each library where they would manage their own presentation. This also means that the libraries should be able to manipulate the template engine (add a handlebars helper function for instance). If we want to go that way (which I think is the cleanest if it is possible), handlebars is probably the best choice as we can reuse our existing mustache and enhance them, it is very well known by the community and very easy to learn.

ClayAtWork commented 7 years ago

The only requirement I would like to add is creating custom functions to be used in the template files, without having to modify the java source. I think Mustache refers to these as Lambdas, and Handlebars or other templating systems might call them filters. For instance, we have a swagger spec that generates codebases for multiple platforms, and we might want to do a simple string transform for a different syntax style. SOME_ENUM -> SomeEnum. It's be great if we didn't have to modify the java source, because this would cause us to have to fork the repo and publish our own artifact, where we'd rather continue to depend on the public one.

I'm actually not sure how this would work in practice, does the Java Mustache implementation support custom Javascript functions for the templates to use?

nzjoel1234 commented 7 years ago

As outlined in this talk https://www.youtube.com/watch?v=Vur2dAFZ4GE#t=4m25s a big problem with templates is that a whole heap of concepts need to be re-invented (i.e. loops, conditionals etc.)

Would it make sense to instead just use something like javascript to build the code?

fehguy commented 7 years ago

@nzjoel1234 I believe if you're just using javascript, you don't need codegen to generate the code. Templates are a must from my POV, we're trying to avoid writing more code...

wing328 commented 7 years ago

FYI. @tgmuender has filed #5365 to support handlebars (as an option)

cbornet commented 7 years ago

Handlebars has the concept of helper methods which could help us escape our current problem of "statically typed language + logic-less template". So shouldn't we try to make it our default template engine ? Seems that option would be the less painful (compared to rewriting the code or our templates in another language/template). We could have a transition phase where each language impl declares what template engine it shall use.

dalbothek commented 7 years ago

Since it might be relevant for others, here's an excerpt of a discussion between me and @wing328:

Do you guys need to convert the mustache templates to handlebars somehow?

No, the reason we chose handlebars and not something a little more full-featured (like Jinja) is the backwards compatibility it offers for mustache templates.

I would like to know more on why you made the switch and the estimated effort to do so for the whole project.

The thing that bugged me most about mustache is the lack of whitespace control. It really made for some ugly, barely readable templates and was a pain to work with. We carried on for a while until we hit another bump when we tried to add a simple conditional to our template which wasn't backed by a boolean property on the model object. Sure, we could have added another such property to one of the classes in the mustache context, or even the additionalProperties, but we thought it ought to be possible to do something like that in the template itself. Proper templating engines often offer expression evaluation in if statements, which mustache clearly doesn't, and handlebars doesn't excel at it either. We debated over a few possible engines and finally settled on handlebars since it allowed us to reuse all existing templates and even merge in upstream changes with little hassle. Like I mentioned before, handlebars doesn't offer anything like Jinja's {% if dataType == "String" %} out of the box, but it was easy to add our own {{#equals "String" dataType}} tag which served our purpose well enough.

After using it for a couple of weeks now I really believe that handlebars offers enough flexibility in this environment and results in both more readable templates and more predictable output. Through custom tags it is even possible to extend the functionality available to templates in some ways, should the need ever arise. Because of the backwards compatibility with mustache I would even suggest to switch the whole project to the handlebars engine without keeping mustache around as the default. It certainly has to be verified, but I believe there should be no difference in output for existing templates.

I suggest we let the discussion in #3950 play out a bit, but I think the 2.3.0 release would be a good opportunity to introduce such a feature. Since I have the impression that you're generally interested in the idea, I'll have a discussion with Tobias (@tgmuender) about how we can improve on his changes and most likely file another pull request with handlebars as the only engine later this week

dalbothek commented 7 years ago

While handlebars is a strict superset of mustache, there exist a number of special variables introduced by jmustache which are not supported in handlebars or the handlebars.java implementation. These are -last, -first, and -index. These are all available in handlebars templates, but with slightly different names: @last, @first, @index. Additionally, @index is a zero-based counter while -index is one-based, but at least in the builtin generators this variable is never used.

With that in mind there are a number of ways we could go forward:

Option 1: Replace mustache with handlebars and adjust existing templates

The existing templates can easily be updated with search-and-replace, but any user-supplied templates would most likely stop working.

Pros

Cons

Option 2: Search template files and show incompatibilities to the user

As an addition to option 1, this might give the user at least some idea about what's wrong.

Pros

Cons

Option 3: Search template files and fix incompatibilities

Ideally, this would prevent option 1 from breaking backwards-compatibility.

Pros

Cons

Option 4: Add special variables to handlebars.java

Currently these variables cause a SyntaxError, but if we could add them manually to the template context we could achieve full backwards-compatibility.

Pros

Cons

Option 5: Keep mustache around as a deprecated legacy option for some time

The simplest way to decide which engine to use would be the file extension. We could convert all templates to .hbs and continue to render .mustache files with jmustache.

Pros

Cons

Personally I think the last option is most appealing, but feel free to discuss the others or add your own idea to the list.

cbornet commented 7 years ago

+1 for option 5. Supporting both engines for some time seems the less brutal. Choosing the engine based on the file extension seems a good idea.

tgmuender commented 7 years ago

I would go with option 5 as well. Though I think the ideal way would be to try to generify/abstract the whole templating part to facilitate adding new template engines.

jimschubert commented 7 years ago

I agree with option 5. Also, handlebars is much more pleasant to work with in my opinion.

dalbothek commented 7 years ago

Small status update: there's a first draft of this feature available here. It still needs a little cleaning, some documentation and of course a lot of testing.

I've started out with the goal to make as little changes as possible, but quickly realized that this approach would just add to the mess in the current codebase. Instead I tried to extract most of the templating functionality to a separate module which is much cleaner and reduces the amount of duplicate code in the generator immensely. I've tried to keep the old interfaces intact, as to not interfere with any other projects, but deprecated some methods in favor of the new templating subsystem.

There currently exists a small issue with whitespace control in the Handlebars implementation I used, I've documented that here and might fix it myself if I can find the time.

ThuF commented 5 years ago

Hi,

I'm trying to introduce support for a new language and it will be really great to have a "logic" in the templates. We've tried in Eclipse Dirigible to use Mustache as templating engine, but it doesn't really fit the requirements. As swagger-codegen really targets way more complex scenarios, I can recommend Apache Velocity and here you can see how it's used in Eclipse Dirigible.

Regards, Yordan

windymindy commented 4 years ago

I was searching for an awesome list of universal schema generators. The best I've found are the GSL tool and a blog post. https://github.com/zeromq/gsl https://tomassetti.me/code-generation/