Compiler/Templater

[info-rchitect]

Hi,

So we got no ERB equivalent in Python so we need to go 3rd party. Here is what I have found so far:

• Mustache
• Jinja
• Mako

In general solutions like Mustache would make the compiler much less powerful vs Jinja and Mako that can execute conditional Python. Perhaps it really just comes down to the templating syntax ease of use and the documentation completeness? Send your opinions.

[info-rchitect]

A nice read here.

[info-rchitect]

There is also this Rust version based on Jinja

[ginty]

I think it has to be done in Python rather than Rust since it is key that the template has access to the application variables (which will be in Python).

To have something on a par with O1 it looks like Jinja2 and Mako are the main contenders.

Personally, I prefer the look of Mako - it looks more equivalent to what we got from ERB. That is, you basically write any Python you like in your templates, whereas Jinja2 seems like more of a language subset.

[coreyeng]

From my experience, Jinja2 is a bit more popular. Its the default templating (and built by the team) for Flask (Closest alternative to Ruby's Sinatra) and I've seen it used a lot with Django.

Though, I think both have Mako plugins available, but just from a popularity perspective, I think Jinja2 might have an edge.

FWIW, I've used Jinja2 in my Flask apps before and its great. Never used Mako, but that doesn't mean its not great too.

[info-rchitect]

Why couldn't we support both? The compiler could detect the syntax of the compiled file and route it appropriately.

[coreyeng]

We definitely could, but the nature of the question made it seem (to me) like we (as the core team) should agree on some standard so at least the stuff we write all looks similar. If that wasn't the purpose, we should probably discuss that since having half of O2's core templates in Jinja2 and the other half in Mako doesn't seem like a good thing.

From the end user perspective, they should always have the option to do whatever. Even in O1, its possible with enough meta-programming override the ERB templating with whatever they want. Its just a matter of how much work they want to do. Supporting that in O2 I think would be as simple as accepting a lambda function to compile, in the event that any of the defaults we support aren't needed. Though, that might be going beyond the scope of what you're originally asking.

[info-rchitect]

We definitely could, but the nature of the question made it seem (to me) like we (as the core team) should agree on some standard so at least the stuff we write all looks similar. If that wasn't the purpose, we should probably discuss that since having half of O2's core templates in Jinja2 and the other half in Mako doesn't seem like a good thing. From the end user perspective, they should always have the option to do whatever. Even in O1, its possible with enough meta-programming override the ERB templating with whatever they want. Its just a matter of how much work they want to do. Supporting that in O2 I think would be as simple as accepting a lambda function to compile, in the event that any of the defaults we support aren't needed. Though, that might be going beyond the scope of what you're originally asking.

We should pick one for internal usage. But for user/app specific templates do we care?

[coreyeng]

We care in that we can expose a frontend/optimize supported templating systems and have them natively built in. For example, supporting Jinja2 ourselves would mean we'd bring in the dependencies, and the compile method would support it out of the box.

Problems can arise when something internal calls compile with given arguments that aren't exposed to the user. In Ruby (and in Python) we can monkey-patch our way around this, but it takes work. Consider the web templating in O1: the web files are run through ERB behind-the-scenes, with a given context. Not having a wrapper to pass that context to whatever means more required work by the end users to support not-ERB.

As far as Jinja2 vs. Mako goes, I'm pretty indifferent. I was just stating my prior experience with Jinja2, but I haven't been doing much templating anyway so take my opinion for what you will.

[ginty]

I agree on consolidating on one tool, and that Jinja2 is the most popular.

I don't have a strong opinion either way, I have never used Mako either, though I have used Jinja2 in the past.

What I liked from Mako (just looking at the website briefly today) is that it seems to be pure Python that you write in the template. So it's similar to ERB and the template-specific syntax you need to know is really small and can be explained fully in a couple of examples.

Jinja2 on the other hand seems to have quite a large spec associated with it, and to me it seems like it could be a cognitive barrier (albeit not a big one) to those coming from O1 and expecting it to work the same way.

Probably we could condense what you need to know about Jinja into a brief-enough Origen doc though.

[info-rchitect]

This might be the deciding issue:

Mako: Supports Python 2.7 and Python 3.5 and above

Jinja2: Jinja 2.7 brings experimental support for Python >=3.3. It means that all unittests pass on the new version, but there might still be small bugs in there and behavior might be inconsistent. If you notice any bugs, please provide feedback in the Jinja bug tracker. Also please keep in mind that the documentation is written with Python 2 in mind, so you will have to adapt the shown code examples to Python 3 syntax for yourself.

[info-rchitect]

@ginty would these sort of utilities need to be installed as a bin-stub?

[info-rchitect]

@ginty Where should unit testing templates be stored?

[info-rchitect]

@ginty would these sort of utilities need to be installed as a bin-stub? Nevermind

[info-rchitect]

Similarly featured compiler when compared to O1 implemented here

[ginty]

Hi @info-rchitect,

I think there are still some more features required before we can call the O2 compiler feature complete - the main ones being support for sub-templates (with layout capability if possible in Python) and it would be good to formalize how/where helper functions should be written (something that O1 was weak on).

Should we open separate tickets for these additional items?

Basically, we should make sure O1 can do everything in the O2 compiler guides e.g. https://origen-sdk.org/origen/guides/compiler/subtemplates/

Thanks!

[info-rchitect]

@ginty yes separate tickets