make it easier to contribute w/easier docs and tooling

[jbergstroem]

After moving to the new home at mozilla, it seems like pace has slowed down while incoming patches are on the rise. As a daily user of nunjucks, its something that concerns me. I'd really enjoy seeing people in the community joining in and improving nunjucks, not just by review. Suggesting (in no particular order) @carljm, @garygreen and @mattbasta.

I guess documenting a review process as well as further expanding on where nunjucks should (and shouldn't be) different to jinja would improve the workflow too. I can help out with that.

[carljm]

Thanks for the vote of confidence, @jbergstroem.

We are using nunjucks heavily and I would be happy to continue to contribute, though I'd like to start by seeing a review of the patches I've submitted so far, so I know where I may be going astray :-)

I will admit that I don't have a fully clear sense of the intended direction of the project. I am using the project because we use Jinja on the server, and nunjucks allows us to share templates. So Jinja-compatibility is high on my list of priorities, and I think it's likely that it's a strong driver of nunjucks usage. But I've gotten the sense from some comments here and there that being "JS-native" is considered a higher priority than Jinja compatibility, so I'd be interested in a clear statement of what the primary goal(s) of the nunjucks project are.

[howardroark]

+1

I'd love to see this project offer up a better narrative as to its direction. I think having a goal of pursuing a template engine that desires to be completely sandboxed and agnostic of language is a really great one. Encourage people to leverage the elegance factor and design intelligent JSON schemas.

Maybe start to champion the idea that the design can traverse languages. Done right this project could start to influence Jinja2 itself or encourage the design to appear in more languages like GO or Rust. Why not build templates that can be useful in many ecosystems!

[jlongster]

You're absolutely right that at least I personally haven't been giving nunjucks very much time recently. The hope when it moved to Mozilla was that some of the foundation folk that use it a lot would start to help manage it, but that hasn't really happened. I keep meaning to spend time on it here and there but then I get busy with other stuff.

I will go through the PRs this week. Some them required decisions that I wanted to take time to think about, but it's easy to never get back to them, which is bad.

I'm happy to try to help manage the PR queue better from now on, and fix bugs. But I'd be even more than happy if @carljm and @garygreen wanted to step up and help manage this project and get commit rights. I'd absolutely love for that to happen, actually, as that's the natural process for a project like this. I can't always be going through PRs.

We should document the review process and give more guidance about what the goal of this project is. In my opinion, performance is pretty important so that means that we can't stray too far from JS, else our runtime will get bigger and slower. We've implemented some Pythonic syntax, but I'm not interested in making is 100% compatible. Jinja has certain syntax that it literally compiles into Python, and I don't feel like we need to bend over backwards to write a Python-to-JS compiler (hyperbolic, yes). There are also things I chose to do (like use true and false instead of True and False) because many people using nunjucks will be JS users and it feels more native. However, I have considered a "compatibility mode" that tweaks a few easy things like that for people that want to share the same templates across languages.

It's a neat idea to share templates across languages, but not one that I'm particularly interested in (beside the JS/Python opportunity here). If someone else wants to champion that, cool. If someone else wants to help figure out a good story for nunjucks, without breaking backwards compatibility, that'd be cool.

@carljm Sorry I waited too long to review your patches. Happy to look at them now. I'm fine if you want to work on things that smooth over the JS/Python bridge, as long as they aren't extremely complicated or slow.

[carljm]

No worries re patch review times; life happens, and I don't subscribe to the theory that releasing code incurs a continuing moral obligation to put time into it. I wasn't meaning to complain above, just to note that it might be premature to consider me for commit rights, since I hadn't actually had any changes accepted into the project yet :-)

[howardroark]

@jlongster I hear yah about not venturing past Python/JS... though that alone is a pretty mighty bridge. Sounds like a pretty good story to tell as well.

I do love writing... Any desire to add one more page to the nunjucks site for a "philosophy"?

[jlongster]

I do love writing... Any desire to add one more page to the nunjucks Jekyll site for a "philosophy"?

I'd be fine with that!

just to note that it might be premature to consider me for commit rights, since I hadn't actually had any changes accepted into the project yet :-)

Well, you've made PRs that should have been accepted (waiting on rebase) that fixed some bigger issues so that's good enough for me. Tomorrow I'll sit down and document the stuff and add you guys as committers.

[carljm]

Oh, and the stuff about compatibility/performance makes a lot of sense. I need to figure out how to run the benchmarks (and when I do so, I'll add some documentation about that.)

[jlongster]

I'd love for the benchmarks to be improved, actually, as they are quite old. We should use this project now: https://github.com/bestiejs/benchmark.js. The current one used is basically dead, and that one is a lot better.

[howardroark]

I'd be fine with that!

Sweet, I'll start mulling around some ideas in my head.

I have considered a "compatibility mode" that tweaks a few easy things like that for people that want to share the same templates across languages.

+1 for compatibility. I prefer capital case for boolean in a visual sense, although outside of that I am not concerned. Maybe just start with only that and see what else tries to find its way in...

[jbergstroem]

From the sounds of above, how about:

• Expand on the "how to ask for features/send a patch" process:
  • which features are undesirable from jinja (as part of its pythonic inheritance)
  • which language features we inherit from javascript that doesn't lead us too far off from "the philosophy"
• What's required in a patch (documentation, tests, performance, code style)
• Use tooling that makes it easier to measure above (jshint, test suite, benchmark suite, ...) and possibly even bake that into a script that people run on their feature branches and paste as part of the issue
• Elaborate on the review process so everyone can jump with feedback. End goal would be to bake it into an easier acceptance for merge (by the committer)
• Possibly also introduce issue janitors and not assume that core submitters should get that task for free.

I guess a combination of docs/code committed to the repo as well as github wiki would be proficient? Travis is a good tool for build testing but not really performance testing since its available resources are unreliable. It could probably verify the tooling step of above suggestion.

[jlongster]

All agreed @jbergstroem. We can work on those things and see where we get. I'll start some docs this week about all that. (I know github does something special with CONTRIBUTORS.md, should look into that).

Regarding docs, I'm not sure where you guys want to go with them. I really like good docs which is why I made a separate site instead of using the wiki. I think github finally updated the wiki, but when I was writing nunjucks docs they didn't even support permalinking to specific sections. They might be good enough now to use, though there would be a significant amount of work to move them over (not to mention there's a Chinese translation of the docs...)

We just need to get people to also add to the docs here: https://github.com/mozilla/nunjucks-docs. It's not that hard, so I guess we just need to make it part of the requirements for PR. Currently when I make a release I go through and update the docs.

I don't think nunjucks gets enough traffic to over-architect all of this; I think if I enabled @garygreen and @carljm to commit then we'll do pretty well.

[jlongster]

Is everyone here on the nunjucks mailing list? Just making sure. I'll say something off-topic here for now though: can you guys try out master on your projects and make sure nothing's broken? I just merged in a good bit of changes, and while all the tests are passing it's good to test on existing projects too.

I'll try to make a release later this week.

[carljm]

My feelings about docs: wikis are bad, separated from code repo is bad. Is there a compelling technical reason the docs have to be at https://github.com/mozilla/nunjucks-docs rather than in a subdirectory of this repo? I think that no matter what we do that will always be a problem; basically it means every significant PR has to become two separate coordinated PRs. That's a major hassle IMO.

[garygreen]

@jlongster justed tested on our SPA for a mobile project and all working fine. I haven't tested any of the new stuff like the call blocks (I think Carl is possibly the best one to test that's all working).

I notice a while ago there was some release notes for supporting browserify in the build process -- is that still the case? I tried pointing browserify to build from the starting point (I assume index.js) and failing to build. I'm running windows at work at the moment and it can be a bit of a pain to build nunjucks as it's reliant on the bash script. Possibly something to look into in terms of making it more platform agnostic.

Error: Cannot find module 'fsevents' from 'D:\xampp\htdocs\ffa-app\bower_components\nunjucks\node_modules\chokidar'

[carljm]

I'm on the ML now. And we're updating our projects to use master, will report if we find any issues.

[howardroark]

RE docs: This project looks promising... http://progrium.viewdocs.io/viewdocs

It lets you keep your docs aligned to each branch and release. It's like ReadTheDocs, but less complicated. I also find RST to be really hard to grok. It is also very auto-magic and requires no hosting on your own behalf.

Jekyll is great, but the need to use an orphaned branch can cause very awkward issues over time. It is not the best for docs. However it is great for marketing!!!

[garygreen]

If we're talking about docs, I've used metalsmith recently and it's been an absolute joy to work with. Incredibly simple, fast and elegant and we can create whatever plugins we like to render things exactly the way we want. I attempted to use Jekyll but got a bit frustrated, mainly because I don't know ruby at all. Metalsmith is a node based docs generator, definitely fitting for this project and worth considering

http://www.metalsmith.io/

[jbergstroem]

re wiki: I don't really care what tool we use as long as it has revision-based history (gh-pages, check. wiki, check) and is easily transferrable onto static sites or as release-bundled documentation (markdown files with scripts that pulls into dist or similar). If we have a nunjucks-docs which is in use and working as-is; I'm happy with that.

edit: @carljm Brings up an important point that pull requests are decoupled -- which in my opinion in a strong enough reason to have it live in the same repo.

Also, this issue is probably moving closer to the end of its purpose. The general discussion might benefit using the mailing list for higher visibility to additional people. I'm really happy with the outcome so far though!

[carljm]

Master seems to be working just fine for both of our current projects (and both of them make heavy use of at least the call tag and for-else).

[jlongster]

Cool. Cutting a release today and will continue to work on docs and such. I'm at a work week this week so I probably won't merge the docs until next week.

[jlongster]

@carljm @garygreen Sorry I haven't given you all write access to this repo yet. For some reason I can't add contributors to that team and I've been meaning to figure that out with some of the Mozilla admins. I will get to it tomorrow.

[garygreen]

Closing this for now. I have been working on new doc tools for nunjucks, hopefully will be ready for you guys to check it out. Will do as PR