Parsing Hugo templates in the browser

[erquhart]

I've been working on template parsing using portions of Hugo with GopherJS. Parsing Golang templates is pretty straightforward, but supporting Hugo's template functions, Scratch, etc, seems to require running most of Hugo.

The goal is to allow Netlify CMS users to make edits to their Hugo sites with realtime previewing, without having to port all of their templates to React components by hand.

We should only ever need to process a single page, and we can provide any needed files for loading into afero's Memory backend, so a test-like approach such as the one used for page_output_test is close to what we need, but as written, still requires pretty much all of Hugo in one way or another.

Any thoughts?

[bep]

seems to require running most of Hugo.

And you cannot run most of Hugo? I'm not sure how many of the template functions etc. would make sense outside of a site context. You could mock a site, but that sounds wasteful when you can just ... build the site you are working on.

[erquhart]

The issues with Gopher are primarily file system ops, which it can't run at all, and large dependencies like fmt. Simply importing the "os" package or "fsnotify" keeps Gopher from successfully building. The hope is to somehow abstract template parsing away from the rest of Hugo, or at a minimum, just getting together a minimized port that will run in Gopher.

[bep]

Yea, I understand GopherJS (https://github.com/bep/gr) -- but I suspect that you will shoot yourself in the foot and create something that isn't very useful if you take the full abstraction route.

I'm not sure what your goal is here, though, so you might start by telling us a little about that. Is it a Go template editor with some syntax highlighting? Or do you render the templates as well? If the latter you kind of need Hugo's data structures. And the easiest way to do that is to somehow build the site.

• GopherJS has no problem with the fmt package (other than its size), and fsnotify is only used in server mode. I suspect that a "site build" could be made working.
• If that does not work, or if it's not important if the preview/site data are real, a simpler option may be to serialize some "site" to disk and create and create a site from that in GopherJS (add some kind of SiteFromJSON func in Hugo.

In any case, you need some "data context" to send into the templates -- not just template funcs.

[KevinGimbel]

The goal is to allow Netlify CMS users to make edits to their Hugo sites with realtime previewing, without having to port all of their templates to React components by hand.

@erquhart Why not run hugo against their site and display it in an iFrame? The preview would re-build in real time and you wouldn't need to parse anything. And since they have an URL with Netlify already, the preview could run on a different port, e.g. orcas-are-cool.netlify.com:8080, which is then displayed via iFrame.

[erquhart]

@bep right, let me explain a bit more of the goal.

Netlify CMS is a single page app that edits content and data files. It's build tool agnostic, so it's not opinionated on what you do with the markdown/json/yaml/etc that you edit with it, and unaware of your templates, but it does show a realtime preview of your edits. This needs to happen in the browser because we need to pass new data to the template every time the user types or otherwise changes content. Netlify.com is a Hugo site that uses Netlify CMS, and editing a blog post looks like this:

Currently, a separate preview template has to be written as a React component to approximate the real template. What I'm trying to do instead is use the actual source template and have it be built as it will in production, e.g. using Blackfriday with their site's configuration.

There are a number of challenges to this approach. For one, you usually just want to preview the content, and not, say, the site header and footer. Many templates require Site context, but I'm wondering if we can just get the data to populate Site without actually building the other pages. Or perhaps we can actually build the site when the editor loads and then "watch" for changes - but I'm guessing that's not possible, and could be a no-go just for memory issues alone (for sites with a large number of pages).

We could also strip certain bits out of templates to dumb them down, but it's hard to imagine this not crippling the experience.

@kevingimbel Netlify CMS runs as a single page app in production, so it all needs to happen in the browser.

[bep]

I will think about this, and get back to this ... I usually get the best ideas while skiing. How is Blackfriday working in the browser?

[erquhart]

Sounds good! In a very small but functional POC I imported Blackfriday into a GopherJS function, along with ports of some Hugo template functions:

https://github.com/erquhart/netlify-cms-template-parser-go/blob/master/main.go

The Scratch stuff in there does not work, but everything else does. Netlify CMS passes a mock Hugo data object with the realtime edited data, along with a template string. Currently have to run some replacements on the template to remove partials and such.

[bep]

The Scratch stuff in there does not work, but everything else does.

I assume the "everything else" is a relative term.

[erquhart]

Sorry, that's extremely relative: by "everything else" I meant everything else provided in that package I linked. A Golang template using only the functions provided in that file can be parsed as expected, including markdownify, although it's just Blackfriday with default configuration rather than Hugo's actual function.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. The resources of the Hugo team are limited, and so we are asking for your help. If this is a bug and you can still reproduce this error on the master branch, please reply with all of the information you have about it in order to keep the issue open. If this is a feature request, and you feel that it is still relevant and valuable, please tell us why. This issue will automatically be closed in the near future if no further activity occurs. Thank you for all your contributions.

[bep]

@erquhart sorry for not getting back to you on this. I certainly have thought about it. It is a hard problem, but I think we can get there in the end...

[erquhart]

Definitely, agreed! Thanks for keeping it alive.

[stp-ip]

Does the latest WebAssembly announcement for Go help? Could part of a website or an iframe be WebAssembly? Would that even be worth it so. Feels quite heavy probably.

[stephendwolff]

@erquhart did you do any more on this in the end? WebAssembly as @stp-ip mentions or more now the registerPreviewTemplate customisation feature is in NetlifyCMS? If you did, is any publicly available - in a fork or separate repo that could be worked on?