Open Jermolene opened 9 years ago
welford
commented
9 years ago SSG entirely in the browser would be fantastic. Requiring nodejs, a batch file and a tiddlywiki.info are big hurdles when recommending TW SSG to non programmers.
I've tried as much as possible with my SSG to have everything available and editable in the actual tiddlywiki itself and only do the necessary viewtemplate switching at export time.
Jermolene
commented
9 years ago necessary viewtemplate switching
I was curious about that - why not just render a completely independent template tree? A lot of the PageTemplate stuff doesn't apply in the SSG case, and we'd still be able to reuse the core view template.
welford
commented
9 years ago I am using a custom template but currently i'm only switching out $:/core/ui/ViewTemplate/subtitle and $:/core/ui/ViewTemplate/tags to render slightly differently in my blog.
It seemed easier to me to keep the rest of the viewtemplate stuff as it was ( and still call <$view tiddler="$:/core/ui/ViewTemplate" format="htmlwikified"/> from the template ) than have to maintain a separate custom ViewTemplate for the blog.
Jermolene
commented
9 years ago That makes sense. A useful refactoring might be to parameterise the view template via variables that can be set outside the transclusion, as we do in $:/core/save/all with the filter identifying the tiddlers to be saved. We might use the variables to disable specific view template segments.
dullroar
commented
9 years ago On 4/25/2015 2:52 AM, Jeremy Ruston wrote:
As a first step, perhaps can we list out the possibilities for the feature set? (I can edit this list to reflect discussion):
- New page templates
- New themes
- Generate Atom/RSS feed
- Generate |sitemap.xml|
- Stretch goal: A way of doing SSG entirely in the browser
What do others think?
I would add a few other things:
a. Better SEO by having links as actual /A /elements. b. Faster page serve times by not having everything in one honking big file. c. Cacheability of individual pages and assets. d. No editing capability of the generated content (obviously). e. No or less Javascript. But which? No Javascript, or a subset of TW's Javascript functionality (for things like tags, sidebars, etc.), or user's choice? If user's choice, does that include frameworks other than TW (jQuery, Underscore)? How to support those in the UI during authoring? In other words, is the authoring (TW) environment expected to allow the pages to work in "preview" mode the same way they are once generated?
I have since moved on to other mechanisms to "scratch my own itch" in terms of all this, but my two TW plugins are still on Github and BSD licensed so feel free to fork them or ignore them as needed. I am sure they will need work, but at least give a starting point.
pmario
commented
9 years ago e. No or less Javascript. But which? No Javascript, or a subset of TW's Javascript functionality (for things like tags, sidebars, etc.), or user's choice? If user's choice, does that include frameworks other than TW (jQuery, Underscore)? How to support those in the UI during authoring? In other words, is the authoring (TW) environment expected to allow the pages to work in "preview" mode the same way they are once generated?
I think, the default static version shouldn't contain any js at all. All the bells and whistles can be added by the user. So the TW export static mechanism should be framework independent but open to add them later.
Since TW itself is a framework, I think a preview that needs eg: jQuery or even jQueryUI is impossible to do within TW. So just do the export and a reload in a second window or tab. ... So no additional development needed in TW. ...
The major browsers already contain enough dev tools to handle that very well. .. eg: FireFox brings the whole Web-IDE. So why should we invent something that would be just a hack, if we can use much better existing tools.
Jermolene
commented
9 years ago I've updated the original post to add support for tags and image galleries.
In terms of the JavaScript question, I'd favour following the principles of progressive enhancement. We should generate semantically correct and complete HTML pages that can optionally include JavaScript modules that enable interactive behaviour.
welford
commented
9 years ago In it's current state It's not so difficult to have tags link to pages which list all posts with that tag, I autogenerate these with my exporter. I'd also prefer not to have JS unless the user wishes to add it in themselves.
twMat
commented
9 years ago I hope the following is relevant for this context. It's several bits so please don't stop reading after one sentence...
I'm re-listening to hangout # 84 where @Jermolene talks about federation and how, if I understand, communication could rely on importing the other persons full wiki to then extract the tiddler that the sender has composed. I'm thinking that in the case of someone actually composing tiddlers intended solely, or at least primarily, to be fetched by someone else then it would make sense to instead export these tiddlers, assuming there is some appropriate storage location and that the tiddler is then in an appropriate format. So, proposals for formats? - that one.
Just maybe one of the generously free tiddlyhosting services woudl be willing to allow for uploading exports in one go. (tiddlyspot? ...tiddlywiki.com? Jeremy did metion an idea with having tw.com as a central hub roaming the net for exported tiddlers) What restrictions would such hosts demand? What formats?
Apropo communication: it would be very cool to be able to email tiddlers. While a "TW email client" is for another day, could we make it so that tiddlers are in the "most interactive format possible" working in standard email clients? As most of you know, typical email clients can post html email. I believe javascript is not permitted generally, but that CSS is. As e.g tabs might be possible, as well as image sliders... hm, I was actually looking for reveal widget type sliders there - what are those called generally?... anyway, maybe such could be incorporated too for folding text, maybe even ToC's. Here's a css scroll. This would make for pretty impressive emails if it were possible. And imagine the spread and exposure TW would get!
Here's a tables-to-tiddlers thing I've been peeking at. Kind of makes me think the reverse might be of value (I don't need this one myself but mention it in case someone else does...)
Possibly trivial? - some years ago it was popular with "mini websites" in marketing. Basically one long static site focussed on convincing talk interspersed with a repeated link to the payment page. I think it would be appealing if it is possible to make a great such site, perhaps following a guide/template, and just press export and it's a ready html file where you must just connect a few links to some payment solution.
Returning back to the Federation theme; a while ago there was much talk about "name plates". I'm not sure how strictly these were defined but for Federation I believe it can become a valuable concept. Maybe the can be used as identifiers? If they include some kind of encryption key... I bring it up here because I envision them (but maybe that's just me?) as static tiddlers that will very much be exported and sent around. (Particularly nice if they could be included also in emails, e.g as signatures.)
...btw, the general thought of exporting encrypted stuff to be unlocked elsewhere, possibly by someone else, is generally appealing. Privly is an interesting project I found out about some time ago. The basic idea if I understand right, is to protect your privacy by not posting stuff directly on e.g social media but instead post links to encrypted posts that those invited can then read directly. kickstarter, homepage, dev, git. - Imagine having your own TW and export, not a tiddler, but such a link to social media to direct people to your TW. Don't ask me exactly what I mean here as I don't know ;-)
twMat
commented
9 years ago Worth peeking at:
Ref. Mgmt software file formats. Notetaking software ...etc, depending on intended application.
danielo515
commented
9 years ago From my point of view, static versions should have a similar interactivity than a real TW. If I choose TW over other solutions is because it's interactivity and the ability to link ideas and contents. I want to use TW for some projects, but I'm a bit worried about SEO and content visibility. Tha's the only thing that attracts me to static site generation.
For example, a key thing is to "resolve" transclusions before generating the static content. I'm not sure if this is done.
Here is a list of things that I think should be included that depens on JS:
Suzy1970
commented
9 years ago My preferences:
I upload the static pages to my own site, as TW isn't suitable for this (it can't be made truly read-only), so making the process of creating a static site easier would be a big factor for me in committing to TW.
Jermolene
commented
9 years ago @twMat I think you're really riffing about the differences between two scenarios:
There are pros and cons with these approaches, and I think it's important that TiddlyWiki supports both (and other related variants). In any case the really important thing is evolving the conventions that we use in the community; for instance, the convention of your "name plate" idea.
Jermolene
commented
9 years ago @danielo515 I think the kind of interactivity you're talking about is perfectly possible, but it might come in phase 2. In the case of macros like the tabs macro, the idea would be for the static site generation to use an alternate tabs macro that generates the markup needed by (say) the Bootstrap tabs JS. Generally, it's going to be easier to adapt tiddlers that use macros rather than raw widgets.
What about search functions?
I think initially we'd rely on Google/Bing etc for that:
https://www.google.com/work/search/products/gss.html
Paging management
I'm not sure what you mean there.
Jermolene
commented
9 years ago Hi @Suzy1970
- I would like the tag buttons to be clickable without Javascript - or replace them with text links; at the moment I am manually adding text links at the bottom of each tiddler to compensate, which is a bit of a chore!
I think tag support is indeed essential.
- Ideally, not need node.js to generate a static site, if that is possible? The process as it is now, is a bit convoluted and not user-friendly.
I think we can do it. The problem is that there's no direct support for saving multiple files in one go from the browser. My plan is to instead generate an uncompressed ZIP file containing the files.
- I am happy with a minimalist/basic static site that is accessible and not JS-dependent, as long as it is standards-compliant (HTML5)
I agree that HTML5 compliance is important. Of course, users will be able to write wikitext that is not HTML5 compliant.
I upload the static pages to my own site, as TW isn't suitable for this (it can't be made truly read-only), so making the process of creating a static site easier would be a big factor for me in committing to TW.
I've added another stretch goal which is to upload the static site directly to a server. I'm pretty sure we can do that for a TiddlySpot server.
danielo515
commented
9 years ago In the case of macros like the tabs macro, the idea would be for the static site generation to use an alternate tabs macro that generates the markup needed by (say) the Bootstrap tabs JS. Generally, it's going to be easier to adapt tiddlers that use macros rather than raw widgets.
What about generating one different HTML page per each possible state? Too many pages?
With page management I mean like blogs and /or forums. For example, if we have a "story river" ordered by some way and it has too many items, divide it in different pages.
I think a cool idea would be to also to construct a "page with links" per each tag. The links on that page should be static tiddlers tagged with that particular tag.
Jermolene
commented
9 years ago With page management I mean like blogs and /or forums. For example, if we have a "story river" ordered by some way and it has too many items, divide it in different pages.
Ah, that makes sense. I'll update the top post.
I think a cool idea would be to also to construct a "page with links" per each tag. The links on that page should be static tiddlers tagged with that particular tag.
That's what the top post means by "tag page". I'll edit to clarify.
welford
commented
9 years ago As far as I know tag support without javascript is currently possible with only small changes to view templates . Some of these changes could easily be brought into the core.
when i export i replace this: https://github.com/Jermolene/TiddlyWiki5/blob/master/core/ui/ViewTemplate/tags.tid with this: https://github.com/welford/twstaticblog/blob/master/export/viewtemplate-tags.tid
Which is just adding a link element around the transcluded tag template. There are some filter changes, but they can be ignored as they are specific to how i do things.
There is a "tag" build in here which exports all the tags i want:
https://github.com/welford/twstaticblog/blob/master/example/tw/tiddlywiki.info
the filter at it simplest could be something like [tag[blog]] instead of my custom filter. All the relevant tags get passed through this template which could also be added to the core (or something like it) https://github.com/welford/twstaticblog/blob/master/helper/template.tag.html.tid
TW as it is is 95% of the way to being able to export tag pages (and already able to do it with a simple plugin)
welford
commented
9 years ago Also:
I have done a lot of work on my plugin to make the exporter be as customizable as possible without the need to know anything about filters or javascript. I think something like this would be of use in the core as it's easy to set up and understand.
I have a few control tiddlers:
ctrl.post Any tags in this tiddler specify what tags we use to identify exportable tiddlers ctrl.hidden Tags in this post are hidden from my autogenerated tag index pages and from being shown as tags in exported posts. ctrl.donotexport Tags in this post can override those in ctrl.post so, for example, ctrl.post may contain "blog" to exort anything i have tagged with "blog" but I can add "draft" to the tags in ctrl.donotexport so any blog posts with that additional tag are ignored until I have fully completed the tiddler.
also, maybe less relevant to the core but still useful:
ctrl.donotindex I autogenerate a story river of my most recent 20 posts, but there are some blog posts that i do no want to include in that index. Adding tags to this specifies which tags will denote a post is not to appear on the main story river.
I have others, but they really depend on how you are constructing your static site. In my usecase I tag the control posts as follows
ctrl.post Tags: blog ctrl.hidden Tags: blog, draft, unindex ctrl.donotexport draft ctrl.donotindex Tags: unindex
Which means that:
But all of this is easily customized, just change or add different tags :smile:
You can see http://phasersonkill.com/tags.html shows none of the hidden tags, even though they exist in my TW.
Jermolene
commented
9 years ago Thanks @welford. I like the indirection of the tags on ctrl.post specifying the tags identifying posts.
Jermolene
commented
9 years ago Another possibility I have thought about in the past is whether we can re-use templates designed for other platforms, eg WordPress themes. In practice, WordPress might not be a good example because I think their themes include lots of executable PHP code.
sukima
commented
8 years ago SEO, Lets chat…
Many single page apps mange the SEO problem with the following solution and I happen to agree this is the correct solution because it addresses every ones concerns. Discourse (as explained in this thread) simply sniffs the user agent. If it is a bot it provides a static page. For normal users it sends the one single page app file and the JS looks at the URL to open the correct data.
How this would look with TiddlyWiki:
index.html.static/ directory.index.html and let the JS manage the URL.There are a few problems with the current TiddlyWiki implementation to accomplish this. However, they can be addressed.
Jermolene
commented
8 years ago Thanks @sukima
An interesting further possibility is to produce individual static tiddler files, but have them each reference a .js file that adds a button to the page that bootstraps the full TiddlyWiki.
sukima
commented
8 years ago produce individual static tiddler files, but have them each reference a .js file
This is better because it would not require web server configuration.
I love it!. Gonna go look into it.
tobibeer
commented
8 years ago To me, the purpose of SSG is not at all to produce content "free of js".
SEO is working just fine with javascript, just not to the extend of slicing up a TiddlyWiki into addressable chunks. The key really is that we have something that search engines understand and accept. It is not, that we don't want users to see a TiddlyWiki around things. Therefore, I do think that the idea of providing a more simplistic, static, "preview page", with a rather dominant link to the actual TiddlyWiki seems quite a workable, if not preferable approach compared to adopting all TiddlyWiki features to be fully supported in SSG. In that scenario, the static pages serve but as entry points to the real deal.
I think, the default static version shouldn't contain any js at all. All the bells and whistles can be added by the user. So the TW export static mechanism should be framework independent but open to add them later.
There could be a JS library that works as a helper to implement the extended features that rely on js to work in order to make SSG more general purpose rather than constrained to basic, literally static output, rather than "js dynamic". But this would still better be built with progressive enhancement in mind.
It could be a library of its own or an extension to something generally useful like jQuery. To me, there is no requirement that says "a static site should do no js". That's a constraint that defines a box we'd possibly be better off thinking outside of for the better part of a static site's content and dynamics, ui-wise.
That is how I understand @danielo515's response which I totally agree with in terms of core tenets:
From my point of view, static versions should have a similar interactivity than a real TW. ... I want to use TW for some projects, but I'm a bit worried about SEO and content visibility. That's the only thing that attracts me to static site generation.
In general, I think the TiddlyWiki core features should be designed with progressive enhancement in mind. In other words, tabs could and possibly should first be rendered such that each tab would be more of a box with two blocks, a header with the tab title and a content block with the tab content, preceded by a list of anchor links that allows to jump to the respective tab-title / -box, which possibly now are the tab-buttons. Only after that accessible version was generated there would be some js magick kicking in, or simply css, really, to turn it all into what we currently see as actual tabs in TiddlyWiki. However, this would probably mean that composing something like tabs as a wikitext macro out of low level components like basic widgets makes progressive enhancement difficult if not impossible. So, these core components were perhaps better of as their own widgets to be able to render accessible output first, before being progressively enhanced into full blown ui candy.
Jermolene
commented
8 years ago Hi @tobibeer
To me, the purpose of SSG is not at all to produce content "free of js".
No, I think the "static" in SSG just means that the files making up the site are statically generated. SSG doesn't imply "free of js". So, in fact, TiddlyWiki itself is (often) already a static site :)
SEO is working just fine with javascript, just not to the extend of slicing up a TiddlyWiki into addressable chunks.
From earlier experiments, I believe that Google runs enough of TW5's JS to see the default tiddlers.
The key really is that we have something that search engines understand and accept. It is not, that we don't want users to see a TiddlyWiki around things. Therefore, I do think that the idea of providing a more simplistic, static, "preview page", with a rather dominant link to the actual TiddlyWiki seems quite a workable, if not preferable approach compared to adopting all TiddlyWiki features to be fully supported in SSG. In that scenario, the static pages serve but as entry points to the real deal.
I agree. That's the approach I used for the TW European Meetup site, for example:
I think, the default static version shouldn't contain any js at all. All the bells and whistles can be added by the user. So the TW export static mechanism should be framework independent but open to add them later.
I imagine a variety of static site templates, some of which use JS and some of which don't.
There could be a JS library that works as a helper to implement the extended features that rely on js to work in order to make SSG more general purpose rather than constrained to basic, literally static output, rather than "js dynamic". But this would still better be built with progressive enhancement in mind.
I agree, the JS should be progressive.
It could be a library of its own or an extension to something generally useful like jQuery. To me, there is no requirement that says "a static site should do no js". That's a constraint that defines a box we'd possibly be better off thinking outside of for the better part of a static site's content and dynamics, ui-wise.
I agree.
In general, I think the TiddlyWiki core features should be designed with progressive enhancement in mind
Do you mean TiddlyWiki itself, or progressive JS-enhanced SSG's generated by TiddlyWiki? I don't think there's much scope for TiddlyWiki itself to use progressive enhancement (beyond the options we've already got for baking a static, rendered copy of the content into the HTML file).
In other words, tabs could and possibly should first be rendered such that each tab would be more of a box with two blocks, a header with the tab title and a content block with the tab content, preceded by a list of anchor links that allows to jump to the respective tab-title / -box, which possibly now are the tab-buttons. Only after that accessible version was generated there would be some js magick kicking in, or simply css, really, to turn it all into what we currently see as actual tabs in TiddlyWiki. However, this would probably mean that composing something like tabs as a wikitext macro out of low level components like basic widgets makes progressive enhancement difficult if not impossible. So, these core components were perhaps better of as their own widgets to be able to render accessible output first, before being progressively enhanced into full blown ui candy.
If you're still talking about the progressive JS-enhanced SSG, then I agree.
tobibeer
commented
8 years ago Do you mean TiddlyWiki itself, or progressive JS-enhanced SSG's generated by TiddlyWiki?
I actually do mean TiddlyWiki itself as the basis for the static content pages it can also generate. In other words, I would imagine a tabs macro (or widget) that at first renders things that can be read just fine w/o the actual tabs mechanism. Only then, the js is built around that in order to handle what to reveal and when. For that purpose I imagined each tab to have its title header, which the progressively enhanced, actual tabs would then hide, perhaps simply via css in the full TiddlyWiki, but not in the static page... since all we need in the full tabs are tab buttons themselves, instead of headers to blocks.
My idea would be to have every TiddlyWiki feature first behave in a minimalist fashion that exposes all content one would want to see in a purely static content, so all sliders would be open and all tabs individual blocks. The full TiddlyWiki, however, would enhance that basic scaffolding with the presentational logic that make tabs behave like tabs and sliders behave like sliders. So, the core would already cater both cases, only with SSG we'd not get the enhanced experience, but we also wouldn't just get but the first tab or a closed slider.
tobibeer
commented
8 years ago Btw. I always liked the thought of using TW as an authoring environment for a web page. Not sure if that would be asking to much of it, but eventually, with SSG, I find the idea has quite a practical appeal to it.
Anyway, hope I'll make it to TWEM 2017.
pmario
commented
8 years ago I did like the idea, that we discussed some time ago. It's about TW and basic zip export. So TW could be a "self contained" static site generator, without the need for a server side production step.
danielo515
commented
8 years ago I did like the idea, that we discussed some time ago. It's about TW and basic zip export. So TW could be a "self contained" static site generator, without the need for a server side production step.
paraphrasing a good team member (which happens to be @pmario ) I hope you are referring to that feature to be in the place that we know as the plugin valley (something so desirable that you want it on the core, but not generic enough to be on the core..)
Jermolene
commented
8 years ago There are many dimensions to the question of the desirableness of a feature to be integrated in the core :smile:
I have in fact already made JSZip into a core plugin, and have used it to create zip files under Node.js in a project for my day-job. The plan is to integrate it with the new server route functionality so that one can have a single definition of a file tree, and use it to generate a ZIP or be served dynamically over HTTP.
In terms of static export, I'm increasingly moving to the idea of an official TiddlyWiki read-only static format that works as a static HTML file (and optionally can use JavaScript-based progressive enhancement) that also contains a snapshot of the raw tiddlers that can be imported into a full TiddlyWiki. Having a standard base format would enable us to grow an ecosystem of interchangeable palettes, themes, and behavioural enhancements.
Jermolene
commented
8 years ago Support for interactive features such as tabs, sliders and tag dropdowns
By the way @tobibeer, as it happens, all three of those examples can be done in pure CSS these days, without needing JS. For example, here's an approach to tabs that pretty much matches the HTML that TW produces:
https://css-tricks.com/css3-tabs/
Of course, there are many other interactive enhancements which would require JS.
sukima
commented
8 years ago Just an FYI: I've been working with @dullroar on the TW5-atomfeed plugin.
pesho-ivanov
commented
5 years ago On generating Atom/RSS feed
With the emergence of the git savers (#3931), it is now convenient to host a static tiddlywiki in a repo and operate it through a browser only.
Do you think TW5-atomfeed can be integrated into the git savers in order to generate a feed file and push it to the repo along with the content changes?
Jermolene
commented
5 years ago Do you think TW5-atomfeed can be integrated into the git savers in order to generate a feed file and push it to the repo along with the content changes?
Yes, I'd like to have decent RSS support for TW5 and we could indeed save it alongside the HTML file when saving to GitHub.
welford
commented
3 years ago Hello @Jermolene I'm revisiting this after several happy years using TW for note keeping and recently updating my TW to the latest version. I take it static blog exporting hasn't gotten much easier yet? I have a few questions.
Is there any scope for being able to run something like the --build and --render commands on the web version, without Node.js? I see that there is a zip plugin and I wonder if we could leverage that to generate a static webpage in a zip and a single download.
I still really like the idea of mixing my webpage updates in with my note keeping and just running some command on TW to generate the public facing side of it. While having to use Node.js isnt the end of the world, being able to do all of this from the TW directly would be ideal. Rereading my original notes 5 years on l definitely could have done a better job of explaining my process.
Jermolene
commented
3 years ago Hi @welford good to see you back! Your timing is impeccable: I'm finally beginning work on an improved framework for static site publishing that does pretty much what you describe here, see #5559. I enjoyed re-reading this thread.
welford
commented
3 years ago Very nice. Will see what's possible with that new change. After reviewing it this weekend I'm still very happy with the workflow I had setup for my existing site, It really is just the dependency on Node.js that is slightly awkward.
Both @welford and @dullroar have done considerable work on improving TiddlyWiki's support for static site generation (SSG). Their sites are http://www.phasersonkill.com and http://dullroar.com respectively.
I think we are now at the point where it would be useful to try to merge the best of @welford and @dullroar's work into the core to make it more accessible for others. I'm envisaging presenting the SSG functionality on tiddlywiki.com, not tiddlywiki.com/dev, and trying to make it as accessible as possible to non-developers.
As a first step, perhaps can we list out the possibilities for the feature set? (I can edit this list to reflect discussion):
sitemap.xmlStretch goals:
Related features that would be useful:
What do others think?