Restructuring tiddlywiki.com | monolithic vs. multiple wikis vs. mixed

This extends upon @morosanuae's efforts on restructuring the docs in #2912 to a discussion regarding the structure of tiddlywiki.com.

Demo

I build a realy quick and dirty demo for the "multiple wikis" approach below to achieve greater separation of concerns. Some demo features: links to a tiddler indicate its context, as does the wiki in which it resides (subtitle). Individual tocs per wiki are not implemented in the demo, but it's clear that that would be the next step.

Two Repositories

What we definitely want to change is separating the tiddlywiki core repo from a repository of content about it. The point is to be able to merge content updates (info) independently from core updates (releases) and to eventually allow others to help @Jermolene maintain the content repo. The main point of the demo is to illustrate a need for a main navigation based around clear separation of concerns, so that a visitor will more easily find targeted content, something our monolithic toc fails at right now, whereas we should not even try to bend it that way, imho.

Status Quo

Right now, tiddlywiki.com presents most all information in one monolithic wiki, except, curiously, for dev which was extracted to a subwiki. Of course there are editions for plugins and languages, bot those are more feature editions than content, although as well providing focused content, of course.

What we can easily agree upon is that the entirety of content about TiddlyWiki benefits from separation of concerns. Recently, I started promoting an approach to much more vigorously do that, in a way where there would not just be dev, but also docs, tips, plugins, community, core, node, etc... all kinds of concerns that address a coherent audience.

Separating Concerns

Generally, each content tiddler should be so atomic as to only belong to one concern, that should be House Rule Number 1. So, we do not mix docs and tips or tips and dev, it's an either or thing.

What we do want, instead, is a a straight forward way to link and reference «across concerns» and eventually to also explore lists of related tiddlers in other concerns than the one being the context of the current tiddler.

Now to the various ways to represent content:

Monolithic

The main design decision here is to have (or keep) one big content wiki.

Pros

high referential integrity => we only have one tiddler with a given name overall
makes for easier linking of related infobits
we could easily introduce a global link template that would allow for tiddlylinks to indicate their concern / context as in the demo

Challenges

only one tiddler with a given name tends to overload content where we actually might want a Filters tiddler explaining what filters are in the docs but also a Filters tiddler for plugins that lists plugins enhancing filter functions
separation of concerns is much harder, especially for globally presenting concerns differently => for example, it would more difficult (but still doable) to establish "contextual tocs"
we end up with one big toc as we have right now which is basically too big to focus and find stuff by way of exploring it
we could still have separate tocs for each concern, that could be implemented through (even dynamic) toc switching, either actively by the user via dropdown in the sidebar or by clicking a link to a tiddler in a different concern 5 size will more quickly become a problem

Multiple Wikis

The multiple wikis approach is hinted at by the dev wiki.

Pros

clear separation of concerns
changing the global representation for a given concern is possible / easy
individual tocs per concern are easy
we can use the same title in a number of wikis in the context of the concern
smaller overall wikis

Challenges

more "isolated" tiddlers, esp. affecting "global search", so we may want some "master index" that is searchable accross wikis", at least in terms of titles
some slower load times (when you switch context)
it would be harder to but nevertheless necessary to inter-link wikis
using the same title in a number of wikis / concerns will end up creating some confusion about which concern we talk about => "are we talking about Filters in the docs or in tips? => however, this discussion is probably actually needed / desirable
constructing related lists across wikis would be difficult
if missing, missing tiddlers are external, not internal to the wiki => so it is slightly more difficult to restructure one wiki and be aware of what could break in another
more wikis create an overall bigger footprint => however, visitors will likely not be visiting each wiki

Mixed: Monolithic + Multiple Wikis

allows the the best out of both

Challenges

we will need the feature set of both approaches (for tocs, inter-concern-links, design considerations, etc...)
will end up being harder to maintain, both regarding design as well as content guidelines

Repository

Once we separate core from content, we should definitely split concerns up into different editions all of which could be build to standalone wikies. Even if eventually put into one build, managing content in distinct editions makes it easier to moderate and review content in the context of the concern being addressed.

In any case, we will need some guidelines that help maintainers decide what content is best represented how and consequently under what concern / wiki / folder / edition it is stored in the content repo.

Final Note

I will update this OP as the discussion grows.
As said before, I am quite agnostic as to which route we take, but we should be very clear about (making) the decision regarding how we will re-structure the content on tiddlywiki.com.

Hi @tobibeer thanks for the summary. My own thoughts have taken a slightly different path. Rather than splitting the docs into separate wikis, I'm interested in splitting the presentation of the docs wiki into multiple static pages.

Refactor the core documentation to be a properly usable plugin
Refactor the tw5.com edition to use the new core docs plugin
Switch tiddlywiki.com/index.html to a static rendering of the tw5.com wiki. I'd advocate making it look like a conventional open source project (eg https://www.baqend.com)
Make a new static template based on something like readthedocs for the main documentation pages
Introduce a consistent top menu along the lines of your demo
Move the wiki presentation of tw5.com to tiddlywiki.com/wiki.html

What we definitely want to change is separating the tiddlywiki core repo from a repository of content about it. The point is to be able to merge content updates (info) independently from core updates (releases) and to eventually allow others to help @Jermolene maintain the content repo

I definitely don't want to split the docs into a different repository (see https://danluu.com/monorepo/). Your concerns about allowing others to help with merging are more easily met by moving the repo to the TiddlyWiki GitHub org so that we get the enhanced team capabilities.

What we definitely want to change is separating the tiddlywiki core repo from a repository of content about it. The point is to be able to merge content updates (info) independently from core updates (releases) and to eventually allow others to help @Jermolene maintain the content repo.

Core as in the source code vs the TiddlyWiki.com content? Because I thought it was already separated.

The main point of the demo is to illustrate a need for a main navigation based around clear separation of concerns, so that a visitor will more easily find targeted content, something our monolithic toc fails at right now, whereas we should not even try to bend it that way, imho.

Unfortunately the demo did not provide enough for me to understand the need. Could this be expanded on. @Jermolene would you be willing explaining what maintenance problems your facing?

Could this not be done with simply offering better navigation then to chop everything into non-interchangeable parts? For example my blog site has many different topics: coding, photography, journals, resume, tips, etc. And I managed that with a top nav bar. But all the content is in one wiki. Something I'm trying to ask about because I though TiddlyWiki's main strangth was in that very notion.

What we can easily agree upon is that the entirety of content about TiddlyWiki benefits from separation of concerns.

Can you explain this because I feel differently. I like one place to go. I even find the dev wiki being separate infuriating. When I'm writing a journal I will often go to tiddlywiki.com and leave it up in a tab. I'll use the search and explore the toc. When developing a plugin I'll do the same only to be disappointed that I have to open yet another site.

I think I am confused because for my understanding one of the advantages of TW was in its dump everything approach. You dump ideas (tiddlers) into the system and the system allows easy navigation of those ideas via links, tags, searches, nav bars, side panels, and content tiddlers. This put everything in one bucket has advantages that they all exist holistically. Where as if you had different web applications each one has a different side bar, search results, story river.

I guess what I'm saying is I am not seeing the need to decapitate the holistic nature of the TiddlyWiki philosophy. Another point to consider is since TiddlyWiki is all about small tiddlers working together is the problem just in bad top level presentation of smaller tiddlers? What about tagging tiddlers as documentation, development, usage, tips, etc.? Then you can have a Dev TOC and a Docs TOC, usage TOC, Tips TOC. All which can be dynamically created with a <$list> widget.

Please don't take this criticism as disapproval. I am all for a better documentation story then what we have now. I would just want more clarifications on some of the more subjective assertions above before buying into the notion of throwing out the baby with the bathwater.

I definitely don't want to split the docs into a different repository (see https://danluu.com/monorepo/). Your concerns about allowing others to help with merging are more easily met by moving the repo to the TiddlyWiki GitHub org so that we get the enhanced team capabilities.

I think, we should "play the edtions" game a lot better. ... At the moment the "editions" are not used very well.

eg: de-DE / de-AT does contain some translated stuff from tiddlywik.com. ... So refactoring the overall structure of the main wiki, will also affect the german version. ... Who does the changes there?

IMO the french version is close to 80-90% translated, containing the "then old" structure. ... IMO it should be improved too. right?

At the moment, we don't have many contributors to the "language based" wikis, because it's hard. If there are several users able to push the stuff, it will be much easier

@sukima

Could this not be done with simply offering better navigation then to chop everything into non-interchangeable parts? For example my blog site has many different topics: coding, photography, journals, resume, tips, etc. And I managed that with a top nav bar. But all the content is in one wiki. Something I'm trying to ask about because I though TiddlyWiki's main strangth was in that very notion.

It can of couse be done... but the charme of it is also somewhat its, well, curse. With this approach you can search everything at once. However, what if you really want to perform a more focused search? Indeed, this could simply be an enhanced search-ui just for tiddlywiki.com that allows you to search for "just tips" or "just docs", "just dev" ...or "just node"... rather than all at once.

Alternatively — or in addition, really — the links to tiddlers could indicate the kind of "scope icons" you see in my demo to make it more straight forward as to type of result you are looking at in a search results list, which may even be grouped by "concern".

What we can easily agree upon is that the entirety of content about TiddlyWiki benefits from separation of concerns.

Can you explain this because I feel differently. I like one place to go.

It's really not that I am suggesting "serveral places to go to". The key is to have it be separated concerns, where concerns need separating, but to have it feel like one coherent bunch.

There's always pros and cons.

All in one is what's good about it but also what's bad.

Separated concerns provides much more focused contexts, e.g. "learn wikitext", but then tieing in all the other infobits becomes the challenge.

I am agnostic as to whether we use one big wiki or multiple ones or static pages or the full js wiki.

What is important to me is:

that we separate concerns to more easily explore all info about tw by concern, e.g. "just docs", "just node", etc... without being overwhelmed by aaaalll the other info and drift off.
that it is (and stays) easy or becomes more easy to find stuff, in a focused manner
that we make better use of visual cues (context / concern iconography)
that we provide a meta-navigation which provides scoped entry points in a more intuitive manner than the current toc does
have the toc be relative to a concern, not global, containing everything at once

Cut short, the goal for me, besides manageability of content about tiddlywiki clearly is to have things feel a little less "scatter brain" and a little more assorted into buckets. I'm not at all suggesting that we don't do that. But I think we can do much better, and we kinda have to since the amount of information we collect is becoming ever greater.

As @Jermolene suggested, we might want to have a "tips" section in the sense of a less strict, less concise collection of thoughts for meeting certain ends. This thing especially requires extensive tagging, and it would not be great to have all that content and its tagging interfere with the tagging structure for the docs or that for node or that for dev... because all that information in one spot is simply overkill, unless we introduced a tabbed ui on top of the core that allows to explore search, toc and tagging in a scoped context. So, you would see search results or tags or references or tocs in a scoped / grouped manner, not all in one flat list

I think I am confused because for my understanding one of the advantages of TW was in its dump everything approach.

That is not my view on TiddlyWiki at all. It is not wikipedia and it should not strive to be. Its strength does not lie in accumulating megabytes of information presented in one html file. Its strength lie in its flexibility to arrange and present content, flexibly.

Of course, you can do that in one monster collection of all the knowledge of the tiddlyerse. But at some point you will find that it's difficult to have this one big encyclopedia not just dust away on the shelves. The all-in-one approach might work for searching stuff, sure (think google) ...but not so much for the representation of said knowledge.

So, in this sense, I truly don't get the "disappointment" about information being represented but in the context to which it belongs ...at least not, so long as there is one "go-to-place" that allows me to find the information I am looking for, easily.

What about tagging tiddlers as documentation, development, usage, tips, etc.? Then you can have a Dev TOC and a Docs TOC, usage TOC, Tips TOC. All which can be dynamically created with a <$list> widget.

We can sure do that. The challenge then becomes "scoped relations". What do I mean by that?

Say I have the tiddler "limit Operator". Clearly, this one belongs to the docs and surely it is tagged "Filter Operators" or some such. Now, there may be plugins also tagged "Filter Operators" but when I open that tag for the "limit Operator" what I want to see by default are other such operators from the docs scope, not all possible related tips or plugins or what not. However, I actually may be interested in that as well, it should just not be intermingled as a "first class relation" as say the "sort Operator" would be.

So, the main scope would always be the scope of the tiddler I am looking at with (tagging-)references to all other scopes perhaps being accessible in an accordion style interface.

Same for search. I am about 99% certain that a user knows first of all "I want to search the docs scope" and only then for "sort". I really don't want to search "sort" only to skim through a looong list in order to maybe find doc tiddler about sorting. So, in search as well a ui enhancement might make search on tiddlywiki.com a whole lot more "scoped", starting with "link iconography".

Some for "accessing scopes". That is what the upper nav tries to hint at: Provide top level access points to "concerns that are most of interest". That is not the stage where I want to find more details in a toc, it is the stage where I want to start a certain journey, quite a different thing.

Once I am in the "docs > Filter Operators" scope, sure, I may want to see a toc that shows me all kinds of operators or other details about Filters. Perhaps it's a matter of preference, but I never wade my way through the toc to eventually arrive at a destiny. I find that "journey" to be utterly cumbersome and counter-intuitive. Instead, I prefer search... and enhancements in that area that clearly indicate scopes rather than all results in one flat bucket.

I hope I'm making myself a little clearer.

So, the point of the exercise definitely is not to slice and dice contents and to cut their direct relations. The point is to make the direct relations in the sense of scopes or concerns much more prominent than, say "secondary relations".

So, if tiddler A is in scope S and tagged X and then there is tiddler B in scope C also tagged X, then that relation is secondary in nature, to me, compared to tiddler C also tagged X but equally in scope S.

What is a scope or concern and how to assign every single tiddler to one main scope? We can use a tag or a field, I don't mind. We an even use independent system tiddlers to define what belongs to what scope

title: $:/scope/docs filter: [tag[Filter Operator]] [tag[WikiText]] ...

However, I believe, we should not do that implicitly, we should do it explicitly. Whereas I went so far to have it be separate "editions", even to the point of separate wikis (which, of course, is not necessary, indeed).

All I was saying, and maybe I'm a minority in this, is that I go to tiddlywiki.com with the express purpose of 10% discovery and 90% type in search box to get to the documentation tiddler I'm using at the moment. Sometime I even have more then one tiddler open of a set of documentations on the current few things I'm working on. Weather it is trying to remember how to type a <$list> or defining my own macro (\define), or developing a plugin. In all cases I immediately go to tiddlywiki.com

I the search I type a term and choose from the menu. If it is more I open up advanced search. I would be a bit frustrated if once at tiddlywiki.com I then had to guess which context my information is in and change the mode each time. If I look up information on getting started to see that neat plugin but also want to grab the reference to that nifty filter I want to use I now have to have two contexts separated while as it is now I can have both in the story river.

My comments were based on the idea that the search was pre-filtered based on what mode or wiki edition you happen to be on. I'd be very confused to wonder why LinkWidget returns no results only to realized Oh I need to be on a different tab. It seem counter intuitive to me.

My suggestion was to better designate tiddlers with tags and then have a well organized series of toc tiddlers that help guide new users through the many contexts but still have one place that advanced people who know what they want to search on can go.

For example my home page has contexts for programming, photography, resume, tip, and journals. I can write tiddlers for any of these. And then I have top navigation tabs that open up a overview tiddler for each context along with a list of related tiddlers. But it is still one wiki. The comment aboput it being a dump is my understanding from applying the slogan "Your messy thoughts, organized" and the Getting Things Done concept of the inbox.

One of the reasons I choose to use TiddlyWiki as my homepage was that to contribute a concept I didn't have to wrap it in a blog post. If I had a code snippet or link I could just make a tiddler and tag it. The discover-ability is based on the tags and if I want to make a full blog post I just tag the tiddler as an Article which I could then transclude other less developed tiddlers tucked away in the bowls of the wiki content.

I honestly am not trying to offend. I was only asking for more clarification because just stating that something can not be disputed or is absolute is not true. I am not yet subscribing to the idea of chopping tiddlywiki.com into several independent and unrelated versions is the best option. Relying on a higher tier search engine to pull the content together seems counter to what TiddlyWiki has been doing so far.

Clearly organizing content into buckets does make sense but to place walls an barbed wire around each bucket seems odd to me. For example, say I wanted to offer a one sentence summary on the LinkWidget. In the case of separated contexts that same sentence now has to exist in two or more places. While in one wiki the new user tiddler that explains widgets at a high level and has a list of specific widgets could transclude that sentence from the more developer-centric tiddler about the same widget. Maybe have a GettingStarted tiddler for new comers and a DeveloperIndex tiddler for listing out a hierarchy table of content for more advanced users and a TipsSection tiddler to list out all the tiddler tags with Tip.

I'll end my grandstanding and support what ever efforts we have reached consensus on. I was only attempting to input my use-case and point of view to be considered. Please don't feel I was trying to actively oppose the ideas. In other words I will step back and allow others to make comments in this thread and let the discussion flow unimpeded from this point. And again I apologize to anyone who has felt like I have over stepped any bounds or deteriorated this thread in any way.

I came upon this discussion while searching for a solution which is precisely described by @tobibeer. I was totally surprised how close the discussion is to what I wanted.

I'm a researcher in academia and have been using TW for the past 3 years. I have almost all of my notes and workflow passing through TW. If I have to create and search through material only for myself, a single HTML file would be enough. However, now when I think of collaboration, I think of keeping my TW separate from our collaborators and yet sharing large pieces of information which they can also edit ( think hierarchical accessibility). A student working with me could also keep their TW which they own and take with them while leaving away from the group. For focus purpose as well, it'd be nice to create such disjoint "concerns".

Simply put, I'd say a collection of information as a monolithic arrangement is ok but, every person has their own way of creating tag universe, and interlinking still aiming to communicate with others. This difficult task would require something like what @tobibeer has described.

P.S.: I don't understand several aspects of websites, databases, and other related algorithmic and programming details which limits my vision to see many other possibilities.

Hi @rahulkashyap411 this ticket is an old discussion about restructuring the documentation on tiddlywiki.com.

It sounds like you're asking how to structure your own content as multiple wikis. I'd suggest making a post on the main TiddlyWiki mailing list to ask for assistance: https://groups.google.com/group/TiddlyWiki

TiddlyWiki / TiddlyWiki5