search

TiddlyWiki / TiddlyWikiLinks

Link aggregator for the TiddlyWiki community
https://links.tiddlywiki.org/
MIT License
33 stars 14 forks source link

Tag/Topic compression - back end or front end? #34

Open Marxsal opened 2 years ago

Marxsal commented 2 years ago

I was looking into using cascade filtering to combine tags for the Topics menu page.

Using cascades on the front end would be more accessible to most people. Unfortunately, the cascades make tag/topic counts very challenging. A single url may be represented by several tags that all combine to form one tag. Working the cascade backwards turns out to be problematic. I could only do it using wikify, which doesn't work on the main topics page because of performance issues.

A version of this approach would be to use the cascade filters in conjunction with a data dictionary, so that the reverse lookup process would not be so intensive. This would require keeping the cascade and the data dictionary in synch. Possibly by generating the dictionary from the cascade.

A completely different approach would be to have, perhaps, a JSON file of conversions that is used on the backend when the wikis are downloaded and analyzed. This approach would produce a clean slate of url tiddlers that would be already encoded. This would make display counts straightforward. In fact, nothing on the front end would have to change. But since it's done in javascript, it would be a little less accessible to the general public. I say "a little less", because there doesn't seem to be that much engagement in the first place.

So I'm looking for feedback which approach, if any, would be preferable. Thanks!

Jermolene commented 2 years ago

Hi @Marxsal my plan had been to have a map of aliases, so that each incoming tag would be looked up and replaced with a normalised tag by the collection process (although we'd probably want to keep the original tags in another field for reference). I got as far as starting a user interface for setting up the mappings, but put that work aside when I realised I had to work on replacing the original static UI with a TW UI. I'll dig up the branch and see if it's worth picking it up again.

Marxsal commented 2 years ago

I was curious who would run the UI? Would there be a backdoor tiddlywiki.org presence?

Jermolene commented 2 years ago

I was curious who would run the UI? Would there be a backdoor tiddlywiki.org presence?

I was thinking that the UI would be run by people like you and me who might have volunteered to help with the mappings. The UI would hidden away but within the usual wiki. Every so often we would fiddle around with it to make adjustments, and then download a JSON file of the mapping data that would committed to GitHub as part of the wiki. Very rough and ready, but doesn't require us to invent anything new to get it up and running, and allows the editing experience to be decent with autocomplete etc. which we wouldn't get if it was just a matter of editing a file in GitHub.