search

erwanm / tw-aggregator

a system to automatically aggregate TiddlyWiki content from a collection of public wikis
6 stars 2 forks source link

Speed up by making the default lists static #75

Closed twMat closed 8 years ago

twMat commented 9 years ago

The "default" lists (i.e tiddlers: CommunityTags , CommunityAuthors, etc) - I can't think of any reason to generate them live. They can be static and pre-constructed as part of the aggregation process.

erwanm commented 9 years ago

Actually it's already the case. I just wrote a brief explanation about that in the doc: http://erwanm.github.io/tw-community-search/#TWCSWikiStructure

twMat commented 9 years ago

Hm, but if I click edit on e.g CommunityAuthors, why is there a two layered list fitler there then?

I'm thinking (guessing!) that load time for these default lists (index by tags, by wiki or by author) would be faster if they are really static, kind of the content in what you get from clicking tiddlertool>Export>HTML, but with working tagpills and links etc. I'm guessing it is the double layered list traversing that makes it a bit slow to load (or?) and it will worsen the more TW's are included in the aggregator.

erwanm commented 9 years ago

oh ok, I completely misunderstood! probably because I read too fast...

You're right, but actually I would think that what takes much more time is computing the number of tiddlers and latest modification for every wiki: in some cases that requires TW going through 100s of tiddlers... I'll have to test that, all these things can be pre-computed anyway.

I didn't think about this kind of optimization, it's a very good idea, thanks!

erwanm commented 9 years ago

To perform accurate performance measures: https://groups.google.com/d/msg/Tiddlywiki/-AdwCjE0wgI/3ZyozEB1BAAJ

erwanm commented 8 years ago