search

erwanm / tw-aggregator

a system to automatically aggregate TiddlyWiki content from a collection of public wikis
6 stars 2 forks source link

New application: "Community Broadcast" #69

Open erwanm opened 9 years ago

erwanm commented 9 years ago

General idea: allow wikis to collect TW content from other wikis through the iframe mechanism, as demonstrated on Mat's wikis http://tiddlersidebar.tiddlyspot.com/ and http://communitycomments.tiddlyspot.com/.

twMat commented 9 years ago

In deed. For anyone who wants the quickie explanation:

Anna has a tiddler AnnasComments with comments on e.g PluginA and TiddlerB. Bob has a tiddler BobsComments with comments on e.g PluginA and TiddlerC

Charlie has a TW with an aggreagator. It fetches AnnasComments and BobsComments. It also has a viewtemplate that takes the name of the current tiddler and uses it as an argument to filter the comments. For instance, if the current tiddler is named "PluginA" it extracts the comments in the comment tiddlers that concern PluginA.

Anyone can in their own TW put in an iframe with it's src being a permalink to PluginA in Charlies TW to see this tiddler. The coolest bit, IMO, is that Charlies TW doesn't have to contain the actual PluginA. Instead a "missing tiddler" is opened, but the viewtemplate for it still does the job.

twMat commented 9 years ago

@erwanm - would it be of interest to feature such a thing on your TW Community Search? I think it makes sense to have access to the comments at the place where you also look for the plugins they concern - particularly if your site only presents meta-data and not the acutal plugins.

erwanm commented 9 years ago

@twMat I'm not sure I understand all the details of the design that you propose, but we'll probably have plenty of time to discuss it before I start the implementation ;) In particular in your example the tiddler AnnasComments contains comments on both PluginA and TiddlerB right? so I don't get how the aggregator knows the topic from the name of the current tiddler: isn't the name "AnnasComments"?

I would have considered an approach based on tags rather than tiddlers titles: for instance someone would tag "pluginA" any comment they have about "pluginA", then the aggregator would group together all the "pluginA" comments from everyone and put them in one "thread" tiddler (possibly using substories http://tiddlywiki.com/#Creating%20SubStories).

erwanm commented 9 years ago

about doing it in the CommunitySearch wiki it would certainly make sense but I want to have a fully working version first. additionally there is a potential problem with the size of the wiki, which is already a bit slow because it's growing big.

btw there might be also a technical limitation about the additional computational power required for the aggregator to do the job: it's probably manageable for a dozen users who comment occasionally, but the computer I currently use would quickly be overwhelmed with too many messages (I have no idea if the limit would be 100s, 1000s, more...)

twMat commented 9 years ago

In particular in your example the tiddler AnnasComments contains comments on both PluginA and TiddlerB right? so I don't get how the aggregator knows the topic from the name of the current tiddler: isn't the name "AnnasComments"?

No, that's the thing - AnnasComments is a dictionarytiddler and can have any titles as indexes.

The iframe calls for the asked-about tiddler (a permalink as source). This, not unlikely, results in a missing tiddler being called for.... which in a default TW would show the "This tiddler is missing, click to create it" which comes from $:/core/ui/ViewTemplate/body...

...but I have modified this tiddler into:

<div class="tc-tiddler-body">

<$list filter="[all[current]!has[plugin-type]!field:hide-body[yes]]">

<$list filter="[tag[metatid]getindex{!!title}]" emptyMessage="No comments yet.">
{{!!title}}
<hr>
</$list>

<$transclude/>

</$list>

</div>

Thus it filters out all tiddlers tagged metatid (these are datatiddlers) and uses the (likely missing) current title as the index. (If it's not missing, I guess something else has to be done.)

This is also one reason I feel it really would mesh well with the aggregator; the tiddlers referred to inte comments do not have to be installed. And really any tiddler can be commented on - it doesn't even have to exist anywhere so you could even use it as a communication system, e.g by calling for @erwan to find all comments (in the metatiddlers) that have this index.

twMat commented 9 years ago

[...] there is a potential problem with the size of the wiki, which is already a bit slow because it's growing big.

I was surprised to read this and it made me download the (full) aggregator (8MB) and I noted the statistics that the aggreagtor has;

4961 tiddlers 50 wikis 115 plugins

This makes me wonder if there is a lot of redundance?

For one thing, the obvious red light among the wikis is tw.com. Deducting the tw.com size of the core from it, I get 1,8 MB of content. tw.com is, however, more accessible than the tw-aggregator itself so one may reconsider including it. Just a thought.

btw there might be also a technical limitation about the additional computational power required for the aggregator to do the job: it's probably manageable for a dozen users who comment occasionally, but the computer I currently use would quickly be overwhelmed with too many messages (I have no idea if the limit would be 100s, 1000s, more...)

I guess t is good that you've separated out the tw-aggegator from the tw-community-search because if there are already now effects from size maybe I guess it's a matter of time before it'll have to be split into niche aggregators and hopefully others have the resources to use the engine. ..or things all into place naturally when the federation gets going...

erwanm commented 9 years ago

Hi Mat,

thanks for the explanations, I'll probably have more questions later but I first need to let it infuse in my brain a little while.

about excluding tw.com I'm starting to consider it indeed, that will probably be unavoidable sooner or later.

Yet another thing I'd like to do (someday...) is to do some experiments with various amount of data: number/size of tiddlers, links, tags, etc. in order to test the limits of a standalone TW. It's relatively easy to do this kind of tests automatically by generating a wiki from a script (with dummy content of course), and probably would be helpful for the community since I saw the question being asked a few times.

twMat commented 9 years ago

experiments with various amount of data:

@pmario and @jermolene and probably others have done some tests like this so probably a good idea to ask them first. Besides, I'm sure they'd be interested in any further discoveries you may make. And as brought up in #76 @danielo515 / TiddlyDrive mgiht have dealt with similar issues.

erwanm commented 9 years ago

@twMat: I thought about a couple of potential issues with your design:

What do you think?