It seems relink-plugin and my uni-link plugin are not compatible

[pmario]

I'm creating a "Stories" sidebar atm. A typical story configuration looks like this:

title: Story/config type: text/plain .... needed because of readability

[subfilter{tiddler-title}]
[[a a]]

Neither tiddler-title nor a a are detected as a reference. ...EVEN if I swith type to "text/vnd.tiddlywiki" ...

How do you calculate [relink:reference ???

[pmario]

If I use type: text/vnd.Tiddlywiki, ... The [[a a]] link is detected, but [subfilter{xxx}] is not

[pmario]

I did also deactivate uni-link plugin

[flibbles]

Currently, Relink only bothers with text/vnd.Tiddlywiki tiddlers. I'm not familiar with your uni-link plugin, but it sounds like its using the Story/config tiddler as a holding place for a filter, and that what you want is for Relink to process its text in that way, instead of as wikitext. It's updating the [[a a]] because it thinks its a wikilink, not a filter title operator.

I'm not sure the best way to get Relink to identify cases like this. There'd need to be some way to tell Relink that certain tiddlers must be processed (regardless of tiddler type) as filters and not text, which I guess would involve another whitelist-like configuration page.

And the bad part about that is that such a configuration page would be clutter for 99% of users. I don't know of any other pages apart from $:/DefaultTiddlers in core where that would be useful.

[pmario]

And the bad part about that is that such a configuration page would be clutter for 99% of users. I don't know of any other pages apart from $:/DefaultTiddlers in core where that would be useful.

You are right $:/DefaultTiddlers is a "story tiddler".

90% of users can use "single links" as [[a a]], and waste time with maintenence. ... Experienced users use something like: [list[$:/StoryList]]

That's 1 reason, why I did choose the format as it is. So [subfilter{story-config}] can be inserted into the default tiddler and it works out of the box.

[pmario]

I'm not sure the best way to get Relink to identify cases like this. There'd need to be some way to tell Relink that certain tiddlers must be processed (regardless of tiddler type) as filters and not text, which I guess would involve another whitelist-like configuration page.

I'm OK if relink plugin doesn't detect text/plain. I can implement a "find / replace" mechanism on my own. BUT we would need to agree on a mechanism.

It seems you overwrite the core EditTemplate

[flibbles]

Yes, that is something I felt very icky doing. But there was no other way. The title edit page is large and monolithic, and I had to make just a few changes to it.

[pmario]

Currently, Relink only bothers with text/vnd.Tiddlywiki tiddlers. I'm not familiar with your uni-link plugin,

uni-link replaces a link like this [[a a]] with a macrocall in the AST. That seems to be the reason, why you can't detect it.

[pmario]

I'm not sure yet, how you detect elements that can be replaced. ... Can you point me to the "entry point" of the replace mechanism in your code. ...

[flibbles]

I'm unfamiliar with this uni-link, and I can't seem to find it on github. Is this something you've got for yourself?

[pmario]

https://wikilabs.github.io/#uni-link:uni-link%20GettingStarted

[pmario]

The main feature is "Alias handling" using the aliases field. Aliases are defined once and never changed again.

[flibbles]

In bulkops I replace the default implementation of relinkTiddler. This, in turn, calls getFreshRelinkableTiddlers which does the actual replacing. Its in that function that I filter out non-tiddlytext tiddlers.

[pmario]

It seems your https://github.com/flibbles/tw5-relink/blob/master/plugins/relink/js/relinkoperations/text/wikitext/prettylink.js gets confused if my uni-link plugin is active.

I don't want you to change anything. ... I actually think "relink" or big parts of it should be part of the core. OR the core should provide an entry point to add plugins like these.

I need to study the code a bit more.

[flibbles]

You're trying to get Relink to process a tiddler's text as a filter, right? If that's so, prettyLink.js isn't actually where you want to be looking.

[flibbles]

Unless what you're trying to do get your aliases to work nicely with Relink. In which case, you're probably close to the right place.

[pmario]

You're trying to get Relink to process a tiddler's text as a filter, right?

That's right.

[pmario]

..If that's so, prettyLink.js isn't actually where you want to be looking.

prettyLink is an issue, for both of us, because if a user uses both plugins, they will have a problem. So we should try to get that fixed, somehow too.

I'll need to know, how I can let your code know, how to identify an "uni-link" link.

[pmario]

You're trying to get Relink to process a tiddler's text as a filter, right? ..

As I wrote, I'm using "text/plain" because of readability. The advantage for the "Story stuff" is, that those config tiddlers can contain tiddler titles only. (I think)

The code to extract the the story is:

var storyFilter = $tw.wiki.getTextReference(storyTitle, "GettingStarted"),
    storyList = $tw.wiki.filterTiddlers(storyFilter),

[flibbles]

The absolute best way would be if your aliasLinks had their own wikirule. Just like how prettyLinks. That way, you could write a Relink rule which would attach to that one. That relink rule would be ignored if Relink weren't installed. But would kick in just fine it it was.

However, your aliasLinks are being picked up by the prettyLink rule. It might be possible for you tamper with prettyLink so it would stop, and then your own aliasLink rule would only apply...?

[flibbles]

For whatever frustrating reason, you can't just inject a new regexp for prettyLink to use, because it sets the regexp for the wikirule per instance, even though it's always the same. Which means this route would have to involve completely overwriting the prettyLink rule.

[pmario]

The uni-link plugin actually overwrites the core prettylink. I wasn't happy about that, but imo there is no other way.

[flibbles]

Then I might recommend that you alter the prettylink rule only to update its regexp to:

this.matchRegExp = /\[\[(.*?)(?:\|([^?].*?))?\]\]/mg;

So it ignores aliasLinks. Then make your own aliasLink rule. Then you can include a relinkwikitextrule for aliasLink.

I can make sure relinkwikitextrule can exist benignly without a corresponding aliasLink. (although now that I think about it, that isn't strictly necessary. I'll check anyway.)

[pmario]

hmmm, Find and replace would be so simple, if .parseText("Hello <!--test-->[[World]]").stringify() would return Hello <!--test-->[[World]]

[pmario]

So it ignores aliasLinks. Then make your own aliasLink rule. Then you can include a relinkwikitextrule for aliasLink.

That's an idea to explore.

[flibbles]

Wait a mo, hold on. Your aliasLinks don't actually use tiddler titles at all. So you shouldn't have to make a corresponding relinkwikitextrule for it. All you'd need to do is change the prettyLink regext and put your aliasLink stuff in its own rule.

Right? I think so.

I pushed a test to Relink ensuring relinkwikitextrules can exist without corresponding wikirules, even though it's completely irrelevant now.

[flibbles]

I think I may have a solution for processing tiddlers as filters. Give me some time.

[pmario]

Wait a mo, hold on. Your aliasLinks don't actually use tiddler titles at all.

You are right. Alias-links don't use titles. ... but the prettylink (caption, subtitle) mods do :/ I think, I shouldn't have implemented those.

[pmario]

The "story issue" is discussed here: https://github.com/Jermolene/TiddlyWiki5/issues/4319 ... Just if you are interested.

[flibbles]

Hmmmmmm.

I'm sitting here trying to figure out how best to manage $:/DefaultTiddlers and other filter tiddlers, when a crazy idea occurs to me.

What if there was a separate type for filter tiddlers? Instead of text/vnd.tiddlywiki or text/plain, core Tiddlywiki could have text/filter which could maybe even render a little differently, like maybe show the results in a split panel or something.

The advantage to that for me is that Relink has a separate relinktextoperator for each tiddler type. Currently it only has one of text/vnd.tiddlywiki, but I always figured I might make ones for the dictionary tiddlers and JSON tiddlers. But this would provide a perfect way to handle tiddlers like $:/DefaultTiddler and your plugin's list filters.

Is this something that Tiddlywiki core could benefit from?

[pmario]

text/plain is an official mime type. see: https://www.iana.org/assignments/media-types/media-types.xhtml

They are standardized. We can't create a text/filter It would be "super high level" and won't be approved.

[pmario]

like so:

[flibbles]

Boooooooooooo okay.

Shows what I know. All right. I'll look into a way to create different handlers based on... filters or something. I guess.

Use this handler for all text/vnd.tiddlywiki types, except for a select few, which get their own handler somehow.

[pmario]

I wanted to release a "proof of concept", but as you can see in the screen-shot, it would be nice to get the subfilter rewritten if the "Story/Tower.... " name changes.

I did install relink and saw, that there are some problems. ... Without proper renaming of stories, I think the concept will be screwed, because I wan to use namespaces. NS gives a much nicer structure, without polluting tags

[flibbles]

Yeah, I guess it's just a matter of how those Story tiddlers are going to be differentiated from other tiddlers. I'd have thought a type, or something like a type would be nice. Whatever it is, I'd like if Relink wouldn't need special handling baked into it, but rather special handling as a separate module.

[flibbles]

I guess what I'm saying is, right now, Relink neatly handles all text/vnd.tiddlywiki one way, and it could easily handle application/javascript another.

If $:/DefaultTiddlers and other arbitrary tiddlers are to be treated as tiddlers, then I need a handler which deals with all [[$:/DefaultTiddlers]] [tag[$:/tags/flibbles/relink/filtertiddler]] and then my text/vnd.tiddlywiki module needs to know to ignore those types.

It's messy. It could be cleaner if module-types could be sorted with list-after and list-before the same way tagged tiddlers can, but they can't. So this feature requires some hard baking.

[flibbles]

It just seems like it would be super cool for everyone if there was a core way to clarify that a tiddler contains a filter, or a story, and that it is not (and should not be treated as) wikitext.

[flibbles]

I researched media types a bit more. Doesn't look like tiddlywiki ever bothered registering the types it does use. text/vnd.tiddlywiki is not on the list you showed me. application/x-tiddler-dictionary is not registered at all, which is what the 'x' means.

Tiddlers with filters as bodies are not wikitext, and should not be rendered as wikitext. They're filters. Do you really think it would be outlandish to introduce a text/x-tiddler-filter type?

If not this, then I guess I'd have to define a tag, which only your plugin would use. And then also manually handle $:/DefaultTiddlers.

[flibbles]

Actually, now looking at core implementation and other plugins like TiddlyMap, I think the most appropriate way may actually be if your filters just go inside a filter field. Seems like that's what everyone else does.

I could add that as a default field that gets treated as a filter. No problem. It's used a lot actually.

[pmario]

It's great, to have someone to discuss stuff like this. Reading someone else's POV helps to make progress.

I think, I can use the "filter" field. ...

[flibbles]

I'm glad you like it. Here's some more thought process for you.

I've got a release ready where the filter field is tracked, though I'm hesitating to add it. It's not a documented field, even though some core shadow tiddlers use it. But on the other hand, who would ever not want this field relinked as a filter? At least two other plugins that I know of use filter field as a filter.

I can push this release, or you can include a tiddler like this in your project:

title: $:/config/flibbles/relink/fields/filter

filter

That's the intended way for plugins to interact with tw-relink. Relink will even make a nice section for your plugin in its configuration page.

Hmm. Can't decide. You know Tiddly design better. Your thoughts?

[pmario]

@flibbles ... I did single step through your plugin and I found out if I use

exports.name = "unilink";    

in ..\relink\js\relinkoperations\text\wikitext\prettylink.js

It works perfectly fine with my uni-link plugin. ... I'm not 100% sure, but it seems you add a "relink" function to every wikitext rule. ... right?

So if I add your prittylink "relink" rule to my uni-link plugin too, I'll be good to go. ... so the following code seems to work nicely.

/*\
title: $:/plugins/wikilabs/uni-link/relinkoperations/text/wikitext/unilink.js
type: application/javascript
module-type: relinkwikitextrule

Handles replacement in wiki text inline rules, like,

[[Introduction]]

[[link description|TiddlerTitle]]

\*/

var prettylink = require('$:/plugins/flibbles/relink/js/relinkoperations/text/wikitext/prettylink.js');

exports.name = "unilink";
exports.relink = prettylink.relink;

[pmario]

@flibbles

Hmm. Can't decide. You know Tiddly design better. Your thoughts?

According to the "filter" tracking. .. I'd activate it. "filter" is a "de-facto" standard field since it is used in the "Advanced Search : Filter" tab dropdown. So imo it's safe to track it.

[flibbles]

If that solution is working for you, that's fantastic. I'll go ahead and close this issue out.

I'll add 'filter' as a core field to track eventually. Right now I am refactoring some code, but it should have no impact on your plugin.

[pmario]

yea, I did make some more tests and it should work now.

What do you think, it would need to include your relink code into the core?

[flibbles]

It would take some work. I imagine everyone who's thought about probably thinks the way to do it is to parse into a parse tree, make changes, and then rebuild the text from the tree, but I don't do that.

I do it the hard way.

I parse the text as is. That's because things like:

<<macro arg1 arg2>>

and

<<macro
   arg1
   arg2
>>

Are the same to the parse tree, and it's important to preserve the user's styling. That, however, meants I basically had to write my own wikiparser which cannibalizes parts of the existing one.

To properly make Relinking a part of core, the wikiparser would best be broken down into two parts: the part that parses the text, and the derived part which makes the tree. That way, you could have a second derived part that does relinking instead of making trees.

I completely 100% understand why nobody has made relinking before me. The amount of work was phenomenal once I really dug into it.

[flibbles]

Oh, make sure with your plugin that it doesn't fail if Relink isn't present. Looks like it would if you're using exactly what you sent me.

[pmario]

Oh, make sure with your plugin that it doesn't fail if Relink isn't present. Looks like it would if you're using exactly what you sent me.

:) Yea I did think the same way, but it's no problem, because of the module-type: relinkwikitextrule, which is only initialized if your plugin is installed.

[pmario]

Are the same to the parse tree, and it's important to preserve the user's styling. That, however, meants I basically had to write my own wikiparser which cannibalizes parts of the existing one.

Yea, that's the problem with the existing AST. It doesn't preserve whitespace. If it would, it would be relatively simple, to write a wikitext to markdown converter and vice versa.

[pmario]

I completely 100% understand why nobody has made relinking before me. The amount of work was phenomenal once I really dug into it.

That's why your plugin is "phenomenal" ;) ... You could have stopped with a simpler implementation, that could only do fields and "prettylinks" but you did go on with makro parameters and filters. ...

AND you made it configurable, which imo doubled the amount of initial work, but made expanding easier. ...

I'm considering, if I add "relink config tiddlers" to my plugins. So they "just work" if the plugin is installed.

Anyway. Nice Work!

[flibbles]

Yea, that's the problem with the existing AST. It doesn't preserve whitespace. If it would, it would be relatively simple, to write a wikitext to markdown converter and vice versa.

It would have been SO easy!

I'm considering, if I add "relink config tiddlers" to my plugins. So they "just work" if the plugin is installed.

Go right ahead. I'm keeping that part of the API the same for that reason.

That's why your plugin is "phenomenal" ;) ... You could have stopped with a simpler implementation, that could only do fields and "prettylinks" but you did go on with makro parameters and filters.

Thank you! I actually did stop with a simpler implementation for years. It's only the frustration of relinking my 4000+ tiddler project which pushed me to make an all-inclusive solution.