Open linonetwo opened 3 months ago
Add
rule = exports.name
field to ast node
See https://github.com/Jermolene/TiddlyWiki5/commit/e4c682d04b5627b148c86ad9ae5c7ce97b84a5f2 where this has been implemented.
Part of the problem that still remains is that the parse tree does not have enough information about whitespace, comments and pragma so recreating the original wikitext still isn't possible.
Need to get original wikitext from AST, when writing WYSIWYG editor.
That's not really possible at the moment. While we did add a lot of new start
and end
positions to the AST lately, the <!-- comments -->
are not there. So at the moment it's kind of impossible to create lossless wikitext from the TW AST.
I also think there is not enough information about function, procedure calls in the AST. eg: <<test>>
and <$macrocall $name=test
may not be distinguishable. -- But I'm not really sure about that one. -- But "The devil is in the details" atm
There is an unused method getParseTreeText
, it used to work for text-slicer plugin, but it's useless now.
Old usage (don't exist anymore): https://github.com/eyecatchup/TiddlyWiki5/blob/72ed4b26731dd3bbac948af6965b050f33ed1c1c/plugins/tiddlywiki/text-slicer/slicer.js#L79C23-L79C39
Usage: $tw.utils.getParseTreeText($tw.wiki.parseTiddler('HelloThere').tree)
I can gradually add wiki rule support to it. Not need to be lostless, it only need to handle 80% of everyday usecase. People who use WYSIWYG editor won't write comment or procedure, support for then can be delayed.
See e4c682d where this has been implemented.
I see. This is not very obvious. I was only reading files in core\modules\parsers\wikiparser\rules
Thanks @linonetwo I hadn't noticed that getParseTreeText()
is no longer used.
I can gradually add wiki rule support to it. Not need to be lostless, it only need to handle 80% of everyday usecase. People who use WYSIWYG editor won't write comment or procedure, support for then can be delayed.
getParseTreeText()
is intended to just extract the value of text nodes. I think you're proposing a function to convert a parse tree back to wiki text? I would think that that should be a separate function.
I take your point that it wouldn't be necessary to handle all wikitext constructions. It would be interesting to see how complex the implementation would be, but if it's not excessive then I would be happy to see that go into the core.
I take your point that it wouldn't be necessary to handle all wikitext constructions. It would be interesting to see how complex the implementation would be, but if it's not excessive then I would be happy to see that go into the core.
I think @flibbles knows best, what's needed for a "rewrite" of tiddler content. IMO the relink-plugin does exactly that.
@linonetwo -- I would have a closer look at the relink-AST (if there is one) and I would try to get or reuse the API from the relink plugin. Most users who want to use WYSIWYG will need the relink-plugin anyway.
Yes, this is interesting, how relink works when there was no start/end pos in the AST?
After reading for a while, I find it is simply use regex to replace old value, not using AST
... I find it is simply use regex to replace old value, not using AST
That may be true. @flibbles -- should know exactly.
Yes, this is interesting, how relink works when there was no start/end pos in the AST?
After reading for a while, I find it is simply use regex to replace old value, not using AST
You're looking at the method used to handle substitution attributes. Relink doesn't just use regexp. It also doesn't utilize the existing AST since it's incomplete for reproducing the original text. It modifies the Wikitext parser and has it "parse" text, with each wikitext rule module getting new report
and relink
methods. the "AST" it produces is the modified text.
@Jermolene Need to add a new parse rule type
exports.name = "commentblock";
exports.types = {void:true, block:true, pragma:true};
When adding void:true
, the AST node will not be rendered.
Currently without this, commentblock / parsermode will render to some text, cause style error, because they break the stylesheet tiddlers.
@Jermolene Need to add a new parse rule type
Yes, that sounds like a reasonable solution.
If you are interested, feel free to PR more test to #8258 . I've already pass all basic tests.
In some case, original whitespace is not preserved, may add/remove some of them to make result beautiful, like prettier. And user don't care about this, because in WYSIWYG editor, user don't care underlying text, they just want a text file so it feels safe.
Is your feature request related to a problem? Please describe.
Need to get original wikitext from AST, when writing WYSIWYG editor.
Describe the solution you'd like
Add
rule = exports.name
field to ast node, and addexports.toString
to wikiparsers.In
toString
, try callhandleChildrenToString
. (The problem is where to find this method, we can't access$tw
in the wikiparser. Can we put it onparser
passed fromexports.init = function(parser) {
?Describe alternatives you've considered
Now it is maintained in https://github.com/tiddly-gittly/wikiast, but is quickly outdated.
Additional context
I remember some other issue we talk about adding a
rule
field to the ast.