Multiline LaTeX

[sp1ritCS]

I'm unsure if I just don't know how (as it appears as https://github.com/blackhole89/notekit/blob/af313fbd3710ce6e2f0f51e770099b91de3d8ad7/sourceview/markdown.lang#L192 matches $$\n\LaTeX\n$$) or if this is a limitation with gtksourceview (no multiline regex)

It is very hard to know what you want to edit, if you have a huge LaTeX Table (See #47 ) in one line.

[blackhole89]

I believe you need to have separate regexen (with <start>...</start> and <end>...</end>) for the Gsv engine to match something across multiple lines. We could change this definition to work like that (maybe something like

    <context id="latex" class="prox latex invisible" style-ref="list-marker">
      <start>(&#xFFFC;?)\$\$</start>
      <end>\$\$</end>
      <include>
        <context sub-pattern="1" where="start" class-disabled="invisible"/>
      </include>
    </context>
    <context id="latex1" class="prox latex invisible" style-ref="list-marker">
      <start>(&#xFFFC;?)(?&lt;!\$)\$</start>
      <end>\$</end>
      <include>
        <context sub-pattern="1" where="start" class-disabled="invisible"/>
      </include>
    </context>

would work, though I just wrote it blindly without testing; and you should for sure add some additional modifiers and rules to make sure that it's possible to write dollar signs (for non-LaTeX purposes) at all, say by rendering \\\$ as \$ with the backslash invisible)

[sp1ritCS]

If I try to enter any LaTeX it segfaults, and $\LaTeX$ is not working anymore.

Working with the resources in sourceview/ is a big pain anyways (I've attempted to add a widget myself, but I am unable to get it working). Is is possible to add a gsv match definition programmatically (this could clean up the current widget code a lot)? Or are there at least Docs on how these files work?

[blackhole89]

it segfaults

You mean after doing the replacement I suggested? I assume the two-$ span is the one that segfaults? (I named the single-$ span a different thing, so you need to include that context explicitly further down in the file.)

There's some possibility that class-disabled (in the sub-context) and class (in the top level context) don't interact like I expected. Either way, the objective is that the $($)...($)$ span gets assigned class invisible, but the optional 0xFFFC (placeholder char for anchored widget) preceding it does not.

programmatically

No, gtksourceview unfortunately exposes almost nothing useful about its syntax highlighting via the API. (See the trickery to get highlighting for different languages...)

docs

Yeah, here.

I've attempted to add a widget myself

What kind? I might be able to figure it out quickly if there's anything obvious with it.

[sp1ritCS]

No, gtksourceview unfortunately exposes almost nothing useful about its syntax highlighting via the API. (See the trickery to get highlighting for different languages...)

Ever considered askig if upstream could implement it?

What kind?

My Idea was to create simple a FileMime Wiget (= big button to open a file) that just renders a file icon based on the mimetype of the file and the file name besides that. I first wanted a function that would render something on the correct input. (&(~/path/to/file.pdf)) but as said - that didn't work. See https://github.com/sp1ritCS/notekit/commit/d1c03fa85fc1dbf1f1dfd675a06785da60667bff

Maybe adding a HACKING.md file that describes how to add a widget would be a good idea. tho the widget code should be cleaned up.

[blackhole89]

Ever considered askig if upstream could implement it?

I've been in touch on the devs' IRC channel, and they didn't seem particularly interested even in a smaller change in the same vein that would lighten our burden (gtksourceview technically associates a string "name" with every tag, but the tags the syntax highlighter generates for the different highlight styles have empty string names instead of being named after the style class in the language spec. It's therefore more or less impossible to figure out what the syntax highlighter has parsed a particular substring to be without modifying the language spec)

FileMime widget

The immediate thing I see that's missing is that the new widget class you're trying to introduce will never be generated, for reasons related to the above. The only syntax highlighter judgement we can reliably read out via Gsv's API is the "class" entry of the language spec (via sbuffer->iter_has_context_class). That's why the latex context has class latex (<context id="latex" class="prox latex" style-ref="list-marker">). The functions CNotebook::on_leave_region and CNotebook::on_enter_region are called as the user's cursor moves out of/into minimal regions of uniform syntax highlighting. Every time, we check whether the region left/entered has one of the classes listed in renderable_tags (annoyingly, there are currently two copies of this; I should fix that), and if it does, call UnrenderWidgets and/or RenderToWidget with wtype set to that class. Therefore, you should at the minimum

• add "mimewgt" to the "mimewgt" context's list of classes

• add "mimewgt" to both copies of renderable_tags.

HACKING.md sounds like a good idea, but I'm not sure how soon I'll be able to spare time for that. There's a lot of little and big things about the code that need to be tightened anyway before I can feel comfortable outright encouraging people to start contributing big patches (not that I'd reject such patches off hand, but I'd need to review them very carefully because the logic of how everything fits together is a mess right now and even I am barely managing to retain a correct mental model...).

[blackhole89]

You also do need to add (\?) to your mimewgt context's regex (and make sure the sub-pattern is numbered correctly to make the part that you care about go invisible). The reason for this is that 0xFFFC is the placeholder character that gtktextview/sourceview inject when you anchor a child widget. The proximity widgets code is written under the assumption that this character (and hence the widget) are also part of the span that has prox and the appropriate entry of renderable_tags as classes.

[sp1ritCS]

Yeah, I managed to get it working :D. The only thing that i'm missing is some kind of shadow around that thing, a click signal and to actually render the icon. But I don't know how to do either of these things 😅.

But I'd still like to know your opinion. https://github.com/sp1ritCS/notekit/tree/mime

Edit: apparently, ubuntu g++ doesn't work with std=c++17 according to the ci.

[sp1ritCS]

So, I've done some more testing, and I've found something that works.

<context id="latex" class="prox latex invisible" style-ref="list-marker">
    <start>(&#xFFFC;?)(?&lt;![\$\\])\$(?!\$)</start>
    <end>(?&lt;!\$)\$</end>
    <include>
        <context sub-pattern="1" where="start" class-disabled="invisible"/>
    </include>
</context>
<context id="latexml" class="prox latex invisible" style-ref="list-marker">
    <start>(&#xFFFC;?)\$\$(?![\$\\])</start>
    <end>(?&lt;!\$)\$\$</end>
    <include>
        <context sub-pattern="1" where="start" class-disabled="invisible"/>
    </include>
</context>

ML LaTeX is centering itselfs tho. I'm guessing cLaTeXMath is doing this.

Another issue I have is, that im unable to match (\\)\$ to hide the \ from \$.

[blackhole89]

The centering happens as a consequence of being multiline? I would've thought that it depends on $$ vs. $. (I'll look into it shortly.)

[sp1ritCS]

The centering happens as a consequence of being multiline? I would've thought that it depends on $$ vs. $. (I'll look into it shortly.)

Never-mind what I said, that was my stupidity. Yes it works with single $ correctly.

[sp1ritCS]

I just committed a battery of fixes to proximity widget handling, which hopefully should get rid of most cases of stray [X] boxes being displayed. Verdict is still out on whether I need to add a manual filter to remove 0xFFFCs that somehow made it past the logic on load/save. It might be worth trying to see if the multiline approach (with the spec of commit d961dadbad75301f5879a3bbdf2b514ba5ce55d4) fails more gracefully now.

Originally posted by @blackhole89 in https://github.com/blackhole89/notekit/issues/77#issuecomment-750712029

I actually experienced a lot of issues testing it; the one in the PR didn't highlight several files in my own tree of notes correctly

I've tested it against my own notes, and at least with https://github.com/blackhole89/notekit/commit/c92dab1c5722203b122d4d169fd0d3273352bf36 I didn't see any issues with LaTeX I've written before, but I guess the stuff that I've written wasn't very complex (see attached image as an example) so I don't know If you still encounter issues with it.

I'm still experiencing the weird behavior of Gsv, where it considers EOF the same as any <end> match. The best way to fix this would be to prevent Gsv from doing this. Otherwise improving prox and invisible would make the behavior of Gsv less weird.

---

Anyways, I wish you a nice ✨ Christmas 🎄, even tho we are currently in a global pandemic :/.

[blackhole89]

I've tested it against my own notes, and at least with c92dab1 I didn't see any issues with LaTeX I've written before, but I guess the stuff that I've written wasn't very complex (see attached image as an example) so I don't know If you still encounter issues with it.

I had some issues with multiple double-$ LaTeX spans in paragraphs of text not getting detected. Also, the behaviour of everything when appending at the end of the buffer was manifestly weird: it seemed to not highlight a newly added line as part of the LaTeX region at all at first, and then created a separate one (so that in input that looked like $$\nL1\nL2, there were two LaTeX blocks rendered, one containing L1 and one containing L2) whose proximity widget didn't work correctly.

The best way to fix this would be to prevent Gsv from doing this.

Yeah, I don't know how feasible that is without maintaining a full fork of it (which would be too much for me). Maybe there's some function I can patch in the gobject vtable as I did for copypaste and moving by word...

Anyways, I wish you a nice Christmas, even tho we are currently in a global pandemic :/.

The same to you! Beware of superspreaders trying to come in through the chimney; full beards are not an effective substitute for facemasks :)