Update lexers

[rnpnr]

This should be just about ready to merge now. One thing I'm still not sure about is the modifying of the default styles. I don't think it is necessary to bloat up the lexers table with all the possible style tags but there should still be some way of indicating to users how they can add styles for missing tags. Let me know if someone has a better idea than just copying from upstream and leaving them commented out.

As indicated by the commit authors this contains the work of a few different people whom I pass on my thanks!

[mcepl]

Very hesitant +1 from me. Shouldn’t we first make some vivisection like what I suggested in https://paste.sr.ht/~mcepl/ef4aba41df3c516bc6a96ed1ee9643fed78e01d0 (and https://github.com/orbitalquark/scintillua/issues/99#issuecomment-1732635230) so that at least we have something half-functional?

[rnpnr]

So right now any styles added with lex:add_style will not get added. This applies to the following (excluding lexer.lua):

# grep 'add_style' lua/lexers/* | cut -d ':' -f 1 | uniq
antlr.lua
clojure.lua
crystal.lua
gleam.lua
icon.lua
jq.lua
julia.lua
lexer.lua
mediawiki.lua
meson.lua
moonscript.lua
rest.lua
txt2tags.lua

Which out of ~150 lexers isn't bad. You can still get styling for all of these by adding extra entries to your theme. For example:

https://github.com/martanne/vis/blob/d37a20c11b9fd3e5e7fc3d0e7b30d7d78fef075a/lua/themes/dark-16.lua#L132-L144

So I don't really think this is half functional as you say.

[mcepl]

Yes, “half functional” is too harsh, but what I meant is that current state of that branch is a regression against the previous released version.

[TwoF1nger]

@rnpnr I think there's a redrawtime leftover in the man page.

[rnpnr]

@rnpnr I think there's a redrawtime leftover in the man page.

Good catch, I will fix it.

[rnpnr]

Do people think that the commented out examples should just be enabled by default? It would result in some small amount of extra runtime memory usage but should make it so this patchset doesn't remove the working existing styles.

[fischerling]

Do people think that the commented out examples should just be enabled by default? It would result in some small amount of extra runtime memory usage but should make it so this patchset doesn't remove the working existing styles.

I don't really care, because I use my own theme definitions. But I think that people would definitely expect proper syntax highlighting (highlighting all tokens) for all supported file types out of the box.

[mcepl]

But I think that people would definitely expect proper syntax highlighting (highlighting all tokens) for all supported file types out of the box.

That +1 was for this, not for the first sentence.

[rnpnr]

Sounds good to me. In that case here is an idea: combine light-16.lua and dark-16.lua into a single file and replace all fore:white,back:black etc. with empty declarations to just use the default terminal foreground and background. This is what I do in my term.lua theme so that I can have a light or dark background depending on the time of day. This should also address some of the complaints about the default style not matching the terminal.

As for the zenburn.lua and solarized.lua I don't know much about them. So I can either try to reuse styles, as in lexers.STYLE_H1 = lexers.STYLE_HEADING, or just leave it for someone else to fix.

[mcepl]

Sounds a like a sound plan ;).

[mcepl]

You made a huge amount of work with reviving those themes. Respect and thank you!

Applied to my devel branch and after five minutes testing it seems to work almost perfectly.

We should also collect what changes (if any) we want to push upstream and somehow resolve https://github.com/orbitalquark/scintillua/issues/99.

[rnpnr]

We should also collect what changes (if any) we want to push upstream

Actually everything here is upstream besides your change to search in the correct path.

and somehow resolve orbitalquark/scintillua#99.

Unfortunately I don't have any knowledge of reST so I don't know which add_styles() can be dropped and which need to stay. Some can probably be changed to the styles added for the other markup languages.

Sounds a like a sound plan ;).

I'm still working on this btw. There are some bugs in the logic for applying themes which is probably why no one else has attempted to merge the styles. Work in progress is in my selstyle branch. If I can figure it out I will apply it to master and then update this patch set accordingly.

[rnpnr]

I think I'm going to cherry-pick the following commits into master: lua: drop redrawtime option lua: filetype: use alt_name field to alias to other lexers These are strictly bugfixes and shouldn't be contentious.

[mcepl]

On Tue Dec 5, 2023 at 8:25 PM CET, Randy Palamar wrote:

I think I'm going to cherry-pick the following commits into master: lua: drop redrawtime option lua: filetype: use alt_name field to alias to other lexers These are strictly bugfixes and shouldn't be contentious.

Certainly, go for it!

[mcepl]

On Wed Mar 27, 2024 at 1:06 PM CET, Randy Palamar wrote:

Merged #1133 into master.

Thank you!

Matěj

P.S.: So, now only to merge tests, right? ;) -- http://matej.ceplovi.cz/blog/, @@.*** GPG Finger: 3C76 A027 CA45 AD70 98B5 BC1D 7920 5802 880B C9D8

If only there were evil people somewhere insidiously committing evil deeds, and it were necessary only to separate them from the rest of us and destroy them. But the line dividing good and evil cuts through the heart of every human being. And who is willing to destroy a piece of his own heart? -- Aleksandr Solzhenitsyn: The Gulag Archipelago