Feature request: improve coloring

[PierreMarchand20]

I was just looking for a file format to dump data from numerical experiences (I am a computer scientist) and I recently discovered eno ! I really like the syntax and I am impressed by all the API and plugins you make. Congrats !

I will use it to dump data from my c++ code and load it with your python API to visualise data. Before I used my own parser to load my results from .txt files with python to then put everything in a SQL database and then I sent requests to my SQL database to plot what I want, but it was pretty heavy compared to directly dump in a "kind of" eno database and using your API ^^

About the vscode plugin, may be it would be possible to improve the contrast ? It is pretty hard to distinguish between sections, fieldsets and their elements. (I am using Solarized dark theme) It seems that in Atom for exemple, it is more contrasted (from your example)

[simonrepp]

Hey @PierreMarchand20,

Thanks for the kind words, and great to hear about your usecase!

I'll gladly look into this, I'm right now going through all editor plugins to reiterate on the grammars and scope definitions and how they interact with the color schemes, I've already done that for atom, I'm right now working on an upgrade for sublime (probably to be released today) and seeing your demand I'll look right into improving things in VSCode following that, thanks for letting me know! :)

Will probably have something ready in the course of the coming week, I'll keep you posted.

Cheers!

[simonrepp]

Just to give an update: I've meanwhile been working on porting the grammar improvements from atom/sublime - although the scopes are semantically more apt I found them to be even less well supported by vscode's default color themes (to the point that there is almost no syntax highlighting left anymore, ouch. ;)).

I've researched quite a bit how I can customize the default themes to ensure really good readability and differentiation of elements (this worked really well for atom/sublime, even in a generic/semi-generic way - see e.g. https://resources.eno-lang.org/plugins/sublime.png), however currently it seems the best, or even only (?) option will be to hand-replicate and ship all default themes in a hand-customized eno variant.

I'll gladly go through this effort (which is not too much work anyway), however I'm not entirely happy about it from the user-perspective, because it means switching the global vscode theme by hand each time one wants to edit an eno file with the custom eno theme, as this is apparently not configurable in vscode (https://github.com/Microsoft/vscode/issues/20652), although there does exist a third-party extension that switches theme programmatically when changing file type. (https://marketplace.visualstudio.com/items?itemName=jsaulou.theme-by-language)

If you have any thoughts about this, let me know, either way I'll follow up on preparing the customized default themes for eno for now, as this seems to be the best bet. :)

[PierreMarchand20]

Well I am really just a user of vscode and I do not really know how syntax coloring works. But I did not understand why you cannot use the generic way to define Eno-lang ? It is what is usually done for example:

• https://github.com/twxs/vs.language.cmake/blob/master/syntaxes/CMake.tmLanguage for Cmake
• https://github.com/James-Yu/LaTeX-Workshop/blob/master/syntax/LaTeX.plist for LaTeX

They never define by hand a syntax colorisation for each theme, it would be quite a hassle oO Did you look to https://code.visualstudio.com/api/language-extensions/syntax-highlight-guide ? (I guess you do)

[PierreMarchand20]

Would it be possible to replicate what is done with json files for example ? They are quite well colorised.

[PierreMarchand20]

Or toml files are also well colorised:

[simonrepp]

Thanks for the input :) The grammar for eno is of course "generically" specified like any other, there isn't any other way to do it in fact (which makes sense because otherwise it would be hassle as you correctly mention ;)).

The problem is that the baseline conventions for "generic" syntax scopes used across most editors (roughly what's specified in 12.4 Naming Conventions in https://macromates.com/manual/en/language_grammars) define a set of classifiers (like comment.line or variable.other.name) that aren't quite as generic as I'd need them for eno. They do have (out of understandable necessity and history) specific ideas about what kind of syntax scopes there are and those don't go along with eno's (and in fact also many other languages') structure very well.

What this means in practice, for instance: Many languages grammars that don't even have the concept of a tag use the generic entity.name.tag classification as the syntax scope for keys, which is semantically wrong (and therefore detrimental to developing additional functionality for that language), it's only done because XML/HTML is so important to most color theme editors that you can expect a distinct and sensible color to be defined for the generic entity.name.tag scope, so the concept of generic scopes is actually subverted and inaccurate semantic scope classifications are used only to satisfy good coloring, which is ultimately what I want to avoid.

This just to provide some background, but don't worry about, it I'l figure things out, I was just interested if maybe you had some in-depth input on that as well, which I wouldn't want to have missed out on in that case. :) Thanks again, I'll keep you updated!

[PierreMarchand20]

Ok, so if I understand it right, the goal is to find the right concepts to use with the eno scopes to get a well-contrasted coloring. Even if they have nothing to do with each other, right ?

Then, we can look at what they have done in https://marketplace.visualstudio.com/items?itemName=bungcip.better-toml to see what concepts they used to get their syntax highlighting

[simonrepp]

Yes that's the obvious solution if one decides to sacrifice semantics and theory for practical results :) - which I will probably do here, given that I see too many problems with vscode's default color schemes and scope architecture decisions right now to solve this in a proper way (though I might follow up on this with the vscode team if time permits).

So, as soon as I have stomached this sacrifice of thorough enginneering in the interest of pragmatism :) I'll probably be releasing the new version which works flawlessly with all default themes of vscode, yielding eg. for your (solarized dark) theme:

[PierreMarchand20]

But for example in https://github.com/bungcip/better-toml/blob/master/syntaxes/TOML.YAML-tmLanguage, the name they used seemed coherent with the scopes no ? I may have misunderstood something then xD

[simonrepp]

@PierreMarchand20 The scopes they use are coherent to some degree yes, it's also less of a problem for toml, yaml, etc. because they have known types for all values (and these are well supported by established scope schemes) and therefore they get a good differentiation between values and other stuff like punctuation "for free". But technically even the scopes used in toml and yaml don't really conform to what's theoretically laid out in https://macromates.com/manual/en/language_grammars in some core aspects and when looking at the vscode default color theme definitions it quickly becomes obvious that a lot of the (not so obviously identifyable) scopes are picked solely on coloring pragmatism, not out of any proper architectural concern ... but well, maybe I'm just too nit-picky. :)

[PierreMarchand20]

Better be too picky than not enough ahah anyway, the new colouring seems better ! Thank you for your work and your answers 👍 you can always discuss this issue with the vscode team, especially if it is better in sublime/atom xD

[simonrepp]

Hooray I just published 0.2.0, thanks for your feedback and help! :)

[PierreMarchand20]

This version is quite better, it would be perfect with a different colour for the markdown-style sections and key/value ? At the end, I noticed that json/toml colouring are usually quite consistent with each other:

• json
• toml

So may be it would be nice to have the same kind of colouring for the values ? Anyway, they are just some ideas, it is already way better and very usable.

[simonrepp]

Regarding sections - in this batch of plugin releases I've opted to visually distinguish the section operator (#) instead of the name, I might very well revisit and change this again (especially given your feedback) but I want to leave it for a while first just to experience this hands-on in production usage and see how it works/doesn't work in certain scenarios before testing another variant. In vscode there's the additional issue here that the operator token is very lightly differentiated by the themes (only italic/slight background - compare with https://resources.eno-lang.org/plugins/atom.png e.g., I was generally struggling with this in vscode a bit). So bottom line, expect an update in the next round of plugin updates, meanwhile I really need to spend some time in other areas again - especially the rust library, some people are eagerly waiting. ;)

Values I would generally like to keep uncolored (which is by itself also a differentiation from the other colored tokens) - they are after all by language definition of unknown type and therefore a neutral text color seems most fitting to me. Even though on a technical level they are seen as strings by the eno libraries, that's not what they are to the application or the user, so coloring them as strings would seem misleading to me as well. :) Maybe an option for some form of speculative type hinting or the possibility to supply a user schema in the plugin options at some future point!

Thanks for the feedback again, like the screenshots especially, really helps the communication process!