Use new API in ST4 to define mappings

[rchl]

The list defined at https://github.com/facelessuser/sublime-markdown-popups/blob/master/st3/mdpopups/st_mapping.py could be at least partially populated using the new sublime.list_syntaxes() API.

Since syntaxes have base scopes like sources.js, source.css, etc., the second component of that string could be used as a code block "tag".

Some syntaxes have more complex base scopes like text.html.markdown, text.html.basic so for those that would not help.

The advantage would be that if someone is using a non-standard scope for some language, it would still likely work as base scope would be the same as of built-in syntax.

And of course, it would also automatically support all unknown syntaxes that happen to have a base scope that matches the code block tag.

Now that I think of it, it does sound a bit messy to incorporate that API but maybe still worth to investigate.

[facelessuser]

Yeah, I'm not sure at this point how to use this in the context of what we want.

[facelessuser]

Okay, I think I better understand, they do provide a short name of sorts, but it doesn't always conform to the fence language rules. For instance, names shouldn't have spaces, but there are plenty in the syntax list that do. The next issue is the prioritization of the syntaxes. It's hard to tell what's default, so I guess you'd just go alphabetically by package names, and let user override all...

You'd have to convert bad names to good names Some bad name --> some-bad-name or something like that (not sure if hyphens are allowed or not...I think they are?). Anyways, maybe it is something to play with. I won't make promises as I imagine this may break plenty of existing usage. But I'll at least investigate.

@gir-bot lgtm @gir-bot add T: feature, P: maybe

[facelessuser]

Hmm...just thinking about this. Currently, we have a number of languages that use the same aliases, and it goes down the list until it finds one on the user's system that is enabled. If we auto-populate with the syntax_list we risk breaking that with very syntax specific short names. This may, in the end, be a bad idea.

[rwols]

At LSP we maintain a map (base scope -> language ID): https://github.com/sublimelsp/LSP/blob/st4000-exploration/language-ids.sublime-settings

mdpopups seems to have the inverse problem: given a (markdown) language ID, what is the corresponding base scope?

>>> sublime.find_syntax_by_scope("source.c")
[Syntax('Packages/C++/C.sublime-syntax', 'C', False, 'source.c')]

The "rule" that mdpopups can use is that the base scope is going to be "source.$ID", except for a few exceptional cases which you can maintain with a map.

[facelessuser]

I guess I'm confused, what problem are we trying to solve by using this new API? How do you envision this to help mdpopups?

[rwols]

The problem is that mdpopups.sublime_user_lang_map is awkward to use.

[rwols]

It's furthermore also not "automatic", in the sense that if you have a code block

```zig
foo

then you *have* to add this foreign language to `mdpopups.sublime_user_lang_map`. But if you follow this rule I mentioned above, it would automatically conclude that `source.zig` [would be the base scope](https://github.com/ziglang/sublime-zig-language/blob/87ecbcae6fb5718369ce3bb3472ca0b2634e78e6/Syntaxes/Zig.tmLanguage#L840), and then find the appropriate syntax automatically with sublime.find_syntax_by_scope.

[facelessuser]

Find out how? How do you propose you'd specify the code blocks?

I personally don't find it awkward. This is basically how most code highlighters work in regards to Markdown code blocks, but I get the impression you are maybe approaching this from a perspective I'm not understanding?

[rwols]

When parsing the markdown we encounter some code block, say

```cpp
int x = 5;

at this point mdpopups needs to decide what syntax to use in order to apply highlighting to the code block. I propose you can use something like this:

```python
language_id = get_language_id_from_code_block()
settings = sublime.load_settings("languageid2basescope.sublime-settings")
base_scope = settings.get(language_id, "source.{}".format(language_id))
syntaxes = sublime.find_syntax_by_scope(base_scope)
if syntaxes:
    syntax = syntaxes[0].syntax
else:
    syntax = "Packages/Text/Plain text.tmLanguage"
# use syntax to highlight the code block

This languageid2basescope.sublime-settings can look like this:

{
  "cpp": "source.c++",
  "latex": "text.tex.latex",
  "markdown": "text.html.markdown",
  "md": "text.html.markdown",
  "html": "text.html.basic",
  "xml": "text.xml",
  "javascript": "source.js"
}

This .sublime-settings file defines the exceptional cases. The default value is "source.{}".format(language_id). It would evaluate to e.g. source.c for a c code block, or source.rust for a rust code block. Or source.js for a js code block.

[facelessuser]

But I'm still not understanding why or how this is better.

If we have:

```cpp
int x = 5;

We use C++. I don't think that is complicated or awkward.

In your suggestion, where am I getting this source from? Are you expecting that we are placing the Markdown content in a view with Markdown syntax and then looking at the scope of the Markdown code block?

Are you suggesting we then specify code as:

int x = 5;

I'm not fully catching the vision here.

[rchl]

I think the main point of this is that there would be less maintenance needed to https://github.com/facelessuser/sublime-markdown-popups/blob/master/st3/mdpopups/st_mapping.py The new languages that are not currently listed would be automatically supported as long as the user has the corresponding syntax file.

Let's say there is a language called Wig with scope source.wig. The user downloads the syntax and then creates a code block with the wig language identifier and it just works. Otherwise, the user has to submit a PR here to add support for the wig language.

[facelessuser]

This is of course assuming that variations of the same language all use the same base scope. I imagine this may not always be the case. I imagine we'll still have instances of source.lang and source.variation.lang or source.variation-lang. I imagine we'll still have to keep an mapping that uses some kind of array of scopes even if we switch, but it may reduce how often variants have to manually be added.

Also, this breaks compatibility with ST3. To maintain compatibility with ST3, I'd have to maintain 2 variations. Yes, I would like to completely drop ST3 support at some time, but I can't do that yet.