Problem loading sublime-color-scheme files

[brupelo]

I was trying to use the official Monokai.sublime-color-scheme with syntect but it seems this file-format is not supported or maybe it's just me doing something fishy.

I've tried to load it both as json or as xml (ie: used this converter https://www.freeformatter.com/json-to-xml-converter.html) but when feeding it with ThemeSet::get_theme it will crash.

On the other hand, if using a proper thTheme such as monokai.tmTheme there won't be any problem.

[trishume]

Yah the new color scheme format is different and not supported yet and possibly ever

[brupelo]

Just for the record... my previous thumbdown icon is mainly because your "possibly ever" end of statement, why do you think this one is not interesting but supporting old verbose meaty xml files that should be burn in hell is better?

You know, I know and everybody knows... but here's some random link google gave me after searching "json vs xml" -> https://blog.cloud-elements.com/json-better-xml.

As for json vs yaml I don't have any strong opinnion though, here's some random link https://www.json2yaml.com/yaml-vs-json

[trishume]

Mostly because I only do open source for fun and out of generosity, and supporting the new theme format is so far down my todo list that I'll never do it myself. I'd be happy to accept a PR to implement it, but I don't think anyone really needs it since nearly all themes are available as tmThemes and I think the new format is reasonably complex, so I don't expect anybody else to contribute it.

[brupelo]

I understand, makes total sense, thanks to let me know :) . Kind of related... today I was wondering about something related... I asked in the math stackexchange site something I'm not sure about, ie: can-be-an-open-source-project-considered-a-cooperative-game, probably it won't receive any answer but it's intriguing me.

Anyway, I do think the first requirement for a project to become cool&successful is the author/contributors/teams having fun so it makes total sense what you're saying there, my thumbdown is not valid anymore lol.

On a serious note, the reason why I've opened this issue was mainly cos last night I was testing some monokai xml file I've found on the internet and I've noticed the outcome wasn't exactly the same than my dear Sublime's monokai, take a look here:

So my first thought was using directly sublime's so I don't need to even learn how to tweak these thThemes mysefl... Too many years using monokai so these subtle differences disturb me :)

Ps. Also the method names are not blue :(

[brupelo]

As of Sublime Text 3 build 3149, a new color scheme format .sublime-color-scheme was introduced for easier editing, customization and addition of new features. The documentation for the new format is available at the main Color Schemes documentation.

This page documents Sublime Text's implementation of the legacy .tmTheme format. Support for this legacy format will remain for backwards compatibility, but new features will only be added to the .sublime-color-scheme format.

I'm sure you were already aware of these docs... but it's quite interesting, specially the part which says new features will only be added to the .sublime-color-scheme format. :(

[ntwb]

FYI: I just installed this Sublime plugin and opened up a relatively new theme.sublime-color-scheme format file and CTRL+SHIFT+P to run the SublimeColorSchemeConverter: Convert to tmTheme command to which I ended up with what appears to be a pretty good XML formatted theme.tmTheme file.

I'm hoping, anything that is a new feature from the source file will simply be ignored by syntect and, fingers crossed, it not throw any errors.

I'll use this tmTheme with https://github.com/sharkdp/bat for a few days to see how it goes 🎉

[jrappen]

Relevant Sublime Text docs here:

https://www.sublimetext.com/docs/3/color_schemes.html

Currently undocumented:

• font_style: glow for ST4051+ (API view.style_for_scope() ST4063+)

There are a few Sublime Text color scheme features only supported by *.sublime-color-scheme (and/or *.hidden-color-scheme) files. Unfortunately I do not have a list of differences. I assume starting with the changelog section on the docs page linked above would be a start.

---

Also I only saw support for hidden syntax files, not hidden color scheme files here. Compare the following file extensions:

# TextMate format

*.tmTheme
*.hidden-tmTheme

# Sublime Text format

*.sublime-color-scheme
*.hidden-color-scheme

[eproxus]

Tried the method suggested by @ntwb and it seems the plugin doesn't work for modern .sublime-color-scheme files anymore. Since more and more Sublime themes are migrated or created in this format, the .tmTheme format feels more dead by the day.

Would love for syntect to support this new format!

[keith-hall]

out of curiosity, what color schemes have you found that can't be converted using that plugin, and what error was shown in the ST console? I noticed it for some reason currently has some unnecessary limitations like not supporting comments in the json color schemes (rather than just ignoring them, if it's too tricky to output in the tmTheme).

[eproxus]

@keith-hall Could be because of Sublime Text 4 and Python 3 though (build 4074). I'm getting this when converting a Gruvbox theme:

Traceback (most recent call last):
  File "/Applications/Sublime Text.app/Contents/MacOS/Lib/python33/sublime_plugin.py", line 1314, in run_
    return self.run(edit)
  File "/Users/alind/Library/Application Support/Sublime Text/Installed Packages/SublimeColorSchemeConverter.sublime-package/sublime_color_scheme_converter.py", line 221, in run
  File "/Users/alind/Library/Application Support/Sublime Text/Installed Packages/SublimeColorSchemeConverter.sublime-package/sublime_color_scheme_converter.py", line 155, in parse
AttributeError: 'NoneType' object has no attribute 'get'

[keith-hall]

I had a play around with it, and this works for me with that color scheme:

"""
SublimeColorSchemeConverter

Converts sublime-color-scheme json files into tmTheme plist files.
"""

import os
import re
import json
#import plistlib
import colorsys
import traceback
import collections

import sublime
import sublime_plugin

from .lib import plistlib

re_var = re.compile(r"var\([^\)]+\)")
re_alpha = re.compile(r"alpha\(((0\.)?[0-9]+)\)")
re_hex = re.compile(r"(#[0-9,a-z,A-Z]{6})([0-9,a-z,A-Z]{2})?")
re_rgb = re.compile(r"rgb\((\d+),\s?(\d+),\s?(\d+)(,\s?(\d+\.?\d*))?\)")
re_hsl = re.compile(r"hsl\((\d+),\s?(\d+)%,\s?(\d+)%(,\s?(\d+\.?\d*))?\)")

def alpha_to_hex(a):
    return "{:02x}".format(int(255*float(a)))

def hexa_to_hex(hex, a):
    return hex[:7] + alpha_to_hex(a)

def rgb_to_hex(r, g, b, a=None):
    hexcode = "#{:02x}{:02x}{:02x}".format(r, g, b)
    if a:
        hexcode += alpha_to_hex(a)
    return hexcode

def get_alpha_adjuster(string, default=None):
    alpha = re_alpha.search(string)
    if alpha:
        return float(alpha.group(1))
    else:
        return default

def get_alpha_hex(hexcode):
    if len(hexcode) < 8:
        return None
    else:
        return int(hexcode[7:], 16)

def match_hex(string):
    result = None
    match = re_hex.search(string)
    if match:
        result = match.group(1)
        alpha = get_alpha_adjuster(string)
        if alpha:
            result += alpha_to_hex(alpha)
        elif match.group(2):
            result += match.group(2)
    return result

def match_rgb(string):
    match = re_rgb.search(string)
    result = None
    if match:
        r = int(match.group(1))
        g = int(match.group(2))
        b = int(match.group(3))
        a = get_alpha_adjuster(string, match.group(5))
        result = rgb_to_hex(r, g, b, a)
    return result

def match_hsl(string):
    match = re_hsl.search(string)
    result = None
    if match:
        h = int(match.group(1))/360
        s = int(match.group(2))/100
        l = int(match.group(3))/100
        a = get_alpha_adjuster(string, match.group(5))
        r, g, b = [255*int(v) for v in colorsys.hls_to_rgb(h, l, s)]
        result = rgb_to_hex(r, g, b, a)
    return result

def try_match_color(string):
    hexcode = match_hex(string)
    if not hexcode:
        hexcode = match_rgb(string)
    if not hexcode:
        hexcode = match_hsl(string)
    if hexcode:
        alpha = get_alpha_adjuster(string, None)
        if alpha:
            hexcode = hexa_to_hex(hexcode[:7], alpha)
    else:
        hexcode = string
    return hexcode

class ConvertSublimeColorSchemeCommand(sublime_plugin.TextCommand):

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.filename = None
        self.theme = None
        self.output = None
        self.output_view = None

    def convert_name(self, string):
        return re.sub(
            "_([a-z,A-Z,0-9])",
            lambda match: match.group(1).upper(),
            string
        )

    def parse_color(self, key, variables):
        if key in variables.keys():
            print(variables[key], key, try_match_color(variables[key]))
            return try_match_color(variables[key])
        else:
            var = re_var.search(key)
            if var:
                color = variables.get(var.group(), var.group())
                return try_match_color(key.replace(var.group(), color))
            else:
                return try_match_color(key)

    def parse_settings(self, settings, variables):
        parsed = {}
        for key in list(settings):
            value = settings[key]
            parsed[self.convert_name(key)] = self.parse_color(value, variables)
        return parsed

    def parse_rules(self, rules, variables):
        parsed = []
        for settings in rules:
            rule = {}
            name = settings.pop("name", None)
            if name:
                rule["name"] = name
            scope = settings.pop("scope", None)
            if scope:
                rule["scope"] = scope
            rule["settings"] = self.parse_settings(settings, variables)
            parsed.append(rule)
        return parsed

    def parse(self):
        name = self.theme.get("name", None)
        author = self.theme.get("author", None)
        variables = self.theme.get("variables", None)
        globals_ = self.theme["globals"]
        rules = self.theme["rules"]

        for key in list(variables):
            variables["var({})".format(key)] = variables.pop(key)

        # handle nested variables
        for key in variables.keys():
            variables[key] = self.parse_color(variables[key], variables)

        settings = self.parse_settings(globals_, variables)
        rules = self.parse_rules(rules, variables)
        self.theme = {
            "name": name,
            "author": author,
            "settings": [{"settings": settings}] + rules
        }

    def convert(self):
        error = False
        try:
            plistbytes = plistlib.dumps(self.theme, sort_keys=False)
            self.output = plistbytes.decode('UTF-8')
        except Exception:
            error = True
            sublime.error_message("Could not convert Sublime Color Scheme")
            print("SublimeColorSchemeConverter:")
            print(traceback.format_exc())
        return error

    def read_source(self):
        self.theme = sublime.decode_value(self.view.substr(sublime.Region(0, self.view.size())))
        return False

    def write_buffer(self, edit):
        error = False
        #output_name = os.path.splitext(os.path.basename(self.filename))[0] \
        #              + ".tmTheme"
        try:
            self.output_view = self.view.window().new_file()
            self.output_view.set_encoding("UTF-8")
            #self.output_view.set_name(output_name)
            self.output_view.replace(
                edit,
                sublime.Region(0, self.view.size()),
                self.output
            )
            self.output = None
        except Exception:
            error = True
            sublime.error_message("Could not write buffer")
            print("SublimeColorSchemeConverter:")
            print(traceback.format_exc())
        return error

    def run(self, edit):
        if not self.read_source():
            if not self.parse():
                if not self.convert():
                    self.write_buffer(edit)
        self.filename = None
        self.theme = None
        self.output = None

I'm not convinced it correctly calculates nested alpha modifiers though...

[eproxus]

@keith-hall I think this is mentioned in the README:

The plugin tries to replace all patterns, where any color is recognized. Therefore color(#000000 blend(#ffffff 50%)) is replaced by #000000, since the blend() adjuster syntax is not supported.