CodeMap doesn't allow custom_mappers\txt.py because Universal Mapper logic error...

[Acecool]

When you try creating a txt.py custom_mapper by commenting out "text": "txt" from the universal mapper supported syntax from code_map.sublime-settings the custom mapper doesn't run at all.

If you add "text", "txt" to the exclusions list, the output window returns this, even when a custom mapper exists: Unsupported file type

The reason this happens is because of this code:

code_map.py

    def get_mapper(file, view=None):
        """"In addition to the file path, the current view syntax is passed to the function, to
        attempt detection from it rather than the file extension alone."""

        # Default map syntax is Python. It just looks better than others
        global using_universal_mapper

        if file:
            if file.lower().endswith('.cs'):
                if 'CSSCRIPT_SYNTAXER_PORT' in os.environ.keys():
                    using_universal_mapper = False
                    return Mapper.csharp_mapper.generate, py_syntax

            try:
                using_universal_mapper = True
                pre, ext = path.splitext(file)
                extension = ext[1:].lower()

                # try with mappers defined in the settings first
                # pass also the current view syntax, so that it will be checked too
                mapper = Mapper.universal_mapper.evaluate(file, extension, view)
                if mapper:
                    return mapper

                # try with mappers defined in files next
                script = extension+'.py'

                # print('trying to get...', script)

                if script in CUSTOM_MAPPERS:

                    using_universal_mapper = False
                    script = mapper_path(extension)
                    mapper = SourceFileLoader(extension + "_mapper", script).load_module()
                    syntax = mapper.map_syntax if hasattr(mapper, 'map_syntax') else py_syntax

                    return mapper.generate, syntax

                # if nothing helps, try using the universal mapping
                return Mapper.universal_mapper.evaluate(file, extension, universal=True)

            except Exception as e:
                print(e)

and this code:

code_map_support.py

class universal_mapper():
    Guess = None
    Using_tabs = False

    def evaluate(file, extension, view=None, universal=False):
        global DEPTH

        sets = settings()
        if file in DEPTH[1]:
            DEPTH[0] = DEPTH[1][file]
        else:
            DEPTH[0] = sets.get('depth')

        # Before checking the file extension, try to guess from the sysntax associated to the view
        if view:
            syntax = os.path.splitext(os.path.split(view.settings().get('syntax'))[1])[0]
            mappers = [m[0] for m in sets.get('syntaxes')]
            for i, m in enumerate(mappers):
                if m.lower() == syntax.lower():
                    universal_mapper.mapping = mappers[i]
                    syntax = sets.get(mappers[i])['syntax']
                    try:
                        with open(file, "r", encoding='utf8') as f:
                            content = f.read()
                        return (universal_mapper.generate(content), syntax)
                    except Exception as err:
                        print(err)
                        return None

        # last resort
        if universal:
            universal_mapper.mapping = "universal"
            syntax = sets.get("universal")['syntax']
            return (universal_mapper.generate(file), syntax)

        # unsupported file types
        unsupported = [syn for syn in sets.get('exclusions')]

        # get mappers/extensions defined in the settings
        mappers = [m[0] for m in sets.get('syntaxes')]
        exts = [m[1] for m in sets.get('syntaxes')]

        # TODO: universal_mapper.Guess
        universal_mapper.Guess = None

        # attempt to map a known file type as defined in the settings
        if extension in unsupported:
            return ("Unsupported file type", "Packages/Text/Plain text.tmLanguage")

        elif extension in exts:
            map = mappers[exts.index(extension)]
            universal_mapper.mapping = map

            mapper = sets.get(map)
            if not mapper:  # wrong config
                return None

            try:
                with open(file, "r", encoding='utf8') as f:
                    file = f.read()
                return (universal_mapper.generate(file), mapper['syntax'])
            except Exception as err:
                print(err)
                return None

        # not a recognized file type, will maybe return here later for fallback
        else:
            return None

In short - because evaluate in universal_mapper is returning Unsupported file type and the highlighting information, the other code evaluates

                mapper = Mapper.universal_mapper.evaluate(file, extension, view)
                if mapper:
                    return mapper

as true... Basically, it never gets to the code to look at custom_mappers\

There are several design flaws in this code - the exclusions list should be looked at before consulting the universal mapper because of this in the sublime-settings file:

    // some extensions can cause trouble if they aren't defined in advance, or
    // even if you try to define them. Extensions in this list will be ignored
    // by the universal mapper, but will be interpreted if a custom mapper is
    // present in User/CodeMap/custom_mappers
    "exclusions":   [
                        "json", "text", "txt"
                    ],

If an exclusion exists and that's the type we're looking at, it means we shouldn't even consult the universal mapper - but because the universal mapper was consulted and it sees the exclusion it returns unsupported file type ( which is probably to let the other function know that it needs to look in custom_mappers )...

There are many other ways to resolve the issue - for example instead of returning None, if we want to keep Unsupported File type return which is text ( which is what is expected because generate is called and that data is returned in the form of mapper from what I can see ) we can alter

if mapper:
    return mapper

to

if mapper and mapper[0] != "Unsupported file type":
    return mapper

and that should resolve the issue...

But I'd highly recommend adjusting some of this code to be more of a building-block style of coding, or use libraries ( classes containing functions and not needing self / instance ) this way you can simply run if universal_mapper.IsEnabled( ) and universal_mapper.HasMapper( file_ext ): return universal_mapper.RunMapper( file_ext ) - IsEnabled checks a config value - HasMapper checks to see if the current extension isn't in exclusions ( or it returns false because it doesn't have a mapper which is active - you could turn it into another function so HasMapper( file_ext ) and IsMapperEnabled( file_ext ) or leave it in HasMapper - either makes sense ) and if not in exclusions and it exists then return True so RunMapper can run it and return the data from generate..

This makes the code look a lot cleaner, reduces repetitions and if code ever needs to be modified it is always in one place instead of n places. ( I use this style as much as possible - this is what I was referring to with design and coding-standards / style when talking about merging earlier - yes it would require work but it would mean a lot less work down the road with maintenance, updates, etc.. )

For now I'll add the hack-fix to the code_map.py I have uploaded ( because the rubberbanding fix update hasn't rolled out yet ) - I'd highly suggest adding at least if mapper and mapper != 'Unsupported File type': and consider resolving the other as you have time..

[Acecool]

Note: For some reason this issue doesn't occur with Python mapper as it exists on the universal mapper - but when trying to add a txt.py custom_mapper it doesn't execute ever.... If you add the txt name to exclusions then you end up with Unsupported file type message without the custom mapper being called...

With this hack, it allows it to work... I'll dig a little deeper later to see why this is - but I did try without my mods installed and I had the same issue....

[mg979]

I just tried it and it works. I created a copy of md.py and named it txt.py. Commented out "text" from syntaxes, and made a test.txt file with markdown content. Codemap displays the map as the md custom mapper (using the txt.py copy of it).

You don't have to use the exclusions list. That one is only for file types you never want a map for. If you put an extension there, it will say 'unsupported file type'. It's meant for safety, for troublesome formats.

Note that you only have to disable a single line in the settings:

    "syntaxes":     [
                        ["universal",     ""],
                        // ["text",          "txt"],
                        ["python",        "py"],
                        ["Settings",      "sublime-settings"],
                        ["Syntax",        "sublime-syntax"],
                        ["Ini",           "ini"],
                ],

[Acecool]

I tried that and it didn't work - even without my mods installed... Odd.. I'll look at it again...

Regardless, if it somehow works without the hack - the hack is still needed because the exclusions list is supposed to allow passing through the universal mapper in favor of a custom_mapper\ but it doesn't do that... The hack allows the functionality to work as described in the settings file...

Thanks for the response mg979 - I'll take a more indepth look to see what would be causing it - but as I noted the hack is still required for the described behavior to function correctly.

[mg979]

Man... I'm not using any hack. Remove all hacks, try it before as I told you, please. It works. Yes there is some design flaw, but nothing that breaks stuff. When I added the universal mapper, I respected the way Codemap was building the map. It works, but if I had to write it again I'd make it as a standard python object, with the generated maps as attribute, instead of returning the map to the main function. But this has nothing to do with this issue. Please try it without hacks, first, and you'll see that it works. I also have to say that I am one commit behind master, so I don't know if there is something different.

[mg979]

the hack is still needed because the exclusions list is supposed to allow passing through the universal mapper in favor of a custom_mapper

I repeat: the exclusion list isn't supposed to do that. I explained it in the post above.

[mg979]

To allow 'passing to the custom mapper', all you have to do is to comment out things in the 'syntaxes' key of the settings, as I wrote above.

[Acecool]

For some reason, as I said - I could comment python out and it'd work but not for text / txt. Also, the exclusions list states that if an entry exists it will prevent the universal mapper from being called and, if a custom mapper file exists then it'll use that...

I will try from a fresh install... again.. to see if the same result happens.. Maybe a different plugin is causing an issue but I did try without my mods - I'll look into it in a bit.

[mg979]

For some reason, as I said - I could comment python out and it'd work but not for text / txt

There are 2 settings files, user and default. Make sure it's disabled in user, if you copied it over.

Also, the exclusions list states that if an entry exists it will prevent the universal mapper from being called and, if a custom mapper file exists then it'll use that...

That is my mistake for sure... The right way to use custom mappers is not to include them among syntaxes(in the settings file).

I will try from a fresh install... again..

Don't you use git? Stash changes if you made any, and checkout master.

[oleg-shilo]

Closing as no longer active