readers_map not getting picked up: RuntimeError: Un-supported file name

[elvis2]

I have the following code:

from yamlinclude import YamlIncludeConstructor

readers = {"yaml": "YamlReader", "yml": "YamlReader", "j2": "YamlReader", "yaml.j2": "YamlReader"}

YamlIncludeConstructor(reader_map=readers).add_to_loader_class(loader_class=yaml.FullLoader)

# Read the main yaml file and include the other yaml files.
with open(file_path) as f:
    data = yaml.load(f.read(), Loader=yaml.FullLoader)

I'm attempting to read files named "filename.yaml.j2". With the above code, I'm still getting the: RuntimeError: Un-supported file name

What am I doing wrong here?

[tanbro]

Sorry for that, the doc-comments are not quite well.

In fact, the reader table is not a dictionary. It is a list of two-elements tuples. First part of the tuple is a regular expression for matching file names; second part is the reader class.

The custom reader table could be:

readers_table = [
    (re.compile(r'^.+\.(([yY][mM][lL])|([Yy][aA][mM][lL]))$'), YamlReader),
    (re.compile(r'^.+\.[jJ]2$', YamlReader),  # *.j2
    (re.compile(r'^.+\.[jJ][sS][oO][nN]$'), JsonReader),
    (re.compile(r'^.+\.[iI][nN][iI]$'), IniReader),
    (re.compile(r'^.+\.[tT][oO][mL][lL]$'), TomlReader),
    (re.compile(r'^.+\.[tT][xX][tT]$'), PlainTextReader),
]

[elvis2]

With the above code, I'm not having success. I'm still getting the same "un-support file name" error.

I had to add a few more classes for the reader_table tuple. With that code, the normal ".yaml" files parse, but not the .j2 files. It seems the readers_table is till not getting picked up. Here is my code:

import argparse
import os
import yaml
import sys
import subprocess
import re

# Required for new reader_table
class Reader(object):
    # pylint: disable=too-few-public-methods
    def __init__(self, path, encoding, *args, **kwargs):  # pylint:disable=unused-argument
        self._path = path
        self._encoding = encoding

    def __call__(self):
        raise NotImplementedError()

# Required for new reader_table
class YamlReader(Reader):
    # pylint: disable=too-few-public-methods
    def __init__(self, path, encoding, loader_class, *args, **kwargs):  # pylint:disable=unused-argument
        super(YamlReader, self).__init__(path, encoding)
        self._loader_class = loader_class

    def __call__(self):
        with io.open(self._path, encoding=self._encoding) as fp:
            return yaml.load(fp, self._loader_class)

def main():
    from yamlinclude import YamlIncludeConstructor
    import in_place

    # yaml and j2 files only
    reader_table = [
        (re.compile(r'^.+\.(([yY][mM][lL])|([Yy][aA][mM][lL]))$'), YamlReader),
        (re.compile(r'^.+\.(([j][2]))$'), YamlReader),
    ]

    # Notice the reader_map argument in the constructor. Is this correct?
    YamlIncludeConstructor(reader_map=reader_table).add_to_loader_class(
        loader_class=yaml.FullLoader)
    # yaml.add_constructor('!include', YamlIncludeConstructor())

    # Read the main yaml file and include the other yaml files.
    with open(file_path) as f:
        data = yaml.load(f, Loader=yaml.FullLoader)

    return data

Does this look right? What am I missing here?

[tanbro]

Emm... i found the problem.

YamlIncludeConstructor.add_to_loader_class() is a classmethod. The method creates a new YamlIncludeConstructor instance and add it to a yaml Loader class.

And the class's constructor arguments shall be passed to the method as **kwargs.

So, if we modify above code to:

# ....

reader_table = [
    # (regex-pattern, ReaderClass),
    # ...
]
constructor = YamlIncludeConstructor(reader_map=reader_table)
yaml.FullLoader.add_constructor('!include', constructor)

# ....

yaml.load(data_or_file, yaml.FullLoader)

# ...

It should work.

[elvis2]

Thank you sir, I'll give that a try.

[elvis2]

Thanks @tanbro, that worked perfectly!