leaving markdownextradata.data empty fails

[le-yams]

Hi,

I'm having a trouble leaving markdownextradata.data empty which fails while it should search for the default _data folder.

mkdocs-markdownextradata-plugin version: 0.2.0

Current behavior:

mkdocs.yml:

plugins:
  - markdownextradata

Output:

mkdocs build
Traceback (most recent call last):
  File "c:\applications\python\python39\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "c:\applications\python\python39\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Applications\Python\Python39\Scripts\mkdocs.exe\__main__.py", line 7, in <module>
  File "c:\applications\python\python39\lib\site-packages\click\core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "c:\applications\python\python39\lib\site-packages\click\core.py", line 782, in main
    rv = self.invoke(ctx)
  File "c:\applications\python\python39\lib\site-packages\click\core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "c:\applications\python\python39\lib\site-packages\click\core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "c:\applications\python\python39\lib\site-packages\click\core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "c:\applications\python\python39\lib\site-packages\mkdocs\__main__.py", line 152, in build_command
    build.build(config.load_config(**kwargs), dirty=not clean)
  File "c:\applications\python\python39\lib\site-packages\mkdocs\commands\build.py", line 239, in build
    config['plugins'].run_event('pre_build', config=config)
  File "c:\applications\python\python39\lib\site-packages\mkdocs\plugins.py", line 96, in run_event
    result = method(**kwargs)
  File "c:\applications\python\python39\lib\site-packages\markdownextradata\plugin.py", line 68, in on_pre_build
    data_source_folders.append(ds_folder)
AttributeError: 'NoneType' object has no attribute 'append'

Expected behavior:

It should search for default _data folder as if mkdocs.yml was

plugins:
  - markdownextradata:
      data: _data

Regards, Yann

[rosscdh]

Interesting, a simple thing to fix but whats the usecase... "Why" are you using the plugin if your data is empty? dont see the point?

[rosscdh]

fixed in https://pypi.org/project/mkdocs-markdownextradata-plugin/ 0.2.1

[le-yams]

Interesting, a simple thing to fix but whats the usecase... "Why" are you using the plugin if your data is empty? dont see the point?

My _data folder is not empty (nor the file in it) ^^

[le-yams]

I just took a look at the code and it seems that the searched folder is data and not _dataas specified in the README, maybe that's the pb?

When I rename my folder to data the error change:

mkdocs build
INFO    -  Cleaning site directory
INFO    -  Building documentation to directory: C:\Applications\projects\o-cucumber\doc\site
ERROR   -  Error reading page 'index.md': 'install' is undefined
Traceback (most recent call last):
  File "c:\applications\python\python39\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "c:\applications\python\python39\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Applications\Python\Python39\Scripts\mkdocs.exe\__main__.py", line 7, in <module>
  File "c:\applications\python\python39\lib\site-packages\click\core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "c:\applications\python\python39\lib\site-packages\click\core.py", line 782, in main
    rv = self.invoke(ctx)
  File "c:\applications\python\python39\lib\site-packages\click\core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "c:\applications\python\python39\lib\site-packages\click\core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "c:\applications\python\python39\lib\site-packages\click\core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "c:\applications\python\python39\lib\site-packages\mkdocs\__main__.py", line 152, in build_command
    build.build(config.load_config(**kwargs), dirty=not clean)
  File "c:\applications\python\python39\lib\site-packages\mkdocs\commands\build.py", line 271, in build
    _populate_page(file.page, config, files, dirty)
  File "c:\applications\python\python39\lib\site-packages\mkdocs\commands\build.py", line 167, in _populate_page
    page.markdown = config['plugins'].run_event(
  File "c:\applications\python\python39\lib\site-packages\mkdocs\plugins.py", line 94, in run_event
    result = method(item, **kwargs)
  File "c:\applications\python\python39\lib\site-packages\markdownextradata\plugin.py", line 97, in on_page_markdown
    return self.apply_template(markdown)
  File "c:\applications\python\python39\lib\site-packages\markdownextradata\plugin.py", line 108, in apply_template
    return md_template.render(**self.mkdocsConfig.get("extra"))
  File "c:\applications\python\python39\lib\site-packages\jinja2\environment.py", line 1090, in render
    self.environment.handle_exception()
  File "c:\applications\python\python39\lib\site-packages\jinja2\environment.py", line 832, in handle_exception
    reraise(*rewrite_traceback_stack(source=source))
  File "c:\applications\python\python39\lib\site-packages\jinja2\_compat.py", line 28, in reraise
    raise value.with_traceback(tb)
  File "<template>", line 3, in top-level template code
  File "c:\applications\python\python39\lib\site-packages\jinja2\environment.py", line 471, in getattr
    return getattr(obj, attribute)
jinja2.exceptions.UndefinedError: 'install' is undefined

still with:

plugins:
  - markdownextradata

and no pb with:

plugins:
  - markdownextradata:
      data: data

[le-yams]

There are two "data" and one "_data" in plugin.py:

L.33

("data", mkdocs.config.config_options.Type(str_type, default=None)),

L.54

data_source_folders = self.config.get("data", {})

L.66

ds_folder = os.path.join(datadir, "_data")

With _data specified in the README.

[rosscdh]

by default it will search in the folder where your mkdocs.yml is kept and in the docs folder for another folder called _data

[le-yams]

Yes my _data is in the same directory as the mkdocs.yml.

My previous comment was not very constructive as I didn't take the time to understand the code and was just pointing out stuff that could be related to the issue. I apologize for that.

After more investigation the following change fixes the issue (still present with v0.2.1).

before:

if not data_source_folders:
            for datadir in [

after:

if not data_source_folders:
            data_source_folders = []
            for datadir in [

Python is not my main language so there might be a better/cleaner way to do it :)

Regards, Yann

[le-yams]

Hi Ross,

Did you have the time to take a look at my fix proposal ? Maybe you'd prefer I open a merge request ?

Regards, Yann

[rosscdh]

Hi Yann, did you test it with the release i mentioned above?

            for datadir in [
                os.path.dirname(config["config_file_path"]),
                config["docs_dir"],
            ]:
                ds_folder = os.path.join(datadir, "_data")
                if os.path.exists(ds_folder):
                    data_source_folders.append(ds_folder)

from above ther are 2 locations that it looks config_file_path and docs_dir for each of those if it finds a _data dir then it will append it to the data_source_folders.

[rosscdh]

please provide a tree of your project structure

tree .

[rosscdh]

so for example mine debug env is

├── cfg
│   └── settings.yaml
└── projectname
    ├── docs
    │   └── index.md
    └── mkdocs.yml

and my mkdocs.yml looks like

site_name: My Docs

plugins:
    - search
    - markdownextradata:
        data: /tmp/project/cfg

extra:
  customer:
    name: Your name here
    web: www.example.com
    salt: salt.example.com

and my index.md looks like

# Welcome to MkDocs

For full documentation visit [mkdocs.org](https://www.mkdocs.org).

## Commands

* `mkdocs new [dir-name]` - Create a new project.
* `mkdocs serve` - Start the live-reloading docs server.
* `mkdocs build` - Build the documentation site.
* `mkdocs -h` - Print help message and exit.

## Project layout

    mkdocs.yml    # The configuration file.
    docs/
        index.md  # The documentation homepage.
        ...       # Other markdown pages, images and other files.

{{ customer.name }} {{ settings.some }}

notice the settings.some. settings is the "namespace" that the variables get injected into ie.. the filename in your extra yaml data files

customer.name comes from the primary extra data-bag and the settings.some comes from the cfg/settings.yaml file you can see in my tree above

I hope thats a little more clear

[le-yams]

Hi Yann, did you test it with the release i mentioned above?

            for datadir in [
                os.path.dirname(config["config_file_path"]),
                config["docs_dir"],
            ]:
                ds_folder = os.path.join(datadir, "_data")
                if os.path.exists(ds_folder):
                    data_source_folders.append(ds_folder)

from above ther are 2 locations that it looks config_file_path and docs_dir for each of those if it finds a _data dir then it will append it to the data_source_folders.

Hi Ross,

I did try with the release 0.2.1 but still get the error

Traceback (most recent call last):
  File "c:\applications\python\python39\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "c:\applications\python\python39\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Applications\Python\Python39\Scripts\mkdocs.exe\__main__.py", line 7, in <module>
  File "c:\applications\python\python39\lib\site-packages\click\core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "c:\applications\python\python39\lib\site-packages\click\core.py", line 782, in main
    rv = self.invoke(ctx)
  File "c:\applications\python\python39\lib\site-packages\click\core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "c:\applications\python\python39\lib\site-packages\click\core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "c:\applications\python\python39\lib\site-packages\click\core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "c:\applications\python\python39\lib\site-packages\mkdocs\__main__.py", line 152, in build_command
    build.build(config.load_config(**kwargs), dirty=not clean)
  File "c:\applications\python\python39\lib\site-packages\mkdocs\commands\build.py", line 239, in build
    config['plugins'].run_event('pre_build', config=config)
  File "c:\applications\python\python39\lib\site-packages\mkdocs\plugins.py", line 96, in run_event
    result = method(**kwargs)
  File "c:\applications\python\python39\lib\site-packages\markdownextradata\plugin.py", line 68, in on_pre_build
    data_source_folders.append(ds_folder)
AttributeError: 'NoneType' object has no attribute 'append'

The problem seems to be that as data_source_folder is not set (the code you mentioned is under a if not data_source_folders:) then it doesn't have any method and so it can't data_source_folders.append(ds_folder).

That's why I was proposing to add a data_source_folder initialization.

please provide a tree of your project structure

So here are the two project structures I tried:

├── src
│   └── ...
└── doc
    ├── _data
    │     ├── foo.yml  // (edited to fix typo)
    ├── docs
    │     ├── index.md
    │     └── other.md
    └── mkdocs.yml

├── src
│   └── ...
└── doc
    ├── docs
    │     ├── _data
    │     │     └── foo.yml  // (edited to fix typo)
    │     ├── index.md
    │     └── other.md
    └── mkdocs.yml

With the following markdownextradata plugin declaration in the mkdocs.yml:

plugins:
  - markdownextradata

Thank you for your time.

Regards, Yann

[rosscdh]

what is _data/foo.md ?

yaml or json are the only supported syntax; these are data objects.. so im not sure what you have in your foo.md files?

foo.yml

my_key: some_value here

in your docs/index.md you can then reference {{ foo.my_key }}

[le-yams]

what is _data/foo.md ?

yaml or json are the only supported syntax; these are data objects.. so im not sure what you have in your foo.md files?

foo.yml

my_key: some_value here

in your docs/index.md you can then reference {{ foo.my_key }}

That was a typo in my post, it actually is foo.yml ... did you try my use case ?

[rosscdh]

Hi @le-yams thanks for sticking with it

1. have added a test for your usecase (not sure why this test was missing, not great apologies)
2. fixed the issue
3. please use 0.2.2
4. https://pypi.org/project/mkdocs-markdownextradata-plugin/