Recursive includes not working

[ksebby]

I have a master.md file which has !include statements for dep1.md and dep2.md. dep2.md also has !include statements. When I run

$ pandoc master.md --filter pandoc-include -o processed.md -t markdown -s

the include statements in dep2.md are not processed. Documentation says that recursive includes are supported since v0.3.1. I am on v0.3.2. Is there anything else needed to make recursive imports work?

[DCsunset]

Hi, @ksebby

Could you post a simple piece of code to reproduce the problem? I have tested locally and can't reproduce it.

[ksebby]

Thanks @DCsunset. I am running

pandoc docs/templates/Tools.md --filter pandoc-include -o docs/ToolsGuide.md -t markdown -s

in the root directory of this repo https://github.com/xDBio-Inc/cwl-tools/tree/combined_karl

[ksebby]

I got rid of the need for recursion in my docs as it wasn't really necessary. If you still want to play around with it you can git checkout 888943a from the main branch. Thanks.

[DCsunset]

Sorry for missing your comment 4 days ago (because Github didn't send to me any notification).

It seems that the paths in the file are not processed correctly. I may fix the bug later.

[DCsunset]

The bug has been fixed in v0.4.0. The usage of recursive includes has been updated in README.md.

[normenmueller]

@DCsunset It looks like this bug reappears in v0.7.0 ... at least on my Mac. I have one Main.md which includes xxx/Main.md which in turn includes xxx/1.md and xxx/2.md. Running pandoc Main.md --filter pandoc-include -o Main.pdf results in

Traceback (most recent call last):
  File "/Users/normenmueller/Library/Python/3.7/bin/pandoc-include", line 8, in <module>
    sys.exit(main())
  File "/Users/normenmueller/Library/Python/3.7/lib/python/site-packages/pandoc_include.py", line 160, in main
    return pf.run_filter(action, doc=doc)
  File "/usr/local/lib/python3.7/site-packages/panflute/io.py", line 258, in run_filter
    return run_filters([action], *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/panflute/io.py", line 239, in run_filters
    doc = doc.walk(action, doc)
  File "/usr/local/lib/python3.7/site-packages/panflute/base.py", line 265, in walk
    ans = list(chain.from_iterable(ans))
  File "/usr/local/lib/python3.7/site-packages/panflute/base.py", line 263, in <genexpr>
    ans = ((item,) if type(item) != list else item for item in ans)
  File "/usr/local/lib/python3.7/site-packages/panflute/base.py", line 260, in <genexpr>
    ans = (item.walk(action, doc) for item in obj)
  File "/usr/local/lib/python3.7/site-packages/panflute/base.py", line 276, in walk
    altered = action(self, doc)
  File "/Users/normenmueller/Library/Python/3.7/lib/python/site-packages/pandoc_include.py", line 108, in action
    fn + ' ' + entry + ' ' + os.getcwd())
TypeError: can only concatenate str (not "NoneType") to str
Error running filter pandoc-include:
Filter returned error status 1

Traceback (most recent call last):
  File "/Users/normenmueller/Library/Python/3.7/bin/pandoc-include", line 8, in <module>
    sys.exit(main())
  File "/Users/normenmueller/Library/Python/3.7/lib/python/site-packages/pandoc_include.py", line 160, in main
    return pf.run_filter(action, doc=doc)
  File "/usr/local/lib/python3.7/site-packages/panflute/io.py", line 258, in run_filter
    return run_filters([action], *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/panflute/io.py", line 239, in run_filters
    doc = doc.walk(action, doc)
  File "/usr/local/lib/python3.7/site-packages/panflute/base.py", line 265, in walk
    ans = list(chain.from_iterable(ans))
  File "/usr/local/lib/python3.7/site-packages/panflute/base.py", line 263, in <genexpr>
    ans = ((item,) if type(item) != list else item for item in ans)
  File "/usr/local/lib/python3.7/site-packages/panflute/base.py", line 260, in <genexpr>
    ans = (item.walk(action, doc) for item in obj)
  File "/usr/local/lib/python3.7/site-packages/panflute/base.py", line 276, in walk
    altered = action(self, doc)
  File "/Users/normenmueller/Library/Python/3.7/lib/python/site-packages/pandoc_include.py", line 132, in action
    raw, extra_args=pandoc_options)
  File "/usr/local/lib/python3.7/site-packages/panflute/tools.py", line 413, in convert_text
    out = inner_convert_text(text, in_fmt, out_fmt, extra_args)
  File "/usr/local/lib/python3.7/site-packages/panflute/tools.py", line 436, in inner_convert_text
    out = run_pandoc(text, args)
  File "/usr/local/lib/python3.7/site-packages/panflute/tools.py", line 337, in run_pandoc
    raise IOError('')
OSError
Error running filter pandoc-include:
Filter returned error status 1

[DCsunset]

@normenmueller I fixed some bugs and release v0.7.2. I hope it solves this problem.

[normenmueller]

@DCsunset Thanks for the fast reply and patch!

However, it still does not work... guess there is a problem with spaces in path names? I tried !include 10\ Anhang/Main.md but

pandoc Main.md --filter pandoc-include -o Main.pdf
Traceback (most recent call last):
  File "/Users/normenmueller/Library/Python/3.7/bin/pandoc-include", line 8, in <module>
    sys.exit(main())
  File "/Users/normenmueller/Library/Python/3.7/lib/python/site-packages/pandoc_include.py", line 161, in main
    return pf.run_filter(action, doc=doc)
  File "/Users/normenmueller/Library/Python/3.7/lib/python/site-packages/panflute/io.py", line 266, in run_filter
    return run_filters([action], *args, **kwargs)
  File "/Users/normenmueller/Library/Python/3.7/lib/python/site-packages/panflute/io.py", line 247, in run_filters
    doc = doc.walk(action, doc)
  File "/Users/normenmueller/Library/Python/3.7/lib/python/site-packages/panflute/base.py", line 265, in walk
    ans = list(chain.from_iterable(ans))
  File "/Users/normenmueller/Library/Python/3.7/lib/python/site-packages/panflute/base.py", line 263, in <genexpr>
    ans = ((item,) if type(item) != list else item for item in ans)
  File "/Users/normenmueller/Library/Python/3.7/lib/python/site-packages/panflute/base.py", line 260, in <genexpr>
    ans = (item.walk(action, doc) for item in obj)
  File "/Users/normenmueller/Library/Python/3.7/lib/python/site-packages/panflute/base.py", line 276, in walk
    altered = action(self, doc)
  File "/Users/normenmueller/Library/Python/3.7/lib/python/site-packages/pandoc_include.py", line 108, in action
    '%r %r %r' % (fn, entry, os.getcwd()))
ValueError: Included file not found: '10\xa0Anhang/Main.md' None '/Users/normenmueller/Worspace/eam/ads'
Error running filter pandoc-include:
Filter returned error status 1

[DCsunset]

The spaces needn't be escaped. You might remove the backslashes and try it again.

[normenmueller]

@DCsunset I guess the error is on my site, but !include 10 Anhang/Main.md also does not work. No error is thrown, though, but it just quits normally with a PDF but without the included content.

[DCsunset]

@normenmueller Could you post the !include lines in your Main.md and xxx/Main.md? If you add some options to pandoc-include, you could post it as well. Then I could try to reproduce it on my machine.

[normenmueller]

/doc
 |- 01 Intro/
 |- ...
 |- 10 Annex/
     |- Main.md
     |- 1.md
     |- 2.md
 |- Main.md

Main.md

---
title: '...'
subtitle: '...'
author: Normen Müller
abstract: |
  tbd
...

!include 10 Annex/Main.md

/10 Annex/Main.md

# Bla Blub

!include ./1.md

!include ./2.md

Shell:

$ cd /doc
$ pandoc Main.md --filter pandoc-include -o Main.pdf

[DCsunset]

There seems no problem when compiling on my computer. Did you update pandoc-include to v0.7.2?

[normenmueller]

My mistake. If you rename the resulting PDF but open the old PDF... then the error is in front of the computer ;-)