Feature request: make include_files recursive

[gabyx]

it would be awesome if somebody could enhance the include_files filter to be recursive (aborting with an error if already found paths are to be included -> cycles)

[gabyx]

Also: Does pandoc.read https://github.com/pandoc/lua-filters/blob/88520397dc2e921f6b1fdfd855def1fe20dc0774/include-files/include-files.lua#L42

apply the same command line settings already given or is it a completely new invocation of pandoc, meaning it will neglect what the user used (e.g. extensions and such) ? This is crucial as it should get parsed with the same settings as the original document, shouldn't it?

[tarleb]

The current method is as follows: if a format is given via the format attribute, then we use that. Extensions can be specified in the same way that they are used on the command line, e.g.

``` {.include format=markdown+emoji}
emoji-table.md
```

Pandoc's Markdown is assumed if no format is given.

One could modify the format of all include blocks by using another Lua filter before calling include-files.lua. E.g.:

-- file: set-format.lua
function CodeBlock (cb)
  if cb.classes:includes('.include') then
    cb.attributes.format = 'markdown+emoji'
  end
  return cb
end

Then call pandoc with pandoc --lua-filter set-format.lua --lua-filter include-files.lua ….

We could rewrite the filter to default to the main format spec, but that would require some changes to main pandoc, as the format extensions are available only as bit-flags, which is not nice to use.

[gabyx]

Ah, thats nice. So I think we are pretty close to a file-transclusion filter, right?, Only the recursion is still needed ? Thanks for your effort!

[gabyx]

@tarleb : Is it possible to read the from: markdown+markdown_in_html_blocks+link_attributes+tex_math_dollars value , which might be in some global variable (?), inside the set-format.lua filter?

[bpj]

The manual says that there is a global variable

PANDOC_READER_OPTIONS Table of the options which were provided to the parser.

I imagine it would include the extensions, although I have a feeling this topic came up before. You can always check by dumping the contents. If you put this in a "filter" you should get the key : value pairs printed to stdout

function Pandoc ()
  for k,v in pairs(PANDOC_READER_OPTIONS) do
    print(k,":",v)
  end
  return nil
end

[tarleb]

Only the internal representation is available, i.e., an integer representing a bitfield of enabled extensions. Decoding into the usual string representation is not supported yet.

[gabyx]

thx!

Von meinem iPhone gesendet

Am 02.08.2020 um 18:38 schrieb Albert Krewinkel notifications@github.com:

Only the internal representation is available, i.e., an integer representing a bitfield of enabled extensions. Decoding into the usual string representation is not supported yet.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[gabyx]

For interested people: The correct set-format.lua filter should be

-- https://github.com/pandoc/lua-filters/issues/101

format = nil

function setFormatIncludeBlock (cb)
  if cb.classes:includes('include') and cb.attributes['format'] == nil then
    cb.attributes.format = format
  end
  return cb
end

function setFormat(doc)
    format = doc.meta["include-format"]
    print("Settings include blocks to format" .. format)

    local blks =  pandoc.walk_block(
      pandoc.Div(doc.blocks),
      { CodeBlock = setFormatIncludeBlock }
    ).content

    return pandoc.Pandoc(blks, doc.meta)
end

return {
  {Pandoc = setFormat}
}

[gabyx]

Actually this set-format.lua filter can not work (IMO) since it does not recursively set the format in all files... It just does it in the main file... @tarleb is my finding correct? So the include-files.lua should read a meta option include-format (if existing) and use that as a default, otherwise markdown as default.

[tarleb]

Sounds about right. Would be nice to update pandoc to make this possible.

[gabyx]

If that is done I would suggest renaming the attribute tag to -> .transclude its more clear :-) (refering to include-files.lua)