jgm / pandoc

Universal markup converter
https://pandoc.org
Other
33.19k stars 3.3k forks source link

New Feature: internal links to tables and figures and headers #813

Open GeraldLoeffler opened 11 years ago

GeraldLoeffler commented 11 years ago

It's currently possible to include internal links to sections. I'd like to propose a similar feature for links to figures/images and tables.

It may make sense to provide this feature only if the figure/image or table that is being linked to has a caption. In that case Pandoc can today automatically generate a number for the figure or table and include it in the caption, e.g. "Figure 15".

At the most basic, the text of the link would be provided by the user, as is currently the case for links to sections.

Of course it would be very convenient if the automatically generated number for the figure or table would also be used for the text of the link, e.g. "as can be seen in Figure 15, blah", where "Figure 15" would be the internal link whose text is auto-generated from the figure it points to.

kovla commented 11 years ago

That would be lovely indeed. In academic writing it is quite often necessary, and while automatic numbering of figures and tables is nice, it really should be linked to what is in the text.

nichtich commented 11 years ago

One could use the figure caption as link target, similar to links to captions:

![la lune](lalune.jpg "Voyage to the moon")

...is shown in figure [la lune]...

And/or without automatic generation of link text:

...is shown in [the figure](#la-lune)...

See also issue #615 on automatic numbering of figures and tables in HTML output.

liob commented 10 years ago

I concur. However, @nichtich suggestion breaks the current syntax. Maybe a less intrusive approach would be a syntax like:

![Voyage to the moon](lalune.jpg){la lune}

It would be great to be able to reference figures. As @nichtich said: it is nearly a requirement in academic writing.

jgm commented 10 years ago

A more consistent format would be

![Voyage to the moon](lalune.jpg){#lalune}

See the current attribute format for headers.

liob commented 10 years ago

indeed, that is a more consistent format.

About the implementation: I see 2 major ways to implement this feature:

  1. Emulate something like the latex figure environment and output the figure as image with plain text underneath. Very much like figures are handled now in docx format, except that you put "Figure 1:" at the beginning. This would be the most portable way and should be fairly easy to implement in all format writers. However, than pandoc has to keep track of the references itself for cross referencing.
  2. Implement it the "proper" way in the corresponding format writer. Sticking with the docx example: Adding a caption to the image and then cross reference it in the text.

Can anybody (@jgm ?) make an educated guess on how much work either of the solutions will be?

aaren commented 10 years ago

I agree - this is essential for academic writing. I wish I knew Haskell!

The current way around this, in the mailing list discussion, is functional but clumsy.

Would this mean using \autoref in the latex? Then from markdown input:

...is shown in [the figure](#la-lune)...

you would get the latex output:

...is shown in \autoref{la-lune}...
aaren commented 10 years ago

sort of relevant pr: https://github.com/jgm/pandoc/pull/509

Mailing list discussion:

AvverbioPronome commented 10 years ago

![Voyage to the moon](lalune.jpg){#lalune}

I just tried to write something like

Some text

![Bla blah](pic.png)   {#something}

Some other text

I was surprised that did not work. It showed the image without caption, and a raw "{#something}" afterwards.

I assumed curly braces were for assigning attributes to anything... :D

CFCF commented 10 years ago

A workaround with numbered example lists is added to https://github.com/jgm/pandoc/issues/904

For my purposes, this method works well with docx.

Utsira commented 10 years ago

I agree that being able to reference figures is essential to academic writing. The workarounds linked to above aren't really satisfactory, in my opinion

![Voyage to the moon](lalune.jpg){#lalune} would be perfect

srhb commented 10 years ago

Similar syntaxes would be very interesting for equations, too. In fact, why not adopt a completely general syntax? It would be especially nice if it could carry over to LaTeX bits, once you have to bail out and use say \begin{align} and friends.

frederik-elwert commented 10 years ago

I have sympathy for the numbered example list approach, mainly for two reasons: Firstly, what we want are not really links but references, and secondly, the use case for numbered example lists is already close to, e.g., numbered equations. The example from the docs is close to a typical use case for figure references:

(@good)  This is a good example.

As (@good) illustrates, ...

This mechanism can already be used for figure references, as CFCF pointed out:

![Figure (@primitive_hut): The primitive hut](Illustrations\primitive_hut.png)

As can be seen in Figure (@primitive_hut), huts may be primitive.

# Index of Figures

(@primitive_hut) *Primitive hut* from the frontispiece of Marc-Antoine Laugier’s 1755 second edition of *Esssay on Architecture*, illustration by Charles-Dominique-Joseph-Eisen.

However, there are a few drawbacks:

Thus, a proper referencing scheme would need a bit additional thinking. Especially, PDF and HTML output should work alike, probably by pandoc adding the Figure: bit to HTML output, while leaving it to LaTeX in the PDF case. Additionally, this should also work for referencing numbered sections, like in see chapter (@mychapter).

btel commented 10 years ago

Your workaround works as suggested, but I had to remove the parentheses when referencing the label, otherwise they were rendered in the output. After this modification my example looks like this:

Figure @figure is about being in time

![Figure @figure: Cubes](cubes.png)

(@figure) Figure 1

To remove the automatic numbering in LaTex (Figure 1:, etc.) you can add to the template:

\usepackage[labelformat=empty]{caption}

After rendering to pdf this produces the following output:

figure

bitsgalore commented 10 years ago

Just came across this issue as well and ended up here. I'm also really in favor of support for the syntax suggested by @jgm above:

![Voyage to the moon](lalune.jpg){#lalune}

Especially since this is the standard way of dealing with this in PHP Markdown Extra:

http://michelf.ca/projects/php-markdown/extra/#spe-attr

mangecoeur commented 9 years ago

Has there been any developments on this? It also seems to me that @jgm suggestiong

![Voyage to the moon](lalune.jpg){#lalune}

is the most consistent internally and with other tools. What would need to happen for this to be implemented?

edwardabraham commented 9 years ago

I was wanting to add support for this addition to the syntax. When trying to replicate papers using markdown for the scholmd project, this is the feature that stands out as most needed by Pandoc . In short this can be addressed through the general use of {#lalune} for labelling elements, and of @lalune for referencing the number of the corresponding element. The syntax (@) may be used to number elements that are otherwise unnumbered.

A general syntax for labels {#lalune}, that are associated with the preceding element would allow for any element to be labelled (paragraphs, equations, tables, etc.). By associating the label with an element in the abstract syntax tree, the properties of the element would be available when the reference was made, and so they can be numbered appropriately. This syntax is already used in one context in Pandoc (section heading labels), and is used by PHP Markdown extra. For elements that don't have numbers, such as equations, the syntax (@) may be used (from the example_lists extension). So an equation would be numbered and labelled as $$ F = G{m_1 m_2 \over r^2$$ (@) {#gravity}. (An alternative could be to use the example_lists extension style and number and label it in one go as $$ F = G{m_1 m_2 \over r^2$$ (@gravity). There are clearly some details and edge cases to be thought through here.)

When the document is rendered, Pandoc would associate a number with each labelled element, based on its type, and its position in the document. This logic would need to be carried out in Pandoc, so that it was available to the range of back-end writers (including HTML). The philosophy would be similar to Pandoc-citeproc, which carries out its own formatting of citations, rather than delegating to writers that support this approach (such as bibtex for latex). An option would to have this behaviour depend on the backend (so that it in latex it inserts \label and \ref commands), but elsewhere it may insert calculated numbers, if referencing is not supported by the backend. This has the advantage that it will work easily in contexts where only a fragment of the document is rendered. If pandoc is calculating the numbering, a syntax would be needed for specifying the start numbers in a fragment that wasn't being compiled in stand alone mode.

Labelled elements may be linked to, with the @ symbol being used to indicate the reference. So a trip to [the moon](@lalune) would be an anchor link to the element labelled {#lalune}. In this case the text is rendered as a trip to the moon.

The syntax The moon is illustrated in Figure @lalune may be used to insert the number of the referenced element, as well as a link to that element, with the text rendered as The moon is illustrated in Figure 1. This follows the syntax used for referencing numbered lists with the example_lists extension.

A further syntax could be to use square brackets [@lalune] to insert the type and number of the element that is referenced, similar to the behaviour of latex's \autoref command. So, the moon is illustrated in [@lalune] would be rendered as the moon is illustrated in Figure 1 (including a link to the anchor). To implement this feature would require some localisation or customisation capability, so that the word used to describe the element could be specified. In its simplest, this customisation could be put in the YAML header, with for example figure_label: Fig. if the style required a shortened label. The syntax for the reference, [@lalune], is the same as is used by the pandoc-citeproc library, so it would be overloading that usage to implement a self-citation. Pandoc would have the information on the context that is needed to either format it as a citation, or as a reference, assuming that there was no collision between the labels and the citation keys.

kovla commented 9 years ago

@edwardabraham It must be pointed out that the syntax [@lalune] is already used in pandoc for bibliographical citations.

timtylin commented 9 years ago

@kovla @edwardabraham I don't see why #lalune couldn't be used also as a reference to the defined symbol. With this scheme [the moon](#lalune) could be a normal text link to the figure, while [#lalune] could do the numbered reference thing as mentioned. In fact, I have a custom build of Pandoc that does exactly this.

edwardabraham commented 9 years ago

@kovla The idea was to deliberately overload the [@lalune] syntax that is used for citations. The reason being that references to another part of the document are similar to citations (in essence they are self-citations). This has the benefit of avoiding introducing additional syntax. During processing the filter would identify which element the label was attached to, and use that information to appropriately format the text that is inserted into the document.

@eVITAERC I prefer using the @ symbol, as it extends functionality that is already used by example_lists. Are you able to structure your pandoc build so that it may be implemented as a filter?

Note that this extension is [@lalune] a convenience and is not necessary, provided that the numbers are able to be accessed through the @lalune method.

timtylin commented 9 years ago

@edwardabraham I tried the @ approach as well, but internal feedback in our lab showed that people get confused by what is a citation and what is an internal reference even when editing. The conclusion was that the mental model of keeping # for internal refs and using @ for external refs is the simplest to grok.

In fact, no one out of ten or so people have used example_lists (we are mostly writing extended abstracts and journal papers in the field of physics/engineering/applied math). When encountering a "list of scenarios" situation, the content was so static that people simply used literal numbers without issue.

Unfortunately the internal reference mechanism required heavy modification of the Markdown reader (additional state must be kept during the parsing process) and a custom AST, so I can't conceive of a filter implementation in the near future.

elcritch commented 9 years ago

Personally, the # symbol and the {#label} syntax would be easier to understand and use. In my mind citations and internal references follow very distinct "mental models". Many academic papers use distinct numbering for figures, tables, and equations but the proposed syntaxes don't appear to have a way to support distinct numberings by type. It would be an important design criteria (I only got to skim the comments, hopefully its not a redundant suggestion). @edwardabraham You mentioned the scholmd. Is it currently just a repository of ideas or have they implemented any of the academic markdown features? @eVITAERC Great work! Is it possible for you to propose submitting the changes to the pandoc project or alternately creating a github fork to allow others to experiment?

mangecoeur commented 9 years ago

+1 for use of # symbol for internal references. But it's really important that the references can distinguish between equations, figures, and tables to have distinct numbering sequences.

There are two approaches to my mind

  1. make the "thing referenced" explicit in the tag, for instance using namespaces like #eqn.maxwells, #fig.hockeystick. Pandoc would have to track the objects in each namespace and format the references appropriately
  2. depend on pandoc's parser to know what type of thing is referenced and handle appropriately. So if you tag an image and then use a # reference pandoc automatically treats it as a "fig" reference, if you embed latex formula it because an equation reference etc. This would be cool but i suspect it would be a) complex and b) fragile - you get issues for instance if someone wants to embed an image for a formula.
bpj commented 9 years ago

I agree very much that internal references, citations and references to numbered examples are different, and @ is already too overloaded by being used for both the latter. The problem with #reference which I can see is that it might get confused with a level one header since atx headers don't requre a space after the hashmarks as far as I know. I think {#anchor} and [#reference] would be good because then the id could be any valid HTML id including LaTeX-y things like {#img:la-lune}. As for doing different things with different anchors that is probably best left to filters.

Note that you could already do something like <span id="img:la-lune">![Voyage to the Moon](lalune.jpg)</span> and then [Voyage to the Moon](#img:la-lune) and get anchors/labels and links which work in both HTML and LaTeX. If you don't want hyperlinks in your LaTeX, numbered images in your HTML etc. that can be done with filters, e.g. replacing links with URLs like the one in my example with a span containing the link text plus a raw LaTeX string \ref{img:la-lune}. (See http://johnmacfarlane.net/pandoc/try/?text=%3Cspan+id%3D%22img%3Afoo-bar%22%3E!%5BA+bar+frequented+by+foos%5D(foo-bar)%3C%2Fspan%3E%0A%0A%5BThe+foo+bar%5D(%23img%3Afoo-bar).&from=markdown&to=latex)

It would be nice to have a less ugly syntax, but note that you would need to turn references into links when generating HTML, while it is rather trivial to have a filter do the opposite when generating LaTeX. Note also that you could abuse the link title, having the filter leave links with a title alone so that you get a hyperlink. It would even be easy to use spans with certain classes and/or attributes in the source and have one filter which turns them into references when generating LaTeX and one which turns them into links when generating HTML. I'm going on holiday tomorrow but I would be happy to write those filters when I get back! :-)

szarnyasg commented 9 years ago

My goal is to produce HTML and PDF outputs from the same Markdown file, with the PDF containing references that can be printed (e.g. "See figure 1") . I found a cumbersome workaround inspired by @bpj's idea. Note that it does not work with pandoc 1.12.2.1 found in the Ubuntu APT repository, so I installed 1.12.4.2 from Cabal instead.

The following Markdown code:

<span id="pic.jpg"></span>

![A bar frequented by foos](pic.jpg)

[The foo bar](#pic.jpg).

Produces the following HTML code:

<p><span id="pic.jpg"></span></p>
<div class="figure">
<img src="pic.jpg" alt="A bar frequented by foos" /><p class="caption">A bar frequented by foos</p>
</div>
<p><a href="#pic.jpg">The foo bar</a>.</p>

This works reasonably well: the empty paragraph is not displayed so the link will navigate you to the image.

The generated LaTeX code is the following:

\label{pic.jpg}{}

\begin{figure}[htbp]
\centering
\includegraphics{pic.jpg}
\caption{A bar frequented by foos}
\end{figure}

\hyperref[pic.jpg]{The foo bar}.

The generated \label is of no use. Instead, we should add the label after the caption has been inserted. To do this, we save the filename of the current figure to a variable (\currentfigure) by redefining the \includegraphics command. We then redefine the \caption command to insert the caption and add the label from the variable. We also have to redefine the \hyperref command to \autoref. To achieve this, we edit the LaTeX template file's preamble:

\let\oldincludegraphics\includegraphics
\renewcommand*{\includegraphics}[1]{\oldincludegraphics{#1}\def\currentfigure{#1}}
\let\oldcaption\caption
\renewcommand*{\caption}[1]{\oldcaption{#1}\label{\currentfigure}}
\renewcommand*{\hyperref}[2][\ar]{%
  \def\ar{#2}
  #2 (\autoref{#1})}

In the final PDF document, the caption and the reference look like this. "Figure 1" is also a hyperlink.

Figure 1: A bar frequented by foos

The foo bar (Figure 1).

While I think this workaround can be used in practice, it would be nice to have a syntax for inserting cross references in a simpler and less error-prone way.

bpj commented 9 years ago

Note that the following works identically without the need to (re)define any LaTeX commands and without generating an 'empty' paragraph (including the fact that at least in my PDF reader the link jumps to the caption rather than to the top of the image):

<div id="fig:lalune">
![A voyage to the moon\label{fig:lalune}](lalune.jpg)

</div>

[The voyage to the moon](#fig:lalune).

It is slightly less elegant in that you have to specify the id/label twice, and slightly more elegant in that you avoid the empty span element and the resulting empty paragraph.

Note that the blank line inside the div is necessary in order to make pandoc see the div contents as a paragraph, and thus to get the image inside a figure environment. With the blank line the resulting LaTeX is like this:

\begin{figure}[htbp]
\centering
\includegraphics{lalune.jpg}
\caption{A voyage to the moon\label{fig:lalune}}
\end{figure}

but without it it is just like this:

\includegraphics{lalune.jpg}

Den 2014-08-10 22:38, Gábor Szárnyas skrev:

My goal is to produce HTML and PDF outputs from the same Markdown file, with the PDF containing references that can be printed (e.g. "See figure 1") . I found a cumbersome workaround inspired by @bpj's idea. Note that it does not work with pandoc 1.12.2.1 found in the Ubuntu APT repository, so I installed 1.12.4.2 from Cabal instead.

The following Markdown code:

<span id="pic.jpg"></span>

![A bar frequented by foos](pic.jpg)

[The foo bar](#pic.jpg).

Produces the following HTML code:

<p><span id="pic.jpg"></span></p>
<div class="figure">
<img src="pic.jpg" alt="A bar frequented by foos" /><p class="caption">A bar frequented by foos</p>
</div>

This works reasonably well: the empty paragraph is not displayed so the link will navigate you to the image.

The generated LaTeX code is the following:

\label{pic.jpg}{}

\begin{figure}[htbp]
\centering
\includegraphics{pic.jpg}
\caption{A bar frequented by foos}
\end{figure}

\hyperref[pic.jpg]{The foo bar}.

The generated \label is of no use. Instead, we should add the label after the caption has been inserted. To do this, we save the filename of the current figure to a variable (\currentfigure) by redefining the \includegraphics command. We then redefine the \caption command to insert the caption and add the label from the variable. We also have to redefine the \hyperref command to \autoref. To achieve this, we edit the LaTeX template file's preamble:

\let\oldincludegraphics\includegraphics
\renewcommand*{\includegraphics}[1]{\oldincludegraphics{#1}\def\currentfigure{#1}}
\let\oldcaption\caption
\renewcommand*{\caption}[1]{\oldcaption{#1}\label{\currentfigure}}
\renewcommand*{\hyperref}[2][\ar]{%
   \def\ar{#2}
   #2 (\autoref{#1})}

In the final PDF document, the caption and the reference look like this. "Figure 1" is also a hyperlink.

Figure 1: A bar frequented by foos

The foo bar (Figure 1).

While I think this workaround can be used in practice, it would be nice to have a syntax for inserting cross references in a simpler and less error-prone way.


Reply to this email directly or view it on GitHub: https://github.com/jgm/pandoc/issues/813#issuecomment-51726484

szarnyasg commented 9 years ago

@bpj, thanks for the quick reply.

(including the fact that at least in my PDF reader the link jumps to the caption rather than to the top of the image)

I looked at this issue and found that it can be fixed easily by adding \usepackage{caption} to the template (see http://tex.stackexchange.com/questions/27096/href-to-an-image-label-how-to-jump-to-the-image-instead-of-the-caption-below-t for details).

It is slightly less elegant in that you have to specify the id/label twice, and slightly more elegant in that you avoid the empty span element and the resulting empty paragraph.

I agree, I also prefer your solution.

Note that the blank line inside the div is necessary in order to make pandoc see the div contents as a paragraph, and thus to get the image inside a figure environment.

Wow. I experimented with using a div element but couldn't get is working. Adding the extra newline did the trick.

bpj commented 9 years ago

Den 2014-08-11 10:03, Gábor Szárnyas skrev:

@bpj, thanks for the quick reply.

@szarnyasg, you are welcome; sleeplessness has its advantages! ;)

(including the fact that at least in my PDF reader the link jumps to the caption rather than to the top of the image)

I looked at this issue and found that it can be fixed easily by adding \usepackage{caption} to the template (see http://tex.stackexchange.com/questions/27096/href-to-an-image-label-how-to-jump-to-the-image-instead-of-the-caption-below-t for details).

It does indeed! If a similar trick were possible in HTML one could just wrap the caption text in a span:

![<span id="fig:lalune">A voyage to the Moon</span>](lalune.jpg)

In practice you don't even see that you end up at a figure caption, though.

It is slightly less elegant in that you have to specify the id/label twice, and slightly more elegant in that you avoid the empty span element and the resulting empty paragraph.

I agree, I also prefer your solution.

Not needing to hack LaTeX is always preferable! :)

Note that the blank line inside the div is necessary in order to make pandoc see the div contents as a paragraph, and thus to get the image inside a figure environment.

Wow. I experimented with using a div element but couldn't get is working. Adding the extra newline did the trick.

See here and here for the explanation!

BTW for those who use Vim with the UltiSnips plugin I made a snippet for using this idiom which endeavors to (optionally) reduce typing duplication to a minimum:

# # ANCHORED FIGURE IN SOURCE FOR BOTH HTML AND LATEX
# 
#     <div id="fig:ID/LABEL/NAME">
#     ![CAPTION\label{fig:ID/LABEL/NAME}](ID/LABEL/NAME.jpg})
# 
#     </div>
# 
# ## Tabs
#
# Tab     Description                  Default
# ------  ---------------------------  -------
# $1      the entire id/label[^a]
# $2      id/label prefix              fig[^b]
# $3      id/label unique part
# $4      caption text
# $5      filename minus extension     $3
# $6      extension                    jpg[^c]
#
# [^a]: You should normally just skip this tab or the 'magic' with $5 will be lost!
# [^b]: The following `:` is automatically removed if you delete the prefix.
# [^c]: The separating dot is automatically removed if you delete the extension
#       -- for use with the --default-image-extension option!
#
# NOTE: The blank line inside inside the div is necessary to make pandoc
#   see a paragraph and thus place the image inside a figure environment!
#
snippet figdiv "Anchored figure in HTML and LaTeX" b
<div id="${1:${2:fig}${2/.+/:/}${3:ID/LABEL/NAME}}">
![${4:CAPTION}\\label\{$1}](${5:$3}${6/.+/./}${6:jpg})

</div>

$0
endsnippet

Reply to this email directly or view it on GitHub: https://github.com/jgm/pandoc/issues/813#issuecomment-51751650

ivotron commented 9 years ago

quick clarification regarding the div method. Redefining \hyperref is still needed in order to include a (Figure #)

srhb commented 9 years ago

@bpj

I'm sorry but I had a bit of trouble following the last bit of your conversation with @szarnyasg. Does it mean that you found a way to to produce correct (LaTeX) references to figures, images and tables?

szarnyasg commented 9 years ago

@srhb: I managed to get it working, although it's a bit cumbersome. I created a Hungarian thesis template which is available here:

It requires Pandoc 1.12.4.2+.

HTH, Gabor

srhb commented 9 years ago

@szarnyasg Thank you kindly!

balachia commented 9 years ago

I spent a bit of time trying to sort through this issue with a pandoc filter instead of by redefining \hyperref in default.latex.

TL;DR: compile the script below and use it as a pandoc --filter when converting to latex. Use the div trick to get html internal linking to work.

https://gist.github.com/balachia/d836f8829aec61cb4b54#file-pandoc-internalref-hs

Pandoc doesn't make \ref's anywhere when writing out latex so instead you have to inject them by using the pandoc format's RawInline type and using some kind of pattern matching to figure out where to do it. Right now I'm pattern matching on any internal link that starts with "#fig:" or "#tab:" and I'm wiping out whatever text the user specifies for the text words in favor of the \ref text. So you get the equivalent of:

Figure words -> Figure \ref*{fig:lefig}

That said, it's probably possible to pattern match on a better pattern. With some thinking, it might be possible to avoid the div trick too, by pulling out images in divs. Not sure about that one yet, though.

balachia commented 9 years ago

I've been trying to port one my papers from LyX to markdown so this was a good project to procrastinate with. Right now I've written up a filter (pandoc-internalref) that tries to implement the ![caption](image){#fig:reference} syntax, and seems to be doing a good job, at least for html and latex.

The haskell source for it is at https://github.com/balachia/pandoc-filters.

I'd love to get some feedback on how that works out.

bpj commented 9 years ago

Den 2014-09-17 00:14, Tony Vashevko skrev:

I've been trying to port one my papers from LyX to markdown so this was a good project to procrastinate with. Right now I've written up a filter (pandoc-internalref) that tries to implement the ![caption](image){#fig:reference} syntax, and seems to be doing a good job, at least for html and latex.

The haskell source for it is at https://github.com/balachia/pandoc-filters.

I'd love to get some feedback on how that works out.

I had the idea to abuse the image title to hold the id/label info: [caption](path "title {#id .class}"), mostly because it was easy to 'parse' with Perl,^1 but this certainly looks nicer, if it's robust enough.

aaren commented 9 years ago

I've written a python filter that:

  1. allows arbitrary attributes on images (![caption](filename){#id .class key=value})
  2. converts these images to labeled figures in latex output
  3. overwrites all internal links with autoref{} in latex output
  4. automatically numbers figures and links to them in html / markdown output
  5. overwrites internal links to figures e.g. see [](#fig:lalune) -> see [Figure 1](#fig:lalune)
  6. does the same for sections

The code is at https://github.com/aaren/pandoc-reference-filter

It is pretty customisable (just look at the source) and avoids redefining latex commands. '\autoref' could easily be replaced by '\ref'. Currently I'm doing the markdown links like [](#fig:lalune), but it might be that there is a more natural syntax. I've considered [#fig:lalune] as a replacement.

KurtPfeifle commented 9 years ago

Thanks aaren,

your filter compiled without a problem. I wanted to play with it, but your guidelines about how to prepare the markdown source is not clear to me.

The current syntax of Pandoc's markdown for images is this

Caption (optional, may be empty

So I tried to add your "new" syntax somehow, but it does not really "fit" into the current syntax:

Caption; optional, may be empty{#mypic} Caption; optional, may be empty {#mypic} ...plus some more variations

with an attempt to link to the image from a different place in the document:

See this picture for more...

Could you please give some additional pointers?

On Sat, Sep 27, 2014 at 9:40 PM, aaren notifications@github.com wrote:

I've written a python filter that:

  1. allows arbitrary attributes on images ([image: caption] http://filename{#id .class key=value})
  2. converts these images to labeled figures in latex output
  3. overwrites all internal links with autoref{} in latex output
  4. automatically numbers figures and links to them in html / markdown output
  5. does section numbering

The code is at https://github.com/aaren/pandoc-reference-filter

It is pretty customisable (just look at the source) and avoids redefining latex commands. Currently I'm doing the markdown links like , but it might be that there is a more natural syntax. I've considered [#fig:lalune] as a replacement.

— Reply to this email directly or view it on GitHub https://github.com/jgm/pandoc/issues/813#issuecomment-57063360.

aaren commented 9 years ago

Regarding syntax: if we went with the [#fig:lalune] syntax, the collision with the existing syntax would be with implicit reference links that begin with a #. For example,

Here is an implicit reference link: [#1]

Here is a link to a figure: [#lalune]

[#1]: example.com

![caption](lalune.png){#lalune}

This would only be a problem when the reference is multiply defined, therefore I suggest the following definition:

This could be implemented with a filter: undefined reference links are passed through the markdown reader as a string. We would just have to walk the tree and find strings that match [#ref] then link them up.

aaren commented 9 years ago

@KurtPfeifle maybe better to keep this list clean - can you start an issue on my repo? I need to see your input file, the pandoc command you are using and the output it is giving you.

the syntax is ![caption](filename){#id} with the image in a separate paragraph.

KurtPfeifle commented 9 years ago

On Sat, Sep 27, 2014 at 10:54 PM, aaren notifications@github.com wrote:

@KurtPfeifle https://github.com/KurtPfeifle maybe better to keep this list clean - can you start an issue on my repo?

Sorry for the "spam" then. It happened inadvertently -- I just replied to a mail that arrived in my inbox, without realizing it could be regarded as polluting a "list".

aaren commented 9 years ago

@KurtPfeifle no problem. I was just meaning that discussion about whether my filter does what you expect it to should happen on the repo for that filter. I'm happy to help, just I'm concious that this issue is getting pretty long and it would be good to avoid too much tangential back and forth.

Start an issue on my repo and I'll see if I can fix your problem.

KurtPfeifle commented 9 years ago

Funnily -- when I now click on the link pointing to your repo 'pandoc-reference-filter' I end up in a different repo than the one where I previously retrieved this Haskell source code from:

1 {- 2 Pandoc filter that cleans up internal references to figures and tables. 3 It's an attempt to deal with: https://github.com/jgm/pandoc/issues/813 4 Compile with: 5 6 ghc --make pandoc-internalref.hs 7 8 and use in pandoc with 9 10 --filter [PATH]/pandoc-internalref 11 -} 12 module Main where 13 import Text.Pandoc.JSON 14 import Data.List 15 - 16 main = toJSONFilter makeref 17 - 18 makeref :: Inline -> [Inline] 19 makeref (Link txt ('#':ident, x))- 20 | Just subident <- stripPrefix "fig:" ident = reflink- 21 | Just subident <- stripPrefix "tab:" ident = reflink- 22 where reflink = [Link [RawInline (Format "latex") ("\ref*{" ++ ident ++ "}")]("#" ++ ident, x)] 23 makeref x = [x]

Dunno what changed meanwhile...

In any case, the filter in the new location isn't Haskell, but Python (as your post said), and the filter code which is written in Haskell from a different repo (https://github.com/aaren/pandoc-filters) has 52 lines of code (not 23) -- but it works :-)

So given that your original outline of how to use the extended syntax (item "1." in your list) seems to have been incorrect and is replaced by

caption{#id}

now, I'm happy for the time being :-)

On Sat, Sep 27, 2014 at 9:40 PM, aaren notifications@github.com wrote:

I've written a python filter that:

  1. allows arbitrary attributes on images ([image: caption] http://filename{#id .class key=value})
  2. converts these images to labeled figures in latex output
  3. overwrites all internal links with autoref{} in latex output
  4. automatically numbers figures and links to them in html / markdown output
  5. does section numbering

The code is at https://github.com/aaren/pandoc-reference-filter

It is pretty customisable (just look at the source) and avoids redefining latex commands. Currently I'm doing the markdown links like , but it might be that there is a more natural syntax. I've considered [#fig:lalune] as a replacement.

— Reply to this email directly or view it on GitHub https://github.com/jgm/pandoc/issues/813#issuecomment-57063360.

aaren commented 9 years ago

Ok, I think you confused me with @balachia! We have both proposed filters that do similar things recently. To clarify:

I edited item 1 in my list because github wasn't rendering the non-backticked code as I wanted. This probably didn't show up on email.

mpickering commented 9 years ago

Feel free to add links to your filters to the wiki page. I'm sure other users will find them useful.

aaren commented 9 years ago

@mb21 whilst you're trying to address #261, I'd suggest a slight alteration to the latex writer that would go some way to solving this issue.

Currently, if you define an image alone in a paragraph it is turned into a figure in latex, i.e.

![blah](fig)

becomes

\begin{figure}[htbp]
\centering
\includegraphics{fig}
\caption{blah}
\end{figure}

What I suggest is passing the id of the attr through to the label of the figure, i.e.

![blah](fig){#thefigure}

becomes

\begin{figure}[htbp]
\centering
\includegraphics{fig}
\caption{blah}
\label{thefigure}
\end{figure}

Whilst this wouldn't solve this issue it would be a big step in the right direction. I understand if you'd like to avoid scope creep though - it's great that anyone is putting Attr on Image at all! Thank you :)

mb21 commented 9 years ago

@aaren Good idea. Should ![blah](fig){#thefigure} go to \label{thefigure} or \label{fig:thefigure}? For consistency with HTML-ids probably the former, although there is a LaTeX convention to use the latter. Since you cannot reference it with [link](#thefigure) anyhow (which gets converted to \thefigure[blah]{link} instead of \ref{thefigure}), this issue will still be only half-solved, but if you're targeting only LaTeX you can just write \ref{thefigure}.

aaren commented 9 years ago

@mb21 definitely the former. Prepending fig: is a convention, not a rule.

Yes it only half solves this issue but it avoids the current figure-labeling workarounds which aren't very graceful (e.g. putting the \label in the caption).

mpickering commented 9 years ago

@mb21 See how the attributes are currently handled for divs and spans.. I can't remember the precise command we use there.

tlnagy commented 9 years ago

Most of the discussion here is centered around labeling single figures, but what about figures containing separate panels, all of which may be referenced independently? It's been my experience that most scientific articles have figures like Fig. 1A, 1B, ... 1N and they must be referenced independently of each other, but displayed in the same panel. Currently, I believe I can hack a version using the latex subfigure package and \label, but this would not work with either the HTML or the DOCX output. This would be a more complex case to handle, but no less important IMO.

aaren commented 9 years ago

@tlnagy I think that comes into a separate issue - having a Figure block element that allows you to have subfigures. Have a look at what @timtylin is doing with scholdoc.

Currently, having a single image in a paragraph become a figure is a bit of a hack. It may be possible for this to coexist in the future with a multi panel schema, but we'd need to decide what that is. There might already be an existing issue for this but I'm not sure.

mb21 commented 9 years ago

(Yes, I went without the fig: prefix, the headers had no prefix either.)

So now you'll be able to add manual internal references to figures like:

![](image.png){#myFigure}

As can be seen in [Figure 1](#myFigure)...

(The LaTeX writer actually already converts [text](#link} to \hyperref[link]{text} and not \href{link}{text}, therefore the above works.)

To generate the reference text (i.e. "Figure 1") automatically, this really should be done in a filter so as to make it consistent across all output formats, as not all output formats (e.g. HTML) have a \autoref command (this is similar to how bibliography entries are handled by the pandoc-citeproc filter instead of LaTeX). I like aaron's idea of overloading the empty link syntax for that ([](#link)), since I cannot image ever actually wanting an empty link.