metadata override through CLI flags is inconsitent

[cderv]

This has been discussed in

• https://github.com/quarto-dev/quarto-cli/issues/6013
• https://github.com/quarto-dev/quarto-cli/discussions/6119

Currently there is --metadata and --metadata-file available at Pandoc command line. This is addition of the shared metadata mechanism in Quarto project.

Those 2 flags does not behave the same and this is not clear - this is explain in https://github.com/quarto-dev/quarto-cli/issues/6013

• --metadata values will be processed through isQuartoMetadata() function, and add to metadata to be used by Quarto process. Otherwise they will be considered Pandoc only metadata.

• --metadata-file ~is only considered as Pandoc only metadata. No quarto metadata (like citation or date-format) can be passed to be processed by Quarto~ EDIT: this is the other way, it will be added with other metadata it seems

if this is the behavior we want, we should document this more clearly.

Anyhow, there is inconsistency in isQuartoMetadata() which are the source of the two links issues (#6013 and #6119). Some of the metadata are not considered as isQuartoMetadata whereas they should. This is the case of quarto render test.qmd -M citation:true or quarto render test.qmd -M date-format:"rev. DD.MM.YYYY HH:mm:ss"

This should probably be fixed to really considered all Quarto Metadata in the --metadata flag, as this is the only way to really override the YAML frontmatter value at command line execution (discussion of #6119)

Again detail explanation of the differences in https://github.com/quarto-dev/quarto-cli/issues/6013#issuecomment-1607318287

[mcanouil]

To note there are --metadata-file and --metadata-files in Quarto CLI.

• https://github.com/quarto-dev/quarto-cli/blob/39ff367eeea700c4466b0fa06ed9c743db32da9c/src/resources/schema/document-includes.yml#L122-L149

[cderv]

Oh sure but they are YAML configuration value, not CLI flags. So it is

metadata-file: mymeta.yaml

and

metadata-files: [mymeta.yml, myothermeta.yml]

This is not --metadata-file and --metadata-files

[cderv]

Some note related to previous comment and #6260:

Changing metadata-file as YAML option does not have the same treatment as --metadata-file

---
title: MWE
metadata-file: authors.yml
format: html
---

Foo

and authors.yml

author: Eve

Gives correctly Eves as author

But removing the YAML fiedl and using CLI don't

---
title: MWE
format: html
---

Foo

and

quarto render test.qmd -M metadata-file:authors.yml

We see the processing of the file is not exactly the same from Log

Using YAML in header	Using CLI
``` pandoc to: html output-file: test.html standalone: true section-divs: true html-math-method: mathjax wrap: none default-image-extension: png metadata-file: authors.yml metadata document-css: false link-citations: true date-format: long lang: en editor: visual title: MWE author: Eve ```	``` pandoc to: html output-file: test.html standalone: true section-divs: true html-math-method: mathjax wrap: none default-image-extension: png metadata-file: authors.yml metadata document-css: false link-citations: true date-format: long lang: en editor: visual title: MWE metadata-file: authors.yml ```

[dicook]

@cderv I'm afraid that I really don't understand what this problem is. But I am now encountering the error (or it seems like its a very similar error) after updating quarto to

/usr/local/bin/quarto --version
1.4.549

on the pdf branch of https://github.com/dicook/mulgar_book/tree/pdf

The exact error message is:

Error running filter /Applications/quarto/share/filters/main.lua:
/Applications/quarto/share/filters/main.lua:20212: attempt to index a nil value (field 'content')

and it only occurs when rendering to pdf. The html renders perfectly.

[cscheid]

@dicook (👋 !) almost certainly a bug on our side. If you can reduce to a small example that would help us fix it faster.

[cscheid]

@dicook I tried cloning your repo and compiling locally but it seems that it's missing the file 19-mv-time-series.qmd.

[dicook]

I don’t know why that file is missing, and it shouldn’t need that file when compiling. I’ve seen that before though.

I’m making a smaller example for you.

On 14 Feb 2024, at 9:26 am, Carlos Scheidegger @.***> wrote:

@dicook I tried cloning your repo and compiling locally but it seems that it's missing the file 19-mv-time-series.qmd. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

cheers, Di

---

Dianne Cook @.***

[dicook]

Did you clone the pdf branch? This might be the error you have with the missing file 19-mv-time-series.qmd.

The pdf branch is the one we are currently developing.

(Anyway, still working on the smaller example for you.)

[cscheid]

@dicook The issue is in your fig-liminal-trees. We should offer a much better error message here (and so it's a quarto bug), but the content of your figure is empty because all of the content is hidden, and so we fail to render it properly. From 5-nldr.qmd:

::: {#fig-liminal-trees}

::: {.content-hidden when-format="pdf"}

![Linked views of t-SNE dimension reduction with a tour of the fake trees data. The t-SNE view clearly shows ten 1D non-linear clusters, while the tour of the full 100 variables suggests a lot more variation in the data, and less difference between clusters. ](images/fake_trees1.png){#fig-trees1}
:::

::: {.content-hidden when-format="pdf"}

![Focus on the green cluster which is split by t-SNE. The shape as viewed in many linear projections shown by the tour shows that it is a single curved cluster. The split is an artifact of the t-SNE mapping.](images/fake_trees2.png){#fig-trees2}
:::

::: {.content-hidden when-format="pdf"}

![Focus on the purple cluster which splits the green cluster in the t-SNE view. The tour shows that these two clusters are distinct, but are close in one neighbourhood of the 100D space. The close proximity in the t-SNE view is reasonable, though.](images/fake_trees3.png){#fig-trees3}
:::

Three snapshots of using the `liminal` linked views to explore how t-SNE has summarised the `fake_trees` data in 2D.
:::

[dicook]

It’s taking longer to make the example than I expected, but on the positive side it might ultimately be a very specific spot in the document, that generates the error.

[dicook]

Oh, brilliant! This is fantastic, thank you!

In the meantime, I have found several other problems that needed fixing on my side.

[dicook]

BTW, my co-author using an earlier version of quarto has no problems compiling to pdf. So the new version is stricter, it seems. A good thing in the end.

[mcanouil]

@dicook please don't include replies when you are replying using email. It makes unnecessary long messages on GitHub.

[cscheid]

@dicook I'm about to check in a change to Quarto that will produce the following warnings:

WARNING (.../floatreftarget.lua:298) FloatRefTarget with no content: fig-liminal-trees
WARNING (.../floatreftarget.lua:298) FloatRefTarget with no content: fig-penguins-gt
WARNING (.../floatreftarget.lua:298) FloatRefTarget with no content: fig-penguins-hfly-plotly

That probably means you need to change fig-penguins-gt and fig-penguins-hfly-plotly on your side as well.

[cderv]

@dicook I was off last week. Glad you found the real change with Carlos. I have hidden all the message as this was slightly different topic that this enhancement issue.