Output to md or qmd

[serhez]

The original jupytext.nvim plugin can output the file to markdown format. It seems this plugin can only output to python format; are there plans to support md or even qmd in the future? Thanks!

[GCBallesteros]

We are now always using the auto mode of jupytext. It's a sensible choice but it does remove some flexibility. I will add this to my ToDo.

Thanks for the suggestion!

[GCBallesteros]

This is addressed in #2. @serhez would you mind test driving the support-more-than-just-auto branch.

[eth3lbert]

I've also implemented another version that solves this issue as well 28ae67d.

[serhez]

Hello @GCBallesteros ! I took some time, but I finally tested it. It works quite well, thank you very much for the feature! This is my config:

require("jupytext").setup({
  style = "hydrogen",
  custom_language_formatting = {
    python = {
      extension = "qmd",
      style = "quarto",
    },
    julia = {
      extension = "qmd",
      style = "quarto",
    },
  },
})

Do you think it would be also good to auto-switch to the created file (e.g., quarto, md, python, etc.) in order to get other plugins to recognize the filetype (e.g., treesitter, quarto, headlines...)? I find myself switching to this file whenever I load a ipynb.

[GCBallesteros]

@serhez Yeah that's a good idea I will implement it into the open PR

[GCBallesteros]

Done. I made it optional to stop the LSP from lighting up like a xmas tree if the user's configuration is not appropriately set up to deal with non-native language formats like quarto. The configuration for the custom_language_formatting is now as follows:

custom_language_formatting = {
    python = {
      extension = "qmd",
      style = "quarto",
      force_ft = false, -- <----- this one behave like before the latest commit
    },
    julia = {
      extension = "qmd",
      style = "quarto",
      force_ft = true, -- <---- Switch the filetype to Julia even if it is not implied by the file extension
    },
  }

I don't have an appropriately set up neovim to test quarto notebooks. @serhez would you mind pulling the latest commit from the PR branch and retesting. Thanks!

[serhez]

Thank you @GCBallesteros ! I gave the new changes a try. One thing I noticed is that the quarto header now is prepended twice to the beginning of the file if I use force_ft = true; it does not happen when this variable is set to false:

The key thing I noticed is that the filetype is set to python, however the file format is quarto (or markdown). This makes pyright (or, I assume, any other LSP, linter, formatter, etc.) to go crazy trying to figure out the syntax. I think the filetype should be set to quarto instead, for my case; in the general case, I'm not sure how you can get the filetype given the user's settings, but there are only so many possible custom_language_formatting[lang].extension right? Another thing I noticed is, if we are now treating the ipynb file as a quarto file (or markdown, python, whatever extension is specified in the settings), why create a second file with that extension? Is it necessary for something in particular? Could we just have the ipynb file with a changed filetype?

I appreciate you listening to the suggestions. Really love the plugin ❤️

[GCBallesteros]

I see. I think that should be working now. I ended up installing quarto to do some actual testing and everything seems to be working now and will merge into main soon enough.

When force_ft=true and style="quarto" the filetype is set to quarto and nvim then knows not to start the LSP but still use the appropriate syntax highlighting. I think that you stilll need something like otter.nvim to really get all the LSP stuff working.

Another thing I noticed is, if we are now treating the ipynb file as a quarto file (or markdown, python, whatever extension is specified in the settings), why create a second file with that extension? Is it necessary for something in particular? Could we just have the ipynb file with a changed filetype?

I don't quite understand the suggestion so I will just reply with a brief explanation of the internals of the plugin. When you open an ipynb file you request jupytext to create a plain text representation of it, the .py, .qmd ... file. This file is then read and loaded into the buffer instead of the ipynb file which is just a bunch of json. When you save the plain text file gets saved and then jupytext updates the corresponding ipynb file. Depending on the situation the plain text file may be deleted once you quit vim.

In the case were the file will be deleted at the end one could mostly get rid of it, maybe even completely(?), by having jupytext read from stdin instead of using actual files. Food for thought!

[serhez]

It works quite well now, I simply work in quarto by just opening the ipynb file without having to do anything else. Thanks!

[GCBallesteros]

That is merged. Along the way I also fixed the header duplication issue and a bug were the after writing the notebook buffer wasn't being set to nomodified which forced the user to force quit.