bug: Injected formatter weirdness

[mehalter]

Neovim version (nvim -v)

0.10.0-dev-3d8f0cb

Operating system/version

Arch Linux

Add the debug logs

• [X] I have set log_level = vim.log.levels.DEBUG and pasted the log contents below.

Log file

Log file: /home/micah/.local/state/nvim/conform.log
          14:22:25[DEBUG] prettierd exited with code 0
          14:22:25[INFO] Run injected on /home/micah/.config/nvim/test.md
          14:22:25[DEBUG] Injected format yaml:2:6: { "prettierd" }
          14:22:25[INFO] Run prettierd on /home/micah/.config/nvim/test.md
          14:22:25[DEBUG] Run command: { "prettierd", "/home/micah/.config/nvim/test.md" }
          14:22:25[DEBUG] prettierd exited with code 0
          14:23:24[DEBUG] Running formatters on /home/micah/.config/nvim/test.md: { "prettierd", "injected" }
          14:23:24[INFO] Run prettierd on /home/micah/.config/nvim/test.md
          14:23:24[DEBUG] Run command: { "prettierd", "/home/micah/.config/nvim/test.md" }
          14:23:24[DEBUG] prettierd exited with code 0
          14:23:24[INFO] Run injected on /home/micah/.config/nvim/test.md
          14:23:24[DEBUG] Injected format yaml:2:6: { "prettierd" }
          14:23:24[INFO] Run prettierd on /home/micah/.config/nvim/test.md
          14:23:24[DEBUG] Run command: { "prettierd", "/home/micah/.config/nvim/test.md" }
          14:23:24[DEBUG] prettierd exited with code 0

Describe the bug

I am getting some weirdness when I am using the injected formatter. In this case it is using prettierd but I can replicate it with prettier as well.

It seems to be formatting the code incorrectly in some very simple cases.

Here is a simple Markdown file:

---
title: Document Title
author: mehalter
output:
  pdf:
    fontsize: 12
---

# Example Markdown Document

metadata headers are in YAML with injected treesitter parser.

When I run Conform with the settings:

opts.formatters_by_ft = {
  ["*"] = { "injected" },
  markdown = { "prettierd" },
  yaml = { "prettierd" },
}

I get the following result:

---
title: Document Title
author: mehalter
output:
pdf:
fontsize: 12
---

# Example Markdown Document

metadata headers are in YAML with injected treesitter parser.

Thank injected formatting of the YAML is wrong.

I was thinking maybe the full formatting was messing up the YAML block before it got to the injected formatter, but the strange thing is if I format the entire file with prettierd or prettier from the commandline it actually outputs the correct result (running prettier test.md). So it leads me to think that it is something to do with the ranged formatting of the YAML block.

If I copy and paste the metadata header into a YAML file such as:

title: Document Title
author: mehalter
output:
  pdf:
    fontsize: 12

prettier also formats this YAML file correct as does Conform.

It seems that the issue is coming directly from the injected formatter.

Steps To Reproduce

1. Open the minimal example file with repro: nvim -u repro.lua test_file.md
2. Format with require("conform").format()

Expected Behavior

Correct formatting, or at least non-destructive formatting

Minimal example file

---
title: Document Title
author: mehalter
output:
  pdf:
    fontsize: 12
---

# Example Markdown Document

metadata headers are in YAML with injected treesitter parser.

Minimal init.lua

-- DO NOT change the paths and don't remove the colorscheme
local root = vim.fn.fnamemodify("./.repro", ":p")

-- set stdpaths to use .repro
for _, name in ipairs { "config", "data", "state", "cache" } do
  vim.env[("XDG_%s_HOME"):format(name:upper())] = root .. "/" .. name
end

-- bootstrap lazy
local lazypath = root .. "/plugins/lazy.nvim"
if not vim.loop.fs_stat(lazypath) then
  vim.fn.system {
    "git",
    "clone",
    "--filter=blob:none",
    "--single-branch",
    "https://github.com/folke/lazy.nvim.git",
    lazypath,
  }
end
vim.opt.runtimepath:prepend(lazypath)

-- install plugins
local plugins = {
  "folke/tokyonight.nvim",
  { -- enable treesitter for injected formatting
    "nvim-treesitter/nvim-treesitter",
    config = function()
      require("nvim-treesitter.configs").setup {
        ensure_installed = { "markdown", "yaml" },
        highlight = { enable = true },
      }
    end,
  },
  {
    "stevearc/conform.nvim",
    config = function()
      require("conform").setup {
        log_level = vim.log.levels.DEBUG,
        formatters_by_ft = {
          ["*"] = { "injected" },
          markdown = { "prettier" },
          yaml = { "prettier" },
        },
      }
    end,
  },
}
require("lazy").setup(plugins, {
  root = root .. "/plugins",
})

vim.cmd.colorscheme "tokyonight"
-- add anything else here

Additional context

I have tried testing this every which way to try and figure out how to reproduce the final result with just running formatting commands from the command line and I can't seem to. Let me know if have any ideas!

[stevearc]

I'm not able to get this to reproduce. For some reason nvim 0.9 doesn't even find the yaml format range, but nightly does. However, the issue still doesn't reproduce even with nightly (v0.10.0-dev-e9b9a86). Can you try upping the log level to TRACE? And do you have any prettier configs in this directory that could affect the formatting?

19:44:16[DEBUG] Running formatters on /tmp/test.md: { "injected" }
19:44:16[INFO] Run injected on /tmp/test.md
19:44:16[TRACE] Input lines: { "---", "title: Document Title", "author: mehalter", "output:", "  pdf:", "    fontsize: 12", "---", "", "# Example Markdown Document" }
19:44:16[TRACE] Injected formatter regions { { "yaml", 2, 6 } }
19:44:16[DEBUG] Injected format yaml:2:6: {}
19:44:16[TRACE] Injected format lines { "title: Document Title", "author: mehalter", "output:", "  pdf:", "    fontsize: 12" }
19:44:16[TRACE] Applying formatting to /tmp/test.md
19:44:16[TRACE] Comparing lines { "---", "title: Document Title", "author: mehalter", "output:", "  pdf:", "    fontsize: 12", "---", "", "# Example Markdown Document" } and { "---", "title: Document Title", "author: mehalter", "output:", "  pdf:", "    fontsize: 12", "---", "", "# Example Markdown Document" }
19:44:16[TRACE] Diff indices {}
19:44:16[TRACE] Applying text edits: {}
19:44:16[TRACE] Done formatting /tmp/test.md
19:44:16[TRACE] Applying formatting to /tmp/test.md
19:44:16[TRACE] Comparing lines { "---", "title: Document Title", "author: mehalter", "output:", "  pdf:", "    fontsize: 12", "---", "", "# Example Markdown Document" } and { "---", "title: Document Title", "author: mehalter", "output:", "  pdf:", "    fontsize: 12", "---", "", "# Example Markdown Document" }
19:44:16[TRACE] Diff indices {}
19:44:16[TRACE] Applying text edits: {}
19:44:16[TRACE] Done formatting /tmp/test.md

[mehalter]

I can help debug this more next week. All of my reproducing was done with the minimal configuration both with nightly and 0.9.

[mehalter]

Just did a few quick tests and I must not have tested in 0.9. I can confirm in 0.9 I am also not getting the YAML language detection.

When installing Neovim nightly I do get the behavior using the minimal repro.lua above and the provided test file:

0. rm -rf .repro, make sure clean environment
1. nvim -u repro.lua, perform the initial setup, treesitter parser compilation, etc.
2. :e test.md, load the file after setup to have treesitter and everything
3. :lua require("conform").format()

Here is a full recording of me doing the process with neovim nightly (NVIM v0.10.0-dev-e9b9a86, forgot to do nvim --version in recording):

https://asciinema.org/a/M1zXukyBon9Enqk4H5KJPJZNQ

[mehalter]

Oh and here are the logs from me changing the debug level to TRACE:

Log file: /home/micah/Documents/test_conform/.repro//state/nvim/conform.log
          17:15:39[TRACE] Output lines: { "title: Document Title", "author: mehalter", "output:", "pdf:", "fontsize: 12" }
          17:15:39[TRACE] Applying formatting to /home/micah/Documents/test_conform/test.md
          17:15:39[TRACE] Comparing lines { "---", "title: Document Title", "author: mehalter", "output:", "  pdf:", "    fontsize: 12", "---", "", "# Example Markdown Document", "", "metadata headers are in YAML with injected treesitter parser." } and { "---", "title: Document Title", "author: mehalter", "output:", "pdf:", "fontsize: 12", "---", "", "# Example Markdown Document", "", "metadata headers are in YAML with injected treesitter parser." }
          17:15:39[TRACE] Diff indices { { 5, 2, 5, 2 } }
          17:15:39[TRACE] Applying text edits: { {
              newText = "pdf:\n",
              range = {
                ["end"] = {
                  character = 4,
                  line = 5
                },
                start = {
                  character = 0,
                  line = 4
                }
              }
            } }
          17:15:39[TRACE] Done formatting /home/micah/Documents/test_conform/test.md

[mehalter]

Oh also, this directory is clean (freshly created) and I have no prettier configuration files anywhere on my machine

[stevearc]

Could I get just a little be more context from that log file? I can see from the Output lines: { line that it's coming back unindented, but the earlier lines should show the output from when prettier is formatting the yaml snippet. Seeing that will either show us that it's formatting the yaml incorrectly, or it will give me the exact inputs that I can use to get a repro even if there's something different about my system (e.g. prettier version)

[mehalter]

Ah, sorry about that!

17:15:38[DEBUG] Running formatters on /home/micah/Documents/test_conform/test.md: { "prettier", "injected" }
17:15:38[INFO] Run prettier on /home/micah/Documents/test_conform/test.md
17:15:38[TRACE] Input lines: { "---", "title: Document Title", "author: mehalter", "output:", "  pdf:", "    fontsize: 12", "---", "", "# Example Markdown Document", "", "metadata headers are in YAML with injected treesitter parser." }
17:15:38[DEBUG] Run command: { "prettier", "--stdin-filepath", "/home/micah/Documents/test_conform/test.md" }
17:15:38[DEBUG] prettier exited with code 0
17:15:38[TRACE] Output lines: { "---", "title: Document Title", "author: mehalter", "output:", "  pdf:", "    fontsize: 12", "---", "", "# Example Markdown Document", "", "metadata headers are in YAML with injected treesitter parser." }
17:15:38[INFO] Run injected on /home/micah/Documents/test_conform/test.md
17:15:38[TRACE] Input lines: { "---", "title: Document Title", "author: mehalter", "output:", "  pdf:", "    fontsize: 12", "---", "", "# Example Markdown Document", "", "metadata headers are in YAML with injected treesitter parser." }
17:15:38[TRACE] Injected formatter regions { { "yaml", 2, 6 } }
17:15:38[DEBUG] Injected format yaml:2:6: { "prettier" }
17:15:38[TRACE] Injected format lines { "title: Document Title", "author: mehalter", "output:", "  pdf:", "    fontsize: 12" }
17:15:38[INFO] Run prettier on /home/micah/Documents/test_conform/test.md
17:15:38[TRACE] Input lines: { "title: Document Title", "author: mehalter", "output:", "  pdf:", "    fontsize: 12" }
17:15:38[DEBUG] Run command: { "prettier", "--stdin-filepath", "/home/micah/Documents/test_conform/test.md" }
17:15:39[DEBUG] prettier exited with code 0
17:15:39[TRACE] Output lines: { "title: Document Title", "author: mehalter", "output:", "pdf:", "fontsize: 12" }
17:15:39[TRACE] Applying formatting to /home/micah/Documents/test_conform/test.md
17:15:39[TRACE] Comparing lines { "---", "title: Document Title", "author: mehalter", "output:", "  pdf:", "    fontsize: 12", "---", "", "# Example Markdown Document", "", "metadata headers are in YAML with injected treesitter parser." } and { "---", "title: Document Title", "author: mehalter", "output:", "pdf:", "fontsize: 12", "---", "", "# Example Markdown Document", "", "metadata headers are in YAML with injected treesitter parser." }
17:15:39[TRACE] Diff indices { { 5, 2, 5, 2 } }
17:15:39[TRACE] Applying text edits: { {
    newText = "pdf:\n",
    range = {
      ["end"] = {
        character = 4,
        line = 5
      },
      start = {
        character = 0,
        line = 4
      }
    }
  } }
17:15:39[TRACE] Done formatting /home/micah/Documents/test_conform/test.md

Aha! I see the issue now, I can reproduce this actually at the command line now (thank you so much for this incredible log format)

The issue is coming form prettier deciding how to format the file based on the file extension. If you create a local file with just the YAML content named test_just_yaml.md and run

cat test_just_yaml.md | prettier --stdin-filepath test_just_yaml.md

then it will format this as if it were markdown and not yaml

cat test_just_yaml.md | prettier --stdin-filepath some_random_string.yml

results in the correct formatting (the filepath string can be a random string since the --stdin-filepath is just there to inspect the filename and it doesn't look at the file at all from there)

For reference, the test_just_yaml.md file is this:

title: Document Title
author: mehalter
output:
  pdf:
    fontsize: 12

[stevearc]

Can you try out #199 and see if it works for you?

[mehalter]

@stevearc yup that seems to work! Thanks so much!