Support projects without a JSON library in the dependencies

[ajayvigneshk]

Creating an issue from the TODO item to have discussion

I explored if using the ETF format could be an option. There exists a lua parser for it. But sadly this seems to make use of lua 5.3 constructs which is very hard to get it working on neovim (lua 5.1). More specifically, the parser uses

• bit shift operators << which seems to be supported under a different module / syntax ref
• string.unpack function for which all the existing options falls short in different ways
  • lua-struct does not return the bytes read as the second return value. Based on an issue comment on the repo, it seems like a simple change to return it.
  • lua-pack is not a drop-in replacement. returns values in base-2 formats 🤷🏻
  • lua-compat-5.3. This is possibly the only option that seemed to work, but it messes up with global values and that breaks somewhere in plenary code

Curious if there are any other ways / options to achive this

[halfdan]

This might be as simple as implementing a subset of the JSON encoding ourselves. Encoding seems to be fairly straightforward: https://github.com/cblage/elixir-json/blob/master/lib/json/encoder/default_implementations.ex

[ajayvigneshk]

Yeah perhaps. Just that writing our own JSON encoder seems way further from the actual project's scope of being a neovim test framework (not to forget the extra maintenance due to it)

[ajayvigneshk]

Mix.install looks interesting (yet to try it out though)

[jfpedroza]

I tried embedding Poison (as it has fewer files) but the command takes 25 seconds compiling every time before running the tests. It's probably the protocols.

Embedding a version that doesn't use protocols is an option, but I agree that is further from the scope of the library.

[jfpedroza]

I'm sure Mix.install would work in a completely independent file, but we would have to test how it behaves when you then run a project after requiring the script.

[jfpedroza]

Basically, we need a way to transfer the following structure to Lua:

%{
  id: "some id with potentially special chars",
  status: "simple string",
  output: "/some/file/path" or nil,
  errors: [
    %{
      message: "some text with potentially special chars",
      line: number or nil
    }
  ]
}

With #3 the special chars in the id would be removed, assuming files don't have anything weird, and I guess the errors could be saved to files and read on the Neovim side.

If we don't have to worry much about escaping characters, then it becomes easier.

[jfpedroza]

Adding Mix.install makes it fail like this:

[ajayvigneshk]

The actual error is this %Mix.Error{message: "Mix.install/2 cannot be used inside a Mix project", mix: 1} Which makes sense, since the formatter is invoked in the same context as the mix project.

Am thinking if we can split the formatter into two GenServers,

• one as a part of a mix project(so we can add deps), start it before running a test under a node name. It should handle the file writing part.
• one that we can register as the formatter that will simply forward to the above GenServer (not sure how easy / possible this is)

Seems like overkill, but I can't think of any other option. I'm not sure if there would be any negative implications due to this

[jfpedroza]

If both instances are nodes then we can connect them and use remote calls for communication but that also adds more complexity and then the embeded encoder option doesn't sound so bad xD

I tried implementing the encoder without protocols and it takes less than 70 lines: https://github.com/jfpedroza/neotest-elixir/blob/jp/embedded-json-encoder/neotest_elixir/json_encoder.exs

[ajayvigneshk]

Yeah that seems good and apt!