Support build tool integration

[alfaix]

Ideally, we would like to automatically discover executables for tests with Bazel, CMake, and Conan.

For CMake cmake-tools.nvim may be useful.

[alejandroclaro]

Agree with this. It's confusing that a manual rebuild is needed to actually get an updated result of the test.

Maybe an option in the configuration to configure a command to execute before running the test suite that receives the executable configured in the registry as parameter.

[jla2000]

There is https://github.com/hfn92/cmake-gtest.nvim which does achieve running Gtest tests by querying for the correct executables using cmake-tools.nvim. Maybe it could be used as a starting point.

[mr-j0nes]

Indeed, this would be much appreciated.

[ll-nick]

For me, this is a bit of a show stopper. I work with very large code bases where this causes two problems:

• It's quite cumbersome to point to the correct executables.
• I only build tests for the parts of the code I am working on. With the current version of the repo, the test summary will be populated with hundreds of tests, most of which I do not care about. With it being so cluttered up, it's almost impossible to use. So it would be amazing, if only tests show up, where the corresponding executable is found.

Since I am very interested in a solution (everything else works flawlessly!) I'd try and give it a shot myself. This one might be a bit beyond my capabilities though, so if you could give some hints/pointers/ideas on how to approach it, that would be much appreciated.

[alfaix]

Hey @ll-nick, assuming your build tool is CMake, you can use cmake-tools.nvim to integrate; I did this MR some time ago and it was recently merged, which should allow you to use the lua API. You will have to convert this API to coroutines for use with nio, see how they do it here via task.wrap. While I was working on it, there were a lot of rough edges in the plugin, stemming mainly from the lack of tests, so if your subset of the API ends up being small, you can use CMake API directly as a last resort.

Then you can write a couple of exploration unit tests for the functionality you would need in CMake (that is: find a target by source file; find a target by executable; recompile a target), and add those in the correct path.

When discovering executables, before prompting the user, you can query CMake first if there's a CMakeLists.txt and use that as the default mapping from files to executables. For the first iteration you can probably ignore ambiguities (if a file belongs to multiple CMake targets).

Recompiling should be the easier part as there's less logic to do; once you know the executable you're going to run, you can unconditionally recompile the relevant target. CMake does incremental compilation for you so even if no changes occurred this is quite fast.

If your tool is Bazel or Mason or anything else, I'm afraid I can't help much. I assume they have some kind of API, but I don't know of lua wrappers for them.

Lately it's very hard to find the time for this, so if you have the time and the desire to work on this I'd be more than happy to review a PR.

[alfaix]

I only build tests for the parts of the code I am working on. With the current version of the repo, the test summary will be populated with hundreds of tests, most of which I do not care about. With it being so cluttered up, it's almost impossible to use. So it would be amazing, if only tests show up, where the corresponding executable is found.

This issue I'm not sure I fully understand; shouldn't tests from other folders be collapsed in the summary? Unless all your tests are in the same folder I think it should be easy to navigate, unless I'm missing something.

You can also change the root of your test tree (I think?..) through neotest, either way UI test filtering sounds like something neotest itself should do for you.

[ll-nick]

Sorry, forgot to mention that. Yes, CMake it is!

Thanks a lot for your response. Time is always hard to come by but I will see what I can do.

This issue I'm not sure I fully understand; shouldn't tests from other folders be collapsed in the summary? Unless all your tests are in the same folder I think it should be easy to navigate, unless I'm missing something.

You're right, there may be a way to filter some tests using neotest directly, I'll look into that.

For some more context: I work with ROS where you basically work with a bunch of CMake projects ("packages") in one so called workspace. Usually, you only work on a hand full of packages directly (hence are interested in the tests) but the workspaces I am working on might contain several hundred of these packages. So even if all folders are collapsed, these might still be in the triple-digits.

Setting some kind of test root directory might fix that issue though, thanks for the suggestion :)

[ll-nick]

Just wanted to give a brief update on this, maybe someone else might also be interested in moving this along. I'm still getting into the whole lua thing and it still feels like I have no clue what I am doing :sweat_smile: . Nevertheless, I did manage to get automatic test discovery working on my fork using this simple function in the ExecutablesRegistry class:

function ExecutablesRegistry:discover_cmake_executables()
  local model_info = cmake_tools.get_model_info()
  local build_dir = utils.normalize_path(tostring(cmake_tools.get_build_directory()))

  for _, target_info in pairs(model_info) do
    if target_info.type == "EXECUTABLE" then
      local executable_path_relative = target_info.artifacts[1].path
      local executable_path = build_dir .. lib.files.sep .. executable_path_relative
      for _, source in ipairs(target_info.sources) do
        if config.is_test_file(source.path) then
          local source_path = utils.normalize_path(source.path)
          self._node2executable[source_path] = executable_path
        end
      end
    end
  end
end

So I guess the next steps include making this an async function as you mentioned above and keeping up with your impressive test coverage.

[alfaix]

Hi @ll-nick , yes I think the process makes sense generally, little details can be figured out after the fact.

One potential source of trouble to look out for is the current cmake directory; likely some switching would have to take place to allow multiple contexts (i.e., something like old_root = cmake.get_directory(); cmake.set_directory(tests_root); do_stuff(); cmake.set_directory(old_root)). I remember I spent some time ironing that out before giving up 😅