Get path to sdk from dotnet cli

[molostovvs]

Test discovery didn't work on my machine (Linux) because the path to the dotnet sdk is hardcoded as /usr/lib/dotnet/sdk here https://github.com/GustavEikaas/easy-dotnet.nvim/blob/942467f4b2148bd9c924dee8dba2867980f9ea25/lua/easy-dotnet/options.lua#L11 but in my case the sdk was installed in /usr/share/dotnet/sdk.

You can get the path to the sdk using the dotnet --info command, or dotnet --list-sdks. Here is the output of dotnet --list-sdks on my machine:

8.0.108 [/usr/share/dotnet/sdk]
8.0.302 [/usr/share/dotnet/sdk]

[GustavEikaas]

Yeah I guess this was eventually going to happen, im not sure how to best handle this. Performance wise its best for users to always set the get_sdk_path in their options but I do not want mandatory config properties. I could obviously run dotnet --info but there is a performance cost to this. I could of course cache the result in a file in the hopes that it will resolve the performance issues but its not an ideal solution.

Any input would be great! :D

[molostovvs]

Just going to leave here a temporary solution that worked for me on Linux:

sudo ln -s /usr/share/dotnet /usr/lib/dotnet

btw, it didn't solve the problem with discovering all tests. In a small solution with ~1k tests Dotnet testrunner found only one test project with 20 tests.

[GustavEikaas]

Creative fix but you can also do like this.

 dotnet.setup({
...
      get_sdk_path = function()
        return "/usr/share/dotnet"
      end,
...
    })

Help me understand more in terms of test discovery, if I understand correctly there are multiple test projects of which the plugin only found one containing 20 tests. It would help if you could provide more information about the .csproj files of the test projects the plugin failed to find.

Below is the code for determining if a project is a testproject

M.is_test_project = function(project_file_path)
  if type(extract_from_project(project_file_path, '<%s*IsTestProject%s*>%s*true%s*</%s*IsTestProject%s*>')) == "string" then
    return true
  end
  if type(extract_from_project(project_file_path, '<PackageReference Include="Microsoft%.NET%.Test%.Sdk" Version=".-" />')) == "string" then
    return true
  end
  if type(extract_from_project(project_file_path, '<PackageReference Include="MSTest%.TestFramework" Version=".-" />')) == "string" then
    return true
  end
  if type(extract_from_project(project_file_path, '<PackageReference Include="NUnit" Version=".-" />')) == "string" then
    return true
  end
  if type(extract_from_project(project_file_path, '<PackageReference Include="xunit" Version=".-" />')) == "string" then
    return true
  end

  return false
end

[molostovvs]

The only project that showed up as a test project had <IsTestProject>true</IsTestProject> in its .csproj. What about the others - I think the problem is that your code implies that the version of the package is specified in the .csproj file. But in our solution we use the central package management system, so in the .csproj file there is only this line: <PackageReference Include=“xunit” />, and the version is set in Directory.Packages.props.

csproj of successfully discovered project:

<Project Sdk="Microsoft.NET.Sdk">

    <PropertyGroup>
        <ImplicitUsings>enable</ImplicitUsings>
        <Nullable>enable</Nullable>

        <IsPackable>false</IsPackable>
        <IsTestProject>true</IsTestProject>
    </PropertyGroup>

    <ItemGroup>
        <PackageReference Include="FluentAssertions" />
        <PackageReference Include="Microsoft.NET.Test.Sdk" />
        <PackageReference Include="NSubstitute" />
        <PackageReference Include="xunit" />
        <PackageReference Include="xunit.runner.visualstudio">
            <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
            <PrivateAssets>all</PrivateAssets>
        </PackageReference>
        <PackageReference Include="coverlet.collector">
            <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
            <PrivateAssets>all</PrivateAssets>
        </PackageReference>
    </ItemGroup>

    <ItemGroup>
      <ProjectReference Include="..\..\src\Shared.DistributedCache\Shared.DistributedCache.csproj" />
    </ItemGroup>

</Project>

an example of a csproj that was not discovered:

<?xml version="1.0" encoding="utf-8"?>
<Project Sdk="Microsoft.NET.Sdk">
  <ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" />
    <PackageReference Include="NSubstitute" />
    <PackageReference Include="xunit" />
    <PackageReference Include="xunit.runner.visualstudio">
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
      <PrivateAssets>all</PrivateAssets>
    </PackageReference>
    <PackageReference Include="coverlet.collector">
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
      <PrivateAssets>all</PrivateAssets>
    </PackageReference>
  </ItemGroup>
  <ItemGroup>
    <ProjectReference Include="..\..\src\Processing.Application\Processing.Application.csproj" />
  </ItemGroup>
</Project>

upd: after I copied property <IsTestProject>true</IsTestProject> into another project, it was also discovered, but I think we need to update the rules by which test projects are discovered.

[GustavEikaas]

Will be fixed in #147, would you mind testing it? I tried handling both without and without Version in PackageReference

[molostovvs]

It works great! I checked the number of tests this way:

1. Dotnet testrunner
2. E
3. search by flask pattern (g/\)

I got:

1. A small solution, Rider reports 1059 tests, my method found 1298 tests
2. big solution, Rider reports 27283 tests, my method found 31065 tests

In this case, each case at a parameterized test also counts as a separate test, but I got pretty realistic results. Is there any way to count only the test methods, without counting the parameterized cases?

[GustavEikaas]

Ah I see, I updated the logic now. the counts should be the same as rider now

[GustavEikaas]

Visual studio counts parameterized cases btw

[molostovvs]

Yes, for some reason I was sure that rider counts only test methods, without parameterized cases, sorry. I got a difference on a small solution of 1059(rider)-978(my method in neovim)=81.

now I counted the number of tests in neovim with this formula: echo count(join(getbufline('%', 1, '$'), ''), '')

It remains to find why some tests were not counted or found.

[molostovvs]

I found these weird parameterized tests that show up in Rider but not in neovim.

Another example:

[Theory]
[MemberData(nameof(ValidationTestCases))]
public async Task Parse_WrongInput_ValidationWithCorrectPath(object input, string validationErrorWildcard)

upd: when I try to run such a test, it gets stuck in \ status, although other tests run and pass.

[GustavEikaas]

I have had some issues with MemberData,ClassData and InlineData attributes in the past.

But for your discovery issue, try pulling latest from the branch and run the discovery in your :messages you should see multiple paths show up, run cat <file> on these and see if it contains the same amount of tests as rider

[molostovvs]

cat /tmp/lua_Wqzt0C | grep Parse_WrongInput_ValidationWithCorrectPath | jq

results in

{
  "Id": "25efb9b5-9299-a3ec-be43-6eaef7ce211a",
  "Namespace": "Processing.Contracts.UnitTests.RetailOrderLineInputDtoParserTests.Parse_WrongInput_ValidationWithCorrectPath",
  "Name": "Processing.Contracts.UnitTests.RetailOrderLineInputDtoParserTests.Parse_WrongInput_ValidationWithCorrectPath",
  "FilePath": "/home/mvs/source/work/processing-orders/tests/Processing.Contracts.IntegrationTests/Parsers/RetailOrderLineInputDtoParserTests.cs",
  "Linenumber": 147
}

although a simple parametrized test turns out like this:

{
  "Id": "2208e58f-9a19-ee85-a965-027ac51bee02",
  "Namespace": "Processing.Application.UnitTests.CompositeProcessingPromotionCriterionTests.IsAcceptable_Theory",
  "Name": "Processing.Application.UnitTests.CompositeProcessingPromotionCriterionTests.IsAcceptable_Theory(innerCriteriaResults: [True], expectedResult: True)",
  "FilePath": "/home/mvs/source/work/processing-orders/tests/Processing.Application.UnitTests/Infrastructure/PromotionFiltration/Variants/DayOfWeekProcessingPromotionCriterionTests.cs",
  "Linenumber": 118
}
{
  "Id": "a777a547-f1a8-7289-5116-426f41a06c7f",
  "Namespace": "Processing.Application.UnitTests.CompositeProcessingPromotionCriterionTests.IsAcceptable_Theory",
  "Name": "Processing.Application.UnitTests.CompositeProcessingPromotionCriterionTests.IsAcceptable_Theory(innerCriteriaResults: [False], expectedResult: False)",
  "FilePath": "/home/mvs/source/work/processing-orders/tests/Processing.Application.UnitTests/Infrastructure/PromotionFiltration/Variants/DayOfWeekProcessingPromotionCriterionTests.cs",
  "Linenumber": 118
}
{
  "Id": "c309909f-c0b0-57c5-498b-a2d0f32732cc",
  "Namespace": "Processing.Application.UnitTests.CompositeProcessingPromotionCriterionTests.IsAcceptable_Theory",
  "Name": "Processing.Application.UnitTests.CompositeProcessingPromotionCriterionTests.IsAcceptable_Theory(innerCriteriaResults: [True, False], expectedResult: False)",
  "FilePath": "/home/mvs/source/work/processing-orders/tests/Processing.Application.UnitTests/Infrastructure/PromotionFiltration/Variants/DayOfWeekProcessingPromotionCriterionTests.cs",
  "Linenumber": 118
}

note: these files are deleted after parsing, so I needed to comment out these lines as well: https://github.com/GustavEikaas/easy-dotnet.nvim/blob/a9434b667c597c608d63f4815765a46f888ea558/lua/easy-dotnet/test-runner/runner.lua?plain=1#L200

[GustavEikaas]

Weird I created a similiar testclass like this

using System.Collections.Generic;
using System.Threading.Tasks;
using Xunit;

namespace NeovimDebugProject.ClassData
{

  public class YourTestClass
  {
    // This is the data provider, returning 3 test cases.
    public static IEnumerable<object[]> ValidationTestCases =>
        new List<object[]>
        {
            // Case 1: Invalid input and corresponding validation error
            new object[] { "input_case_1", "Error for case 1" },

            // Case 2: Another invalid input and corresponding validation error
            new object[] { "input_case_2", "Error for case 2" },

            // Case 3: Yet another invalid input and corresponding validation error
            new object[] { "input_case_3", "Error for case 3" }
        };

    // This is the test method that will be run with each set of data from ValidationTestCases.
    [Theory]
    [MemberData(nameof(ValidationTestCases))]
    public async Task Parse_WrongInput_ValidationWithCorrectPath(object input, string validationErrorWildcard)
    {
      // Arrange: Set up any necessary conditions for your test.
      // For example, if you are calling a method named "Parse" that processes the input.

      // Act: Call the method you are testing.
      var result = await Parse(input);

      // Assert: Verify the result meets your expectations.
      // This could involve checking the validation error matches the expected wildcard.
      Assert.Contains(validationErrorWildcard, result.ValidationError);
    }

    // Dummy implementation of Parse method for demonstration purposes.
    // Replace this with your actual logic.
    private Task<(string ValidationError, int StatusCode)> Parse(object input)
    {
      // Example logic for generating a validation error based on input
      string validationError = $"Error for {input}";

      // Example status code, could be 400 for validation errors
      int statusCode = 400;

      // Return both the validation error and the status code as a tuple
      return Task.FromResult((validationError, statusCode));
    }
  }
}

This is the output from my tmp file

{
  "Id": "ea7dafa7-1951-c325-9bf4-c9d2f1aeacb9",
  "Namespace": "NeovimDebugProject.ClassData.YourTestClass.Parse_WrongInput_ValidationWithCorrectPath",
  "Name": "NeovimDebugProject.ClassData.YourTestClass.Parse_WrongInput_ValidationWithCorrectPath(input: \"input_case_1\",
 validationErrorWildcard: \"Error for case 1\")",
  "FilePath": "C:\\Users\\Gustav\\repo\\NeovimDebugProject\\src\\NeovimDebugProject.ClassData\\MemberData.cs",
  "Linenumber": 28
}
{
  "Id": "0144cf93-900d-85e8-afe7-7939138810b9",
  "Namespace": "NeovimDebugProject.ClassData.YourTestClass.Parse_WrongInput_ValidationWithCorrectPath",
  "Name": "NeovimDebugProject.ClassData.YourTestClass.Parse_WrongInput_ValidationWithCorrectPath(input: \"input_case_2\",
 validationErrorWildcard: \"Error for case 2\")",
  "FilePath": "C:\\Users\\Gustav\\repo\\NeovimDebugProject\\src\\NeovimDebugProject.ClassData\\MemberData.cs",
  "Linenumber": 28
}
{
  "Id": "3180559c-a31e-9c1d-8150-9cc630fc766c",
  "Namespace": "NeovimDebugProject.ClassData.YourTestClass.Parse_WrongInput_ValidationWithCorrectPath",
  "Name": "NeovimDebugProject.ClassData.YourTestClass.Parse_WrongInput_ValidationWithCorrectPath(input: \"input_case_3\",
 validationErrorWildcard: \"Error for case 3\")",
  "FilePath": "C:\\Users\\Gustav\\repo\\NeovimDebugProject\\src\\NeovimDebugProject.ClassData\\MemberData.cs",
  "Linenumber": 28
}
{
  "Id": "62bdc21d-8091-5b84-fa77-6f35195cce0d",
  "Namespace": "NeovimDebugProject.ClassData.MathTests.CanAddNumberss",
  "Name": "NeovimDebugProject.ClassData.MathTests.CanAddNumberss(value1: 1, type: null)",
  "FilePath": "C:\\Users\\Gustav\\repo\\NeovimDebugProject\\src\\NeovimDebugProject.ClassData\\UnitTest1.cs",
  "Linenumber": 28
}
{
  "Id": "8515d005-5dcb-6c99-ea5f-4cb7d0cd8ca4",
  "Namespace": "NeovimDebugProject.ClassData.MathTests.CanAddNumberss",
  "Name": "NeovimDebugProject.ClassData.MathTests.CanAddNumberss(value1: 4, type: typeof(System.Exception))",       
  "FilePath": "C:\\Users\\Gustav\\repo\\NeovimDebugProject\\src\\NeovimDebugProject.ClassData\\UnitTest1.cs",       
  "Linenumber": 28
}
{
  "Id": "56ed99dc-d0c9-b453-9abf-cf89dfc44117",
  "Namespace": "NeovimDebugProject.ClassData.MathTests.CanAddNumberss",
  "Name": "NeovimDebugProject.ClassData.MathTests.CanAddNumberss(value1: 4, type: typeof(NeovimDebugProject.ClassData.Exc
eptions.CustomException))",
  "FilePath": "C:\\Users\\Gustav\\repo\\NeovimDebugProject\\src\\NeovimDebugProject.ClassData\\UnitTest1.cs",
  "Linenumber": 28
}
{
  "Id": "340ccebf-235e-0de0-fc11-4a63b4528e90",
  "Namespace": "NeovimDebugProject.ClassData.MathTests.Check",
  "Name": "NeovimDebugProject.ClassData.MathTests.Check",
  "FilePath": "C:\\Users\\Gustav\\repo\\NeovimDebugProject\\src\\NeovimDebugProject.ClassData\\UnitTest1.cs",
  "Linenumber": 43
}

And the tests are running and parsed correctly

[molostovvs]

I tested with the simple dotnet test -t command, and the results showed only one test case for the Parse_WrongInput_ValidationWithCorrectPath, so i think it's a dotnet issue.

[GustavEikaas]

Yeah I have had some issues with that previously. Specifically with vstest. I started using TranslationLayer to circumvent the issue.

Fixed in #74

[GustavEikaas]

Can we close this issue and possibly open new ones for specific problems you may still have?

Im hoping your original issue with sdk path is solved, either by symlink or by defining the get_sdk_path in options?

[molostovvs]

Yes, of course. I'll look at the case of the problematic test method a bit later, at a glance it's not clear to me why it's not detecting normally.

[GustavEikaas]

A repro would be awesome, I have struggled a lot with these types of cases in previous issues

[NWVi]

Hi,

On Linux and using the dotnet-install script the dotnet SDKs defaults to being installed under ~/.dotnet/sdk/[version]. This causes the default get_sdk_path function to return an invalid path since /usr/lib/dotnet/sdk/[version] does not exist.

I've made a workaround for this in my setup function, but thought I'd share it in case others run into the same issue. The function was copied and modified based on the original get_sdk_path function adding a call to dotnet --list-sdks and finding the correct sdk base path from the result.

get_sdk_path = function()
  local sdk_version = vim.trim(vim.system({ "dotnet", "--version" }):wait().stdout)
  local sdk_list = vim.trim(vim.system({ "dotnet", "--list-sdks" }):wait().stdout)
  local base = nil
  for line in sdk_list:gmatch("[^\n]+") do
    if line:find(sdk_version, 1, true) then
      base = line:match("%[(.-)%]")
      break
    end
  end
  local sdk_path = vim.fs.joinpath(base, sdk_version)
  return sdk_path
end,

If wanted I can create a PR for this change.

[GustavEikaas]

@NWVi Awesome! That would be very nice! I would love a PR on this ♥