Issue with debugging test

Agomezvi commented 10 months ago

Brief Issue Summary

Using the test explorer and selecting debugging a test does not work correctly. Regardless of the test you select, the test indicated in the cmake extension options is always executed.

launch.json

        {
            "name": "(gdb) Acds Unit WSL2",
            "type": "cppdbg",
            "request": "launch",
            "program": "${command:cmake.launchTargetPath}",
            "cwd": "${workspaceFolder}",
            "stopAtEntry": true,
            "externalConsole": false,
            "logging": {
               // "engineLogging": "verbose",
                "trace": true,
                "traceResponse": true
            },
            "environment": [
            {"name": "LD_LIBRARY_PATH", "value": "${input:sdkPath}/sysroots/x86-rcs-linux-runtime/usr/lib:/${input:sdkPath}/sysroots/x86-rcs-linux-runtime/lib"}
            ],
            "MIMode": "gdb",
            "setupCommands": [
                {
                    "text": "-gdb-set auto-load safe-path /opt/",
                    "description": "enable safe path",
                    "ignoreFailures": false
                },

            ],
        },

CMake Tools Diagnostics

No response

Debug Log

No response

Additional Information

No response

gcampbell-msft commented 10 months ago

@xisui-MSFT FYI.

xisui-MSFT commented 10 months ago

The reason why it's always running that test is "program": "${command:cmake.launchTargetPath}",. Please follow https://github.com/microsoft/vscode-cmake-tools/blob/main/docs/debug-launch.md#debugging-tests

Agomezvi commented 10 months ago

The reason why it's always running that test is "program": "${command:cmake.launchTargetPath}",. Please follow https://github.com/microsoft/vscode-cmake-tools/blob/main/docs/debug-launch.md#debugging-tests

@xisui-MSFT Great, this works. But it ignores the "stopAtEntry": true

borjamunozf commented 10 months ago

I have faced this error also. Seems that the suggested configuration helps regarding the test selection, but it's breaking or impacting two important aspects:

stopAtEntry does not work.
No breakpoints are reached, says that modules could not be loaded for debugging. So actually the Debug Launch it's not able to debug anything.

Any idea?

The reason why it's always running that test is "program": "${command:cmake.launchTargetPath}",. Please follow https://github.com/microsoft/vscode-cmake-tools/blob/main/docs/debug-launch.md#debugging-tests

xisui-MSFT commented 10 months ago

@Agomezvi @borjamunozf I wasn't able to reproduce the stopAtEntry or module could not be loaded issues. It would be great if you can provide a simple sample, so we can do further investigations.

borjamunozf commented 9 months ago

Sadly we cannot reproduce the error with simple projects, but only in our company ones (all of them). Not sure how we could try to show this as they're private codebase.

Any chance how we could try to share useful info for you to help to fix this?

xisui-MSFT commented 9 months ago

@borjamunozf Again, this is likely because the debugger can't load the module. I can give you some (limited) suggestions

Are you running test against release build or debug build?
Do you have a working launch config that can debug your product correctly? And what's the difference between it and the config used for tests?
Enable MIEngine trace log by adding "logging": {"engineLogging": true} to your launch.json, and then check the debug console for module load details, and figure out why modules are not loaded.

gcampbell-msft commented 7 months ago

@Agomezvi @borjamunozf Is it possible that these issues are now fixed in the pre-release channel?

borjamunozf commented 7 months ago

Not for me.

Same build / target - project, just changing the configuration

Using this config it's able to stop at entry + stop at breakpoints (anyway the test selected from side panel is ignored like you said, we should use cmake.testProgram)

 {
      "name": "(gdb) whatever Launch Debug UnitTest",
      "type": "cppdbg",
      "request": "launch",
      "program": "${command:cmake.launchTargetPath}",
      "cwd": "${workspaceFolder}",
      "stopAtEntry": true,
      "externalConsole": false,
      "logging": {
        "trace": false
      },
         "environment": [
        {
          "name": "LD_LIBRARY_PATH",
          "value": "/lib/i386-linux-gnu:${input:whateverPath}/sysroots/x86-whate-inux-runtime/usr/lib:${input:sdkPath}/sysroots/x86-linux-runtime/lib"
        },
      ],
      "MIMode": "gdb",
      "setupCommands": [
        {
          "text": "-gdb-set auto-load safe-path /opt/",
          "description": "enable safe path",
          "ignoreFailures": false
        }
      ]
    },

Using this config throws same issue with pre-release. No stop at entry, no stop at any breakpoints

{
  "name": "(gdb) Launch Debug UnitTest",
  "type": "cppdbg",
  "request": "launch",
  "program": "${cmake.testProgram}",
  "cwd": "${cmake.testWorkingDirectory}",
  "args": [ "${cmake.testArgs}"],
  "stopAtEntry": true,
  "externalConsole": false,
  "logging": {
    "trace": true,
    "engineLogging": true,
    "traceResponse": true,
  },
  "environment": [
    {
      "name": "LD_LIBRARY_PATH",
      "value": "/lib/i386-linux-gnu:${input:sdkPath}/sysroots/x86-whate-inux-runtime/usr/lib:${input:sdkPath}/sysroots/x86-linux-runtime/lib"
    },

  ],
  "miDebuggerPath": "/usr/bin/gdb",
  "MIMode": "gdb",
  "setupCommands": [
    {
      "text": "-gdb-set auto-load safe-path /",
      "description": "enable safe path",
      "ignoreFailures": false
    }
  ]
}

Checking the logs we could see that the file running is cmake pip package.

--> E (output): {"type":"event","event":"output","body":{"category":"console","output":"1: (424) <-1005-**gdb-set solib-search-path /usr/local/lib/python3.10/dist-packages/cmake/data/bin**:\n"},"seq":72}
1: (424) <-1005-gdb-set solib-search-path /usr/local/lib/python3.10/dist-packages/cmake/data/bin:
--> E (output): {"type":"event","event":"output","body":{"category":"console","output":"1: (432) <-1008-file-exec-and-symbols /usr/local/lib/python3.10/dist-packages/cmake/data/bin/cmake\n"},"seq":96}

More debug output launch options (replaced some info for privacy (it's real project from my company)


1: (102) LaunchOptions{"name":"(gdb) Launch Debug UnitTest","type":"cppdbg","request":"launch","program":"/usr/local/lib/python3.10/dist-packages/cmake/data/bin/cmake",
"cwd":"/home/borjamf/workspace/whatever/build/whatever/Debug/whatever/Lib/whateverMath/UnitTests",
"args":["-DTARGET=whateverMathUnitTests","-DBUILD_TYPE=Debug","-DBINDIR=/home/borjamf/workspace/whateverk/build/whatever/Debug","
-DRESULTS_FILE_DIR=/home/borjamf/workspace/whatever/build/whatever/Debug/whatever/Lib/whateverMath/UnitTests",
"-DTARGET_FILE_NAME=WhateverMathUnitTests","-DWORKING_DIRECTORY=/home/borjamf/workspace/whatever/build/whatever/Debug/whatever/Lib/whateverath/UnitTests",
"-P","/home/borjamf/workspace/whatever_ok/build/whatever/Debug/whatever/Lib/whateverMath/UnitTests/runtest.cmake"],
"stopAtEntry":true,
"externalConsole":false,
"logging":{"trace":true,"engineLogging":true,"traceResponse":true},
"environment":[{"name":"LD_LIBRARY_PATH","value":"/lib/i386-linux-gnu:/WTHAEVER"}],"miDebuggerPath":"/usr/bin/gdb",
"MIMode":"gdb",
"setupCommands":[{"text":"-gdb-set auto-load safe-path /","description":"enable safe path","ignoreFailures":false}],"ctest.magic.key":4,"__sessionId":"7107ea78-b8b7-40a9-a1a8-58d25e653b06"}

gcampbell-msft commented 7 months ago

@borjamunozf Thank you for the update.

@xisui-MSFT Have you seen this before and maybe have any context?

xisui-MSFT commented 7 months ago

@borjamunozf Thanks for this info! Since you have a working config, are you able to replace the fields one by one, so we can see which one causes this issue? Please start with "program": "${cmake.testProgram}",, and select the test from the test explorer.

Agomezvi commented 6 months ago

@xisui-MSFT I have tried the following configuration:

{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "name": "(gdb) Launch Debug UnitTest",
            "type": "cppdbg",
            "request": "launch",
            "program": "${cmake.testProgram}",
            "cwd": "${cmake.testWorkingDirectory}",
            "args": [ "${cmake.testArgs}"],
            "environment": [
              {
                "name": "LD_LIBRARY_PATH",
                "value": "/lib/i386-linux-gnu:${input:sdkPath}/sysroots/x86-whate-inux-runtime/usr/lib:${input:sdkPath}/sysroots/x86-linux-runtime/lib"
              },

            ],
            "miDebuggerPath": "/usr/bin/gdb",
            "MIMode": "gdb",
          }
    ]
}

No stop at any breakpoints.

The problem is that we do not have a valid configuration that works for us.

xisui-MSFT commented 6 months ago

@Agomezvi Can you check @borjamunozf 's reply, and try to get a working launch config, then replace the fields one by one with cmake.test* to see which config caused this issue? (And please let us know which field it is...)

Agomezvi commented 6 months ago

@xisui-MSFT I think I do not understand you. Borja does not have a configuration that works either. Only the configuration that always launches the same test works "program": "${command:cmake.launchTargetPath}",. Using "program": "${cmake.testProgram}", I have not been able to have a configuration that works

xisui-MSFT commented 6 months ago

@Agomezvi So does "program": "${command:cmake.launchTargetPath}" work for you?

If so, does replacing it with "program": "${cmake.testProgram}" causes the issue?

If you are sure that's the cause, can you add "logging": {engineLogging": true to your launch config, and then check the debug console to see if these two cases have any different args?

Agomezvi commented 6 months ago

@xisui-MSFT Using "program": "${command:cmake.launchTargetPath}" it's able to stop at entry + stop at breakpoints but the test selected from side panel is ignored. Always run the same test, regardless of which one you select. For this reason, I do not have any valid configuration.

xisui-MSFT commented 6 months ago

stop at breakpoints but the test selected from side panel is ignored. Always run the same test, regardless of which one you select.

This is OK. As long as there's a working launch config for a single test, you can proceed the checks (replacing with "program": "${cmake.testProgram}", and select the same test).

The purpose now is to narrow it down to (hopefully) one launch config arg that causes this problem.

Agomezvi commented 6 months ago

Not for me.

Same build / target - project, just changing the configuration

Using this config it's able to stop at entry + stop at breakpoints (anyway the test selected from side panel is ignored like you said, we should use cmake.testProgram)

 {
      "name": "(gdb) whatever Launch Debug UnitTest",
      "type": "cppdbg",
      "request": "launch",
      "program": "${command:cmake.launchTargetPath}",
      "cwd": "${workspaceFolder}",
      "stopAtEntry": true,
      "externalConsole": false,
      "logging": {
        "trace": false
      },
         "environment": [
        {
          "name": "LD_LIBRARY_PATH",
          "value": "/lib/i386-linux-gnu:${input:whateverPath}/sysroots/x86-whate-inux-runtime/usr/lib:${input:sdkPath}/sysroots/x86-linux-runtime/lib"
        },
      ],
      "MIMode": "gdb",
      "setupCommands": [
        {
          "text": "-gdb-set auto-load safe-path /opt/",
          "description": "enable safe path",
          "ignoreFailures": false
        }
      ]
    },

Using this config throws same issue with pre-release. No stop at entry, no stop at any breakpoints

   {
      "name": "(gdb) Launch Debug UnitTest",
      "type": "cppdbg",
      "request": "launch",
      "program": "${cmake.testProgram}",
      "cwd": "${cmake.testWorkingDirectory}",
      "args": [ "${cmake.testArgs}"],
      "stopAtEntry": true,
      "externalConsole": false,
      "logging": {
        "trace": true,
        "engineLogging": true,
        "traceResponse": true,
      },
      "environment": [
        {
          "name": "LD_LIBRARY_PATH",
          "value": "/lib/i386-linux-gnu:${input:sdkPath}/sysroots/x86-whate-inux-runtime/usr/lib:${input:sdkPath}/sysroots/x86-linux-runtime/lib"
        },

      ],
      "miDebuggerPath": "/usr/bin/gdb",
      "MIMode": "gdb",
      "setupCommands": [
        {
          "text": "-gdb-set auto-load safe-path /",
          "description": "enable safe path",
          "ignoreFailures": false
        }
      ]
    }

Checking the logs we could see that the file running is cmake pip package.

--> E (output): {"type":"event","event":"output","body":{"category":"console","output":"1: (424) <-1005-**gdb-set solib-search-path /usr/local/lib/python3.10/dist-packages/cmake/data/bin**:\n"},"seq":72}
1: (424) <-1005-gdb-set solib-search-path /usr/local/lib/python3.10/dist-packages/cmake/data/bin:
--> E (output): {"type":"event","event":"output","body":{"category":"console","output":"1: (432) <-1008-file-exec-and-symbols /usr/local/lib/python3.10/dist-packages/cmake/data/bin/cmake\n"},"seq":96}

More debug output launch options (replaced some info for privacy (it's real project from my company)

1: (102) LaunchOptions{"name":"(gdb) Launch Debug UnitTest","type":"cppdbg","request":"launch","program":"/usr/local/lib/python3.10/dist-packages/cmake/data/bin/cmake",
"cwd":"/home/borjamf/workspace/whatever/build/whatever/Debug/whatever/Lib/whateverMath/UnitTests",
"args":["-DTARGET=whateverMathUnitTests","-DBUILD_TYPE=Debug","-DBINDIR=/home/borjamf/workspace/whateverk/build/whatever/Debug","
-DRESULTS_FILE_DIR=/home/borjamf/workspace/whatever/build/whatever/Debug/whatever/Lib/whateverMath/UnitTests",
"-DTARGET_FILE_NAME=WhateverMathUnitTests","-DWORKING_DIRECTORY=/home/borjamf/workspace/whatever/build/whatever/Debug/whatever/Lib/whateverath/UnitTests",
"-P","/home/borjamf/workspace/whatever_ok/build/whatever/Debug/whatever/Lib/whateverMath/UnitTests/runtest.cmake"],
"stopAtEntry":true,
"externalConsole":false,
"logging":{"trace":true,"engineLogging":true,"traceResponse":true},
"environment":[{"name":"LD_LIBRARY_PATH","value":"/lib/i386-linux-gnu:/WTHAEVER"}],"miDebuggerPath":"/usr/bin/gdb",
"MIMode":"gdb",
"setupCommands":[{"text":"-gdb-set auto-load safe-path /","description":"enable safe path","ignoreFailures":false}],"ctest.magic.key":4,"__sessionId":"7107ea78-b8b7-40a9-a1a8-58d25e653b06"}

@xisui-MSFT This is the test that @borjamunozf did. I have tried it with the same result. I still think we don't understand each other. I do not have a valid initial configuration in which a test selected from the test explorer works correctly

xisui-MSFT commented 6 months ago

@Agomezvi Can you also provide the launch options in the debug output for launch config "(gdb) whatever Launch Debug UnitTest"?

Yingzi1234 commented 6 months ago

@Agomezvi Is your problem currently reproducing? If it does, could you please provide the launch options in the debug output for launch config "(gdb) whatever Launch Debug UnitTest"? Thanks!

Agomezvi commented 6 months ago

Hi @xisui-MSFT @Yingzi1234, sorry for waiting.

Using this config throws same issue. No stop at entry, no stop at any breakpoints:

{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "name": "(gdb) Launch Debug UnitTest",
            "type": "cppdbg",
            "request": "launch",
            "program": "${cmake.testProgram}",
            "cwd": "${cmake.testWorkingDirectory}",
            "args": [ "${cmake.testArgs}"],
            "stopAtEntry": true,
            "externalConsole": false,
            "logging": {
              "trace": true,
              "engineLogging": true,
              "traceResponse": true
            },
            "environment": [
              {
                "name": "LD_LIBRARY_PATH",
                "value": "/lib/i386-linux-gnu:${input:sdkPath}/sysroots/x86-whate-inux-runtime/usr/lib:${input:sdkPath}/sysroots/x86-linux-runtime/lib"
              },

            ],
            "miDebuggerPath": "/usr/bin/gdb",
            "MIMode": "gdb",
          }
    ]
}

I have attached the DebugConsole output in the message. DebugConsole.txt

xisui-MSFT commented 6 months ago

@Agomezvi Can you also share the output of the config you mentioned that works for a single test? I think it's this one:

   {
      "name": "(gdb) whatever Launch Debug UnitTest",
      "type": "cppdbg",
      "request": "launch",
      "program": "${command:cmake.launchTargetPath}",
      "cwd": "${workspaceFolder}",
      "stopAtEntry": true,
      "externalConsole": false,
      "logging": {
        "trace": false
      },
         "environment": [
        {
          "name": "LD_LIBRARY_PATH",
          "value": "/lib/i386-linux-gnu:${input:whateverPath}/sysroots/x86-whate-inux-runtime/usr/lib:${input:sdkPath}/sysroots/x86-linux-runtime/lib"
        },
      ],
      "MIMode": "gdb",
      "setupCommands": [
        {
          "text": "-gdb-set auto-load safe-path /opt/",
          "description": "enable safe path",
          "ignoreFailures": false
        }
      ]
    },

Agomezvi commented 6 months ago

@xisui-MSFT Of course here is: DebugConsole_SingleTest.txt

xisui-MSFT commented 6 months ago

@gcampbell-msft cmake.testProgram resolves to cmake for some reason. Can you continue the investigation based on this?

gcampbell-msft commented 4 months ago

@Agomezvi Does this issue reproduce still in the most recent release of the product? This PR #3631 claims to have fixed it and it is now released in the official release channel.

Thanks!

Agomezvi commented 4 months ago

@gcampbell-msft Yes, it does not solve it

jophippe commented 4 months ago

@Agomezvi Unfortunately, I am unable to repro this issue with cmake.testProgram on my machine. Here's a few things to try:

In the build output directory, run the command "ctest --show-only=json-v1", assuming you have CMake version 3.14 or greater. Make sure that the "command" property in each of the tests are the correct command, since that is what we should be using to replace the cmake.testProgram variable.
Just in case we're storing something wrong, try running the "CMake: Reset CMake Tools Extension State" command to reset everything, then try deleting the cache and reconfiguring and see if that allows you to debug using cmake.testProgram.

Let me know if one of these helps you out. Thanks!

Agomezvi commented 3 months ago

@jophippe Thanks for your input, but it still doesn't work. I think the test properties are correct, anyway I did the “CMake: Reset CMake Tools Extension State” command, but I got the same result.

I attach a file with the properties of one of the tests.

TestProperty.txt

jophippe commented 3 months ago

Thanks for the test properties @Agomezvi! I think I understand what's going on here. It looks like CTest is executing a CMake script to actually run the test. That's why the first entry in the command array is the path to CMake, and the last two entries are "-P" and the path to "runtest.cmake". I assume that's just a consequence of how your codebase is set up to run tests?

I'm not sure exactly how we would support test debugging in this scenario. The debug configuration relies on the program entry being the actual test program so it knows which symbols to load, and I can't think of a way to populate that with a macro when using this CMake script layer. The only workaround I can think of is to have a different debug config for every test so you could manually populate the program and args, but obviously that would only be viable if your testing suite is extremely small.

@xisui-MSFT do you have any other ideas for supporting test debugging for this scenario?

xisui-MSFT commented 3 months ago

Thanks Jonathan! I don't have a very good idea either. The best way I can think of is probably to re-write the script into launch configs or tests themselves?

jophippe commented 3 months ago

I'm going to mark this as an enhancement and move it to the backlog. If this issue gets more activity, we'll consider our options.

microsoft / vscode-cmake-tools