search

bneumann / CppUTest-Test-Adapter

A test adapter for Visual Studio Code for the C/C++ Unit Test framework CppUTest.
MIT License
16 stars 7 forks source link

VSCode integration #1

Closed chuchaki closed 2 years ago

chuchaki commented 4 years ago

Hello, I was very happy to see that there is a cppu plugin for better visualization. I've installed it using the extension manager and added the following lines in the settings file:

"cpputestExplorer.testExecutable": "${workspaceFolder}/Software/test/main",
"cpputestExplorer.testExecutablePath": "${workspaceFolder}/Software/",

Where main is the executable file that shows the passed/failed etc. output.

When I switch to the test explorer view, nothin appears. The "reloading tests" icon keeps revolving.

The Log File states:

[2020-02-02 09:58:19.300] [INFO] Test Explorer found [2020-02-02 09:58:19.301] [INFO] Creating adapter for "PathToProjectRoot" [2020-02-02 09:58:19.301] [INFO] Initializing adapter [2020-02-02 09:58:19.301] [INFO] Registering adapter for "PathToProjectRoot" [2020-02-02 09:58:19.302] [INFO] Loading tests

Any Idea?

Thanks & with best regards

bneumann commented 4 years ago

Hi, thanks for the issue. Your settings look good so far. I presume you are using Linux? Can you post the output of the main executable if you run it on the command line?

Which version of cpputest are you using?

I will try to reproduce it on my side, maybe I have overseen something in the Linux case.

bneumann commented 4 years ago

Hi @chuchaki . So I fiddled a bit with my test project and found that if I use the ${workspaceFolder} variable it doesn't work. Then I did some digging why it is not resolved from the settings and came across this issue here: https://github.com/microsoft/vscode/issues/46471 So it seems to be a feature request for VSCode and it's not yet fixed. I will implement a small internal replacement API for at least the workspacefolder variable.

bneumann commented 4 years ago

Hi @chuchaki, if you be so kind and see if the new version 0.1.2 works for you now.

chuchaki commented 4 years ago

Hey,

thanks for the fast reply. Sadly no changes.

Here the console output from the test executable:


.............
OK (13 tests, 13 ran, 40 checks, 0 ignored, 0 filtered out, 0 ms)

Here the log from the Extension host

[2020-02-02 15:05:28.263] [exthost] [info] eager extensions activated
[2020-02-02 15:10:01.726] [exthost] [info] ExtensionService#_doActivateExtension bneumann.cpputest-test-adapter {"startup":false,"extensionId":{"value":"bneumann.cpputest-test-adapter","_lower":"bneumann.cpputest-test-adapter"},"activationEvent":"*"}
[2020-02-02 15:10:01.727] [exthost] [info] ExtensionService#loadCommonJSModule file:///home/till/.vscode/extensions/bneumann.cpputest-test-adapter-0.1.2/out/main.js
[2020-02-02 15:11:00.328] [exthost] [info] ExtensionService#_doActivateExtension vscode.configuration-editing {"startup":false,"extensionId":{"value":"vscode.configuration-editing","_lower":"vscode.configuration-editing"},"activationEvent":"onLanguage:jsonc"}
[2020-02-02 15:11:00.328] [exthost] [info] ExtensionService#loadCommonJSModule file:///usr/share/code/resources/app/extensions/configuration-editing/dist/extension
[2020-02-02 15:11:00.333] [exthost] [info] ExtensionService#_doActivateExtension vscode.json-language-features {"startup":false,"extensionId":{"value":"vscode.json-language-features","_lower":"vscode.json-language-features"},"activationEvent":"onLanguage:jsonc"}
[2020-02-02 15:11:00.334] [exthost] [info] ExtensionService#loadCommonJSModule file:///usr/share/code/resources/app/extensions/json-language-features/client/dist/jsonMain
[2020-02-02 15:11:00.357] [exthost] [info] ExtensionService#_doActivateExtension vscode.typescript-language-features {"startup":false,"extensionId":{"value":"vscode.typescript-language-features","_lower":"vscode.typescript-language-features"},"activationEvent":"onLanguage:jsonc"}
[2020-02-02 15:11:00.357] [exthost] [info] ExtensionService#loadCommonJSModule file:///usr/share/code/resources/app/extensions/typescript-language-features/dist/extension
chuchaki commented 4 years ago

Ok, I think I found the issue. I cloned the current repository to use. So the current master branch. I checked out v3.8 and now its working.

My bad, thanks for the fast help :-)

bneumann commented 4 years ago

Now I am confused. Which version 3.8 are you referring to? I already updated the extension in the marketplace. Should be fixed there also

chuchaki commented 4 years ago

Hey, I was refering to the CppUTest v3.8 as tagged in their github repo. My workspace script did not do a proper checkout on that tag and thus I had a non-released version of CppUTest running (current master).

Using the correct version from CppUTest (tag v3.8) and the current version of the test Adapter ( 0.1.2 ), everything works

bneumann commented 4 years ago

Wonderful. But I think the issue was the missing variable resolution for extensions really. Anyways glad it works now and you are happy 😁 will close this now

chuchaki commented 4 years ago

Yes, very happ, thank you ;-).

chuchaki commented 4 years ago

Wonderful. But I think the issue was the missing variable resolution for extensions really. Anyways glad it works now and you are happy will close this now

Just to clarify: The Test Adapter ( 0.1.2 ), did not work with the current master (HEAD) of CppUTest. It worked, once i changed to the CppUTest release v3.8.

But thats not really important since support for a non released version cant be guaranteed as far as i see it ;-)

bika-c commented 3 years ago

Hi, I have the same issue here. I am using Windows. I tried the following:

"cpputestExplorer.testExecutable": "${workspaceFolder}/test/unitTest.exe",
"cpputestExplorer.testExecutablePath": "${workspaceFolder}/test/",

If I just run the unitTest.exe, it shows the proper output. The Test Explorer shows nothing. Really confused.

bneumann commented 3 years ago

Really confused.

Me too. Will have a look ASAP. From what @chuchaki wrote there might have been a change in the cpputest interface. Which version are you using?

bneumann commented 3 years ago

Can you also paste the output of unitTest.exe -ln here? That'd be a great help as it is the command I am using to get all the tests and classes

bika-c commented 3 years ago

I wrote a very simple function to see if a number is a prime. That is the output of unitTest.exe:

unitTest.c:24:test_IsPrime:PASS

-----------------------        
1 Tests 0 Failures 0 Ignored   
OK
bneumann commented 3 years ago

I would need the output of unitTest.exe -ln which returns a list of all tests not the test results themselves

bika-c commented 3 years ago

Ah, it shows the same thing. There is only one test and I did not write anything for passing arguments. Here is the source file if you are interested.

#include <unity.h>
#include <stdbool.h>

bool isPrime(int num)
{
    for (int i = 2; i <= (int)sqrt((float)num); i++)
        if (num % i == 0)
            return false;
    return true;
}

void test_IsPrime(void)
{

    TEST_ASSERT(isPrime(2));
}

void setUp() {}
void tearDown(){}

int main(void)
{
    UNITY_BEGIN();
    RUN_TEST(test_IsPrime);
    return UNITY_END();
}
bneumann commented 3 years ago

Oooooh. You are using the Unity Test Framework. This test adapter works for CppUTest.

There is a VSCode Extension for Unity. You can find it here: https://marketplace.visualstudio.com/items?itemName=fpopescu.vscode-unity-test-adapter

bika-c commented 3 years ago

Ohh! My bad. I am confused with them. I am gonna switch to the CppUtest🤣.

jdpcmt commented 3 years ago

I have the same issue with the plug-in version 0.1.8 and CppuTest v3.8 on VsCode 1.54.3 on linux. When executing the test with the -ln option, all the tests of the test suites are shown. I tried to use both the full path and relative to workspace path (using ${workspaceFolder}) for the exe and execution directories location.

The log files shows: [2021-03-22 10:25:15.974] [INFO] Test Explorer found [2021-03-22 10:25:15.975] [INFO] Creating adapter for /home/dev/workspace/t3platform [2021-03-22 10:25:15.975] [INFO] Initializing adapter [2021-03-22 10:25:15.975] [INFO] Registering adapter for /home/dev/workspace/t3platform [2021-03-22 10:25:15.976] [INFO] Loading tests [2021-03-22 10:25:15.993] [INFO] Initialization finished

Any idea about what might be wrong?

bneumann commented 3 years ago

Hey, thanks for the details. Can you double check if the output of testexe -ln looks something like this:

benni@Benni2:/mnt/c/Users/benni/Documents/CppUTest-Test-Adapter$ /mnt/c/Users/benni/Documents/CppUTest-Workspace/test/testrunner -ln
ClassName.Create ClassName.ShouldPass SecondClass.ShouldPass SecondClass.ShouldFail SecondClass.ShouldBeNew SecondClass.ShouldAlsoBeIgnored SecondClass.ShouldBeIgnored Specials.Crash Specials.AnotherStuff
jdpcmt commented 3 years ago

Hello,

The format is the same as the one you showed: _./myapptests -ln CoolerCheckTest.createDestroy CoolerCheckTest.init CoolerCheckTest.notCheckingFlowAndTemperature ... and a lot more, but don't need to show all of them.

bneumann commented 3 years ago

That is super weird. Let me check if I can somehow reproduce it. Otherwise I will release a new version with more logging

lokimon commented 3 years ago

Hey Benjamin, I also was about to create an issue here. I was wondering why unit tests stopped working from awhile back.. just realized I did upgrade cpputest to latest. If there is something I can do to help, let me know. Keep up the good work!

$ unit_tests-debug.out -ln test_group.init_pool

bneumann commented 3 years ago

Hey Scott, thanks for your input. I tried and ran my tests with the latest version of CppUTest and it still works. I will provide a more verbose version that prints all the single steps into the VS console.

jdpcmt commented 3 years ago

I will give a try with the more verbose version when available. I'll be out of the office next week, so pardon me if you don't receive my feedback immediately.

bneumann commented 3 years ago

@jdpcmt, @lokimon I just updated to version 0.1.10. It should contain more verbose logging now. Can check if there is anything unusal? In my test cases it looks like so:

[2021-03-31 11:01:22.149] [INFO] Test Explorer found
[2021-03-31 11:01:22.149] [INFO] Creating adapter for /mnt/c/Users/benni/Documents/CppUTest-Workspace
[2021-03-31 11:01:22.149] [INFO] Initializing adapter
[2021-03-31 11:01:22.150] [INFO] Registering adapter for /mnt/c/Users/benni/Documents/CppUTest-Workspace
[2021-03-31 11:01:22.152] [INFO] Loading tests
[2021-03-31 11:01:22.153] [INFO] Loading tests using /mnt/c/Users/benni/Documents/CppUTest-Workspace/test/testrunner inside /mnt/c/Users/benni/Documents/CppUTest-Workspace/test
[2021-03-31 11:01:22.165] [INFO] Initialization finished
[2021-03-31 11:01:22.204] [INFO] The testrunner returned: ClassName.Create ClassName.ShouldPass SecondClass.ShouldPass SecondClass.ShouldFail SecondClass.ShouldBeNew SecondClass.ShouldAlsoBeIgnored SecondClass.ShouldBeIgnored Special_characters.in_test_name Specials.Crash Specials.AnotherStuff. Splitting now
[2021-03-31 11:01:22.204] [INFO] Trying to resolve symbol names, file and line numbers using compiler tools. This might throw warnings with ignored tests
[2021-03-31 11:01:22.204] [INFO] Adding Create to group ClassName
[2021-03-31 11:01:22.369] [INFO] Adding Create to group ClassName
[2021-03-31 11:01:22.449] [INFO] Adding Create to group ClassName
[2021-03-31 11:01:22.504] [INFO] Adding Create to group ClassName
[2021-03-31 11:01:22.565] [INFO] Adding ShouldPass to group ClassName
[2021-03-31 11:01:22.623] [INFO] Adding ShouldPass to group ClassName
[2021-03-31 11:01:22.678] [INFO] Adding ShouldPass to group ClassName
[2021-03-31 11:01:22.734] [INFO] Adding ShouldPass to group ClassName
[2021-03-31 11:01:22.788] [INFO] Adding ShouldPass to group SecondClass
[2021-03-31 11:01:22.846] [INFO] Adding ShouldPass to group SecondClass
[2021-03-31 11:01:22.896] [INFO] Adding ShouldPass to group SecondClass
[2021-03-31 11:01:22.955] [INFO] Adding ShouldPass to group SecondClass
[2021-03-31 11:01:23.007] [INFO] Adding ShouldFail to group SecondClass
[2021-03-31 11:01:23.059] [INFO] Adding ShouldFail to group SecondClass
[2021-03-31 11:01:23.133] [INFO] Adding ShouldFail to group SecondClass
[2021-03-31 11:01:23.189] [INFO] Adding ShouldFail to group SecondClass
[2021-03-31 11:01:23.236] [INFO] Adding ShouldBeNew to group SecondClass
[2021-03-31 11:01:23.284] [INFO] Adding ShouldBeNew to group SecondClass
[2021-03-31 11:01:23.331] [INFO] Adding ShouldBeNew to group SecondClass
[2021-03-31 11:01:23.378] [INFO] Adding ShouldBeNew to group SecondClass
[2021-03-31 11:01:23.425] [INFO] Adding ShouldAlsoBeIgnored to group SecondClass
[2021-03-31 11:01:23.643] [WARN] stderr objdump -lSd /mnt/c/Users/benni/Documents/CppUTest-Workspace/test/testrunner | grep -m 1 -A 2 TEST_SecondClass_ShouldAlsoBeIgnored
[2021-03-31 11:01:23.643] [INFO] Adding ShouldAlsoBeIgnored to group SecondClass
[2021-03-31 11:01:23.842] [WARN] stderr objdump -lSd /mnt/c/Users/benni/Documents/CppUTest-Workspace/test/testrunner | grep -m 1 -A 2 TEST_SecondClass_ShouldAlsoBeIgnored
[2021-03-31 11:01:23.842] [INFO] Adding ShouldAlsoBeIgnored to group SecondClass
[2021-03-31 11:01:24.056] [WARN] stderr objdump -lSd /mnt/c/Users/benni/Documents/CppUTest-Workspace/test/testrunner | grep -m 1 -A 2 TEST_SecondClass_ShouldAlsoBeIgnored
[2021-03-31 11:01:24.056] [INFO] Adding ShouldAlsoBeIgnored to group SecondClass
[2021-03-31 11:01:24.268] [WARN] stderr objdump -lSd /mnt/c/Users/benni/Documents/CppUTest-Workspace/test/testrunner | grep -m 1 -A 2 TEST_SecondClass_ShouldAlsoBeIgnored
[2021-03-31 11:01:24.269] [INFO] Adding ShouldBeIgnored to group SecondClass
[2021-03-31 11:01:24.491] [WARN] stderr objdump -lSd /mnt/c/Users/benni/Documents/CppUTest-Workspace/test/testrunner | grep -m 1 -A 2 TEST_SecondClass_ShouldBeIgnored
[2021-03-31 11:01:24.491] [INFO] Adding ShouldBeIgnored to group SecondClass
[2021-03-31 11:01:24.692] [WARN] stderr objdump -lSd /mnt/c/Users/benni/Documents/CppUTest-Workspace/test/testrunner | grep -m 1 -A 2 TEST_SecondClass_ShouldBeIgnored
[2021-03-31 11:01:24.692] [INFO] Adding ShouldBeIgnored to group SecondClass
[2021-03-31 11:01:24.893] [WARN] stderr objdump -lSd /mnt/c/Users/benni/Documents/CppUTest-Workspace/test/testrunner | grep -m 1 -A 2 TEST_SecondClass_ShouldBeIgnored
[2021-03-31 11:01:24.893] [INFO] Adding ShouldBeIgnored to group SecondClass
[2021-03-31 11:01:25.094] [WARN] stderr objdump -lSd /mnt/c/Users/benni/Documents/CppUTest-Workspace/test/testrunner | grep -m 1 -A 2 TEST_SecondClass_ShouldBeIgnored
[2021-03-31 11:01:25.094] [INFO] Adding in_test_name to group Special_characters
[2021-03-31 11:01:25.132] [INFO] Adding in_test_name to group Special_characters
[2021-03-31 11:01:25.172] [INFO] Adding in_test_name to group Special_characters
[2021-03-31 11:01:25.214] [INFO] Adding in_test_name to group Special_characters
[2021-03-31 11:01:25.254] [INFO] Adding Crash to group Specials
[2021-03-31 11:01:25.341] [INFO] Adding Crash to group Specials
[2021-03-31 11:01:25.426] [INFO] Adding Crash to group Specials
[2021-03-31 11:01:25.513] [INFO] Adding Crash to group Specials
[2021-03-31 11:01:25.596] [INFO] Adding AnotherStuff to group Specials
[2021-03-31 11:01:25.635] [INFO] Adding AnotherStuff to group Specials
[2021-03-31 11:01:25.672] [INFO] Adding AnotherStuff to group Specials
[2021-03-31 11:01:25.713] [INFO] Adding AnotherStuff to group Specials
[2021-03-31 11:01:25.752] [INFO] Tests loaded
jdpcmt commented 3 years ago

As per the new log information, the problem might come from the LD_LIBRARY_PATH which seems not defined. I tried setting it in the vscode terminal but this has no impact on the extension behaviour. Do you know where shall the path to the libraries be defined?

Log: [2021-03-31 11:43:19.458] [INFO] Test Explorer found [2021-03-31 11:43:19.458] [INFO] Creating adapter for /home/dev/workspace/t3platform [2021-03-31 11:43:19.458] [INFO] Initializing adapter [2021-03-31 11:43:19.459] [INFO] Registering adapter for /home/dev/workspace/t3platform [2021-03-31 11:43:19.462] [INFO] Loading tests [2021-03-31 11:43:19.462] [INFO] Loading tests using /home/dev/workspace/t3platform/IFC/apps/op-svc/build/op-svc_tests inside /home/dev/workspace/t3platform/IFC/apps/op-svc/ [2021-03-31 11:43:19.467] [INFO] Initialization finished [2021-03-31 11:43:19.502] [ERROR] stderr /home/dev/workspace/t3platform/IFC/apps/op-svc/build/op-svc_tests: error while loading shared libraries: libbson-1.0.so.0: cannot open shared object file: No such file or directory

[2021-03-31 11:43:19.502] [INFO] The testrunner returned: . Splitting now [2021-03-31 11:43:19.502] [INFO] Trying to resolve symbol names, file and line numbers using compiler tools. This might throw warnings with ignored tests [2021-03-31 11:43:19.502] [INFO] Adding undefined to group

lokimon commented 3 years ago

Hmm.. looks like I get everything but that "Tests loaded" line.

[2021-03-31 12:54:41.970] [INFO] Test Explorer found
[2021-03-31 12:54:41.971] [INFO] Creating adapter for /home/lokimon/source/repos/test
[2021-03-31 12:54:41.971] [INFO] Initializing adapter
[2021-03-31 12:54:41.971] [INFO] Registering adapter for /home/lokimon/source/repos/test
[2021-03-31 12:54:41.971] [INFO] Loading tests
[2021-03-31 12:54:41.972] [INFO] Loading tests using /home/lokimon/source/repos/test/test-oms/deploy/unit_tests-debug.out inside /home/lokimon/source/repos/test/test-oms/deploy
[2021-03-31 12:54:41.992] [INFO] Initialization finished
[2021-03-31 12:54:42.048] [INFO] The testrunner returned: test_group.init_pool. Splitting now
[2021-03-31 12:54:42.048] [INFO] Trying to resolve symbol names, file and line numbers using compiler tools. This might throw warnings with ignored tests
[2021-03-31 12:54:42.048] [INFO] Adding init_pool to group test_group
lokimon commented 3 years ago

I added another test to to check something.. looks like it gets hung up "Adding" Tests. It only gives me a "Adding" line for the first test.

[2021-03-31 13:06:13.105] [INFO] Test Explorer found
[2021-03-31 13:06:13.105] [INFO] Creating adapter for /home/lokimon/source/repos/test
[2021-03-31 13:06:13.105] [INFO] Initializing adapter
[2021-03-31 13:06:13.106] [INFO] Registering adapter for /home/lokimon/source/repos/test
[2021-03-31 13:06:13.106] [INFO] Loading tests
[2021-03-31 13:06:13.106] [INFO] Loading tests using /home/lokimon/source/repos/test/test-oms/deploy/unit_tests-debug.out inside /home/lokimon/source/repos/test/test-oms/deploy
[2021-03-31 13:06:13.111] [INFO] Initialization finished
[2021-03-31 13:06:13.145] [INFO] The testrunner returned: test_group2.init_pool2 test_group.init_pool. Splitting now
[2021-03-31 13:06:13.145] [INFO] Trying to resolve symbol names, file and line numbers using compiler tools. This might throw warnings with ignored tests
[2021-03-31 13:06:13.145] [INFO] Adding init_pool2 to group test_group2
lokimon commented 3 years ago

I can confirm now that CppUTest Test Adapter version 0.1.6 still works. I have Test Explorer UI 2.19.6 (latest) as well.

I just went down the list until this one worked.

[2021-03-31 17:36:35.078] [INFO] Test Explorer found
[2021-03-31 17:36:35.078] [INFO] Creating adapter for /home/lokimon/source/repos/test
[2021-03-31 17:36:35.078] [INFO] Initializing adapter
[2021-03-31 17:36:35.078] [INFO] Registering adapter for /home/lokimon/source/repos/test
[2021-03-31 17:36:35.079] [INFO] Loading tests
[2021-03-31 17:36:35.084] [INFO] Initialization finished
[2021-03-31 17:36:35.172] [INFO] Tests loaded
bneumann commented 3 years ago

Okay that is really interesting. With 0.1.6 I implemented the line and file number match. I am using objdump to get the information from the executables. What compiler are you using?

lokimon commented 3 years ago

To be clear... 0.1.6 works. 0.1.7 does not.

$ gcc --version
gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-44)
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
bneumann commented 3 years ago

Oh yes that is what I meant ;) I implemented the jump to file stuff from 16 to 17. I will try to get my hands on this pretty old version of gcc. I compiled with 8 and 9

lokimon commented 3 years ago

Sounds good.. Our team is stuck on Centos 7. :(

bneumann commented 3 years ago

Phew ;) Sounds like medical software. Or very penny-pinching management. With my shiny Ubuntu I can get as low as gcc-7 without digging deeper and screwing my setup. I will try a docker build but that needs more time. For now I just wrapped the objdump code with another try catch and logging. Should be live under 0.1.11 in a phew. If that doesn't help I will try and build with the older version.

lokimon commented 3 years ago

I didn't notice anything different with 0.1.11

[2021-03-31 18:45:58.112] [INFO] Test Explorer found
[2021-03-31 18:45:58.112] [INFO] Creating adapter for /home/lokimon/source/repos/test
[2021-03-31 18:45:58.112] [INFO] Initializing adapter
[2021-03-31 18:45:58.113] [INFO] Registering adapter for /home/lokimon/source/repos/test
[2021-03-31 18:45:58.113] [INFO] Loading tests
[2021-03-31 18:45:58.114] [INFO] Loading tests using /home/lokimon/source/repos/test/test-oms/deploy/omss_unit_tests-debug.out inside /home/lokimon/source/repos/test/test-oms/deploy
[2021-03-31 18:45:58.118] [INFO] Initialization finished
[2021-03-31 18:45:58.186] [INFO] The testrunner returned: test_group2.init_pool2 test_group.init_pool. Splitting now
[2021-03-31 18:45:58.186] [INFO] Trying to resolve symbol names, file and line numbers using compiler tools. This might throw warnings with ignored tests
[2021-03-31 18:45:58.186] [INFO] Adding init_pool2 to group test_group2
bneumann commented 3 years ago

Okay that's something I guess. I can rule out the objdump problem.

Oh @jdpcmt I will check why the test run doesn't find the LD_LIBRARY_PATH. My guess is that test explorer spawns a new isolated shell every time and ignores previously set environment variables. As I said, just a guess. What you could do is to check if setting the path via ldonf works. If it does I can see if I can bring in the LD_LIBRARY_PATH into the testrun shell. Or add another option to the adapter to define library folders.

jdpcmt commented 3 years ago

I guess that you meant ldconfig when you suggested to set the path via ldonf. In fact I tried today to add the LD_LIBRARY_PATH into my .profile and there's an improvement. Now the log shows:

[2021-04-01 06:10:02.128] [INFO] Test Explorer found [2021-04-01 06:10:02.128] [INFO] Creating adapter for /home/dev/workspace/t3platform [2021-04-01 06:10:02.128] [INFO] Initializing adapter [2021-04-01 06:10:02.133] [INFO] Registering adapter for /home/dev/workspace/t3platform [2021-04-01 06:10:02.134] [INFO] Loading tests [2021-04-01 06:10:02.134] [INFO] Loading tests using /home/dev/workspace/t3platform/IFC/apps/op-svc/build/op-svc_tests inside /home/dev/workspace/t3platform/IFC/apps/op-svc/ [2021-04-01 06:10:02.138] [INFO] Initialization finished [2021-04-01 06:10:03.614] [INFO] The testrunner returned: CoolerCheckTest.createDestroy CoolerCheckTest.init CoolerCheckTest.notCheckingFlowAndTemperature ............ many other tests hidden ................ [2021-04-01 06:10:03.615] [INFO] Trying to resolve symbol names, file and line numbers using compiler tools. This might throw warnings with ignored tests [2021-04-01 06:10:03.615] [INFO] Adding createDestroy to group CoolerCheckTest [2021-04-01 06:11:17.994] [INFO] Adding createDestroy to group CoolerCheckTest [2021-04-01 06:12:41.361] [INFO] Adding createDestroy to group CoolerCheckTest [2021-04-01 06:13:47.545] [INFO] Adding createDestroy to group CoolerCheckTest [2021-04-01 06:14:56.584] [INFO] Adding createDestroy to group CoolerCheckTest [2021-04-01 06:16:10.402] [INFO] Adding createDestroy to group CoolerCheckTest

As you can see, it seems hanging on the 1st test.

bneumann commented 3 years ago

Alrite. I just pusehd a new patch 0.1.12 which has a disable optione for the debugging stuff. And it has even more logging now. I suspect that it either crashes or hangs during the objdump stuff. If you have huge test files that whole process might actually be to much for the extension to handle. Try and set the cpputestExplorer.disableDebug setting to true (best restart the application to be sure nothing is processing in background) and see if the tests are added now. That should be more or less the same workflow as with version 0.1.6.

jdpcmt commented 3 years ago

With version 0.1.12 and disabling the debug information, the extension is now able to show all the tests. Running the tests is also working. Of course, the down-side of disabling the debug information is that debugging a test isn't possible anymore, which is a pity.

When enabling the debug information option, I get the following log entry: 2021-04-01 11:22:38.990] [ERROR] Resolving debug information is enabled, analyzing test createDestroy

I don't know if this it's expected or not, but when the debug option is disabled, the log is filled with 21 identical groups of 2 messages per test item:

[2021-04-01 11:21:32.881] [INFO] Adding calcUsedCurrentMf to group OpHelpersTest [2021-04-01 11:21:32.881] [ERROR] Resolving debug information is disabled, skipping test calcUsedCurrentMf .. repeating the same messages 21 times .. [2021-04-01 11:21:32.881] [ERROR] Resolving debug information is disabled, skipping test calcUsedCurrentMf [2021-04-01 11:21:32.881] [INFO] Adding calcUsedCurrentMf to group OpHelpersTest

lokimon commented 3 years ago

With Debug enabled, doesn't work

[2021-04-01 14:31:27.476] [INFO] Test Explorer found
[2021-04-01 14:31:27.477] [INFO] Creating adapter for /home/lokimon/source/repos/test
[2021-04-01 14:31:27.477] [INFO] Initializing adapter
[2021-04-01 14:31:27.477] [INFO] Registering adapter for /home/lokimon/source/repos/test
[2021-04-01 14:31:27.478] [INFO] Loading tests
[2021-04-01 14:31:27.478] [INFO] Loading tests using /home/lokimon/source/repos/test/test-oms/deploy/omss_unit_tests-debug.out inside /home/lokimon/source/repos/test/test-oms/deploy
[2021-04-01 14:31:27.481] [INFO] Initialization finished
[2021-04-01 14:31:27.487] [INFO] The testrunner returned: test_group2.test_init_pool2 test_group.test_init_pool. Splitting now
[2021-04-01 14:31:27.487] [INFO] Trying to resolve symbol names, file and line numbers using compiler tools. This might throw warnings with ignored tests
[2021-04-01 14:31:27.487] [INFO] Adding test_init_pool2 to group test_group2
[2021-04-01 14:31:27.487] [ERROR] Resolving debug information is enabled, analyzing test test_init_pool2

With Debug disabled, does work

[2021-04-01 14:35:30.992] [INFO] Test Explorer found
[2021-04-01 14:35:30.992] [INFO] Creating adapter for /home/lokimon/source/repos/test
[2021-04-01 14:35:30.992] [INFO] Initializing adapter
[2021-04-01 14:35:30.992] [INFO] Registering adapter for /home/lokimon/source/repos/test
[2021-04-01 14:35:30.992] [INFO] Loading tests
[2021-04-01 14:35:30.992] [INFO] Loading tests using /home/lokimon/source/repos/test/test-oms/deploy/omss_unit_tests-debug.out inside /home/lokimon/source/repos/test/test-oms/deploy
[2021-04-01 14:35:30.996] [INFO] Initialization finished
[2021-04-01 14:35:31.000] [INFO] The testrunner returned: test_group2.test_init_pool2 test_group.test_init_pool. Splitting now
[2021-04-01 14:35:31.000] [INFO] Trying to resolve symbol names, file and line numbers using compiler tools. This might throw warnings with ignored tests
[2021-04-01 14:35:31.000] [INFO] Adding test_init_pool2 to group test_group2
[2021-04-01 14:35:31.000] [ERROR] Resolving debug information is disabled, skipping test test_init_pool2
[2021-04-01 14:35:31.000] [INFO] Adding test_init_pool2 to group test_group2
[2021-04-01 14:35:31.000] [ERROR] Resolving debug information is disabled, skipping test test_init_pool2
[2021-04-01 14:35:31.000] [INFO] Adding test_init_pool to group test_group
[2021-04-01 14:35:31.000] [ERROR] Resolving debug information is disabled, skipping test test_init_pool
[2021-04-01 14:35:31.000] [INFO] Adding test_init_pool to group test_group
[2021-04-01 14:35:31.000] [ERROR] Resolving debug information is disabled, skipping test test_init_pool
[2021-04-01 14:35:31.000] [INFO] Tests loaded
bneumann commented 3 years ago

Okay. Wonderful! Now I can investigate the why, @jdpcmt which compiler are you using?

I am using objdump -lSd to get the symbols from the binary. I am grepping for the test and group name in the output. I presume that somehow objdump has a problem here. Maybe you guys know a more robust way?

lokimon commented 3 years ago

Does this help?

$ objdump -lSd omss_unit_tests-debug.out | grep test_group
0000000000404028 <_ZN57TEST_test_group_test_init_pool_Test8testBodyEv>:
_ZN57TEST_test_group_test_init_pool_Test8testBodyEv():
TEST(test_group, test_init_pool)
  4040af:   eb 1a                   jmp    4040cb <_ZN57TEST_test_group_test_init_pool_Test8testBodyEv+0xa3>
TEST(test_group, test_init_pool)
  4040f5:   48 8d 05 64 bd 84 00    lea    0x84bd64(%rip),%rax        # c4fe60 <TEST_test_group_test_init_pool_TestShell_instance>
  4040ff:   e8 32 3e 39 00          callq  797f36 <_ZN62TEST_test_group_test_init_pool_TestShellC1Ev>
  40410b:   48 8d 05 4e bd 84 00    lea    0x84bd4e(%rip),%rax        # c4fe60 <TEST_test_group_test_init_pool_TestShell_instance>
  404115:   48 8d 05 ae 3e 39 00    lea    0x393eae(%rip),%rax        # 797fca <_ZN62TEST_test_group_test_init_pool_TestShellD1Ev>
  40413f:   48 8d 05 1a bd 84 00    lea    0x84bd1a(%rip),%rax        # c4fe60 <TEST_test_group_test_init_pool_TestShell_instance>
  404149:   48 8d 3d 48 bd 84 00    lea    0x84bd48(%rip),%rdi        # c4fe98 <_ZL62TEST_test_group_test_init_pool_Installer>
  40415c:   48 8d 35 35 bd 84 00    lea    0x84bd35(%rip),%rsi        # c4fe98 <_ZL62TEST_test_group_test_init_pool_Installer>
000000000040418a <_ZN59TEST_test_group2_test_init_pool2_Test8testBodyEv>:
_ZN59TEST_test_group2_test_init_pool2_Test8testBodyEv():
TEST(test_group2, test_init_pool2)
  404211:   eb 1a                   jmp    40422d <_ZN59TEST_test_group2_test_init_pool2_Test8testBodyEv+0xa3>
TEST(test_group2, test_init_pool2)
  404257:   48 8d 05 62 bc 84 00    lea    0x84bc62(%rip),%rax        # c4fec0 <TEST_test_group2_test_init_pool2_TestShell_instance>
  404261:   e8 ec 3e 39 00          callq  798152 <_ZN64TEST_test_group2_test_init_pool2_TestShellC1Ev>
  40426d:   48 8d 05 4c bc 84 00    lea    0x84bc4c(%rip),%rax        # c4fec0 <TEST_test_group2_test_init_pool2_TestShell_instance>
  404277:   48 8d 05 68 3f 39 00    lea    0x393f68(%rip),%rax        # 7981e6 <_ZN64TEST_test_group2_test_init_pool2_TestShellD1Ev>
  4042a1:   48 8d 05 18 bc 84 00    lea    0x84bc18(%rip),%rax        # c4fec0 <TEST_test_group2_test_init_pool2_TestShell_instance>
  4042ab:   48 8d 3d 46 bc 84 00    lea    0x84bc46(%rip),%rdi        # c4fef8 <_ZL64TEST_test_group2_test_init_pool2_Installer>
  4042be:   48 8d 35 33 bc 84 00    lea    0x84bc33(%rip),%rsi        # c4fef8 <_ZL64TEST_test_group2_test_init_pool2_Installer>
0000000000797e16 <_ZN45TEST_GROUP_CppUTestGrouptest_groupC1Ev>:
_ZN45TEST_GROUP_CppUTestGrouptest_groupC2Ev():
TEST_GROUP(test_group)
  797e32:   48 8d 15 a7 d1 34 00    lea    0x34d1a7(%rip),%rdx        # ae4fe0 <_ZTV45TEST_GROUP_CppUTestGrouptest_group>
0000000000797e42 <_ZN45TEST_GROUP_CppUTestGrouptest_groupD1Ev>:
_ZN45TEST_GROUP_CppUTestGrouptest_groupD2Ev():
  797e52:   48 8d 15 87 d1 34 00    lea    0x34d187(%rip),%rdx        # ae4fe0 <_ZTV45TEST_GROUP_CppUTestGrouptest_group>
  797e73:   74 0c                   je     797e81 <_ZN45TEST_GROUP_CppUTestGrouptest_groupD1Ev+0x3f>
0000000000797e84 <_ZN45TEST_GROUP_CppUTestGrouptest_groupD0Ev>:
_ZN45TEST_GROUP_CppUTestGrouptest_groupD0Ev():
  797e97:   e8 a6 ff ff ff          callq  797e42 <_ZN45TEST_GROUP_CppUTestGrouptest_groupD1Ev>
0000000000797eaa <_ZN57TEST_test_group_test_init_pool_TestC1Ev>:
_ZN57TEST_test_group_test_init_pool_TestC2Ev():
TEST(test_group, test_init_pool)
  797ebd:   e8 54 ff ff ff          callq  797e16 <_ZN45TEST_GROUP_CppUTestGrouptest_groupC1Ev>
  797ec6:   48 8d 15 73 cf 34 00    lea    0x34cf73(%rip),%rdx        # ae4e40 <_ZTV57TEST_test_group_test_init_pool_Test>
0000000000797ed6 <_ZN62TEST_test_group_test_init_pool_TestShell10createTestEv>:
_ZN62TEST_test_group_test_init_pool_TestShell10createTestEv():
  797f01:   e8 a4 ff ff ff          callq  797eaa <_ZN57TEST_test_group_test_init_pool_TestC1Ev>
  797f09:   eb 22                   jmp    797f2d <_ZN62TEST_test_group_test_init_pool_TestShell10createTestEv+0x57>
0000000000797f36 <_ZN62TEST_test_group_test_init_pool_TestShellC1Ev>:
_ZN62TEST_test_group_test_init_pool_TestShellC2Ev():
  797f52:   48 8d 15 27 cf 34 00    lea    0x34cf27(%rip),%rdx        # ae4e80 <_ZTV62TEST_test_group_test_init_pool_TestShell>
0000000000797f62 <_ZN57TEST_test_group_test_init_pool_TestD1Ev>:
_ZN57TEST_test_group_test_init_pool_TestD2Ev():
  797f72:   48 8d 15 c7 ce 34 00    lea    0x34cec7(%rip),%rdx        # ae4e40 <_ZTV57TEST_test_group_test_init_pool_Test>
  797f87:   e8 b6 fe ff ff          callq  797e42 <_ZN45TEST_GROUP_CppUTestGrouptest_groupD1Ev>
  797f93:   74 0c                   je     797fa1 <_ZN57TEST_test_group_test_init_pool_TestD1Ev+0x3f>
0000000000797fa4 <_ZN57TEST_test_group_test_init_pool_TestD0Ev>:
_ZN57TEST_test_group_test_init_pool_TestD0Ev():
  797fb7:   e8 a6 ff ff ff          callq  797f62 <_ZN57TEST_test_group_test_init_pool_TestD1Ev>
0000000000797fca <_ZN62TEST_test_group_test_init_pool_TestShellD1Ev>:
_ZN62TEST_test_group_test_init_pool_TestShellD2Ev():
  797fda:   48 8d 15 9f ce 34 00    lea    0x34ce9f(%rip),%rdx        # ae4e80 <_ZTV62TEST_test_group_test_init_pool_TestShell>
  797ffb:   74 0c                   je     798009 <_ZN62TEST_test_group_test_init_pool_TestShellD1Ev+0x3f>
000000000079800c <_ZN62TEST_test_group_test_init_pool_TestShellD0Ev>:
_ZN62TEST_test_group_test_init_pool_TestShellD0Ev():
  79801f:   e8 a6 ff ff ff          callq  797fca <_ZN62TEST_test_group_test_init_pool_TestShellD1Ev>
0000000000798032 <_ZN46TEST_GROUP_CppUTestGrouptest_group2C1Ev>:
_ZN46TEST_GROUP_CppUTestGrouptest_group2C2Ev():
TEST_GROUP(test_group2)
  79804e:   48 8d 15 6b d1 34 00    lea    0x34d16b(%rip),%rdx        # ae51c0 <_ZTV46TEST_GROUP_CppUTestGrouptest_group2>
000000000079805e <_ZN46TEST_GROUP_CppUTestGrouptest_group2D1Ev>:
_ZN46TEST_GROUP_CppUTestGrouptest_group2D2Ev():
  79806e:   48 8d 15 4b d1 34 00    lea    0x34d14b(%rip),%rdx        # ae51c0 <_ZTV46TEST_GROUP_CppUTestGrouptest_group2>
  79808f:   74 0c                   je     79809d <_ZN46TEST_GROUP_CppUTestGrouptest_group2D1Ev+0x3f>
00000000007980a0 <_ZN46TEST_GROUP_CppUTestGrouptest_group2D0Ev>:
_ZN46TEST_GROUP_CppUTestGrouptest_group2D0Ev():
  7980b3:   e8 a6 ff ff ff          callq  79805e <_ZN46TEST_GROUP_CppUTestGrouptest_group2D1Ev>
00000000007980c6 <_ZN59TEST_test_group2_test_init_pool2_TestC1Ev>:
_ZN59TEST_test_group2_test_init_pool2_TestC2Ev():
TEST(test_group2, test_init_pool2)
  7980d9:   e8 54 ff ff ff          callq  798032 <_ZN46TEST_GROUP_CppUTestGrouptest_group2C1Ev>
  7980e2:   48 8d 15 37 cf 34 00    lea    0x34cf37(%rip),%rdx        # ae5020 <_ZTV59TEST_test_group2_test_init_pool2_Test>
00000000007980f2 <_ZN64TEST_test_group2_test_init_pool2_TestShell10createTestEv>:
_ZN64TEST_test_group2_test_init_pool2_TestShell10createTestEv():
  79811d:   e8 a4 ff ff ff          callq  7980c6 <_ZN59TEST_test_group2_test_init_pool2_TestC1Ev>
  798125:   eb 22                   jmp    798149 <_ZN64TEST_test_group2_test_init_pool2_TestShell10createTestEv+0x57>
0000000000798152 <_ZN64TEST_test_group2_test_init_pool2_TestShellC1Ev>:
_ZN64TEST_test_group2_test_init_pool2_TestShellC2Ev():
  79816e:   48 8d 15 eb ce 34 00    lea    0x34ceeb(%rip),%rdx        # ae5060 <_ZTV64TEST_test_group2_test_init_pool2_TestShell>
000000000079817e <_ZN59TEST_test_group2_test_init_pool2_TestD1Ev>:
_ZN59TEST_test_group2_test_init_pool2_TestD2Ev():
  79818e:   48 8d 15 8b ce 34 00    lea    0x34ce8b(%rip),%rdx        # ae5020 <_ZTV59TEST_test_group2_test_init_pool2_Test>
  7981a3:   e8 b6 fe ff ff          callq  79805e <_ZN46TEST_GROUP_CppUTestGrouptest_group2D1Ev>
  7981af:   74 0c                   je     7981bd <_ZN59TEST_test_group2_test_init_pool2_TestD1Ev+0x3f>
00000000007981c0 <_ZN59TEST_test_group2_test_init_pool2_TestD0Ev>:
_ZN59TEST_test_group2_test_init_pool2_TestD0Ev():
  7981d3:   e8 a6 ff ff ff          callq  79817e <_ZN59TEST_test_group2_test_init_pool2_TestD1Ev>
00000000007981e6 <_ZN64TEST_test_group2_test_init_pool2_TestShellD1Ev>:
_ZN64TEST_test_group2_test_init_pool2_TestShellD2Ev():
  7981f6:   48 8d 15 63 ce 34 00    lea    0x34ce63(%rip),%rdx        # ae5060 <_ZTV64TEST_test_group2_test_init_pool2_TestShell>
  798217:   74 0c                   je     798225 <_ZN64TEST_test_group2_test_init_pool2_TestShellD1Ev+0x3f>
0000000000798228 <_ZN64TEST_test_group2_test_init_pool2_TestShellD0Ev>:
_ZN64TEST_test_group2_test_init_pool2_TestShellD0Ev():
  79823b:   e8 a6 ff ff ff          callq  7981e6 <_ZN64TEST_test_group2_test_init_pool2_TestShellD1Ev>
lokimon commented 3 years ago

I misspoke earlier... while having the "cpputestExplorer.disableDebug": true, does populate the test window with the tests, if you run the tests, they "fail". But if you run from command line they pass. Also going back to version 0.1.6 they pass.

bneumann commented 3 years ago

Oh come on ;) I am about to give up that doesn't even make sense IMO why would they fail? gnar! Anyways. Maybe I can find some time on the weekend to check it out. Sorry for the inconvenience

jdpcmt commented 3 years ago

@jdpcmt which compiler are you using?

I'm using gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0

lokimon commented 2 years ago

Hello bneumann. Our team has really started embracing this extension. I've been super busy, but I'm thinking I would like to help get this back into shape? We are stuck using Version 0.1.6 because later version fail for 1 reason or the other.

Would be open to pulling this down, and doing some source level debugging, if you are ok with that. Also, any pointers on how to debug an extension while running it in another instance of VSC would be helpful.

bneumann commented 2 years ago

Hi @lokimon, oh man sorry for not coming back for this topic. I was a bit occupied (new job, Corona, etc). Anyways help would be appreciated! Also there is the new testing API for VSCode that I haven't had the time to look into yet but from the documentation it looks fantastic.

About debugging an Extension: Just clone this repository and open it in VSCode. If you start debugging it should already open a second instance of VSCode with the extension being installed. I usually create a new workspace or folder which I setup for the extension with the paths and debugger information (in the application settings).

We could also have a chat in a google meet or so if you run into problems

lokimon commented 2 years ago

I have cloned the repo and am able to debug the extension.

It seems to be getting here and then blowing up. That split doesn't work, given the output of that grep, but I'm unsure what it should be trying to pull out. https://github.com/bneumann/CppUTest-Test-Adapter/blob/d2dc2001b883abc8ed708d6204c4c368e7af9437/src/cpputest.ts#L61

I have to obfuscate some of the data, but

sourceGrep is something like this "objdump -lSd /home/lokimon/deploy/ut_omss_strike_sequencer_abort.out | grep -m 1 -A 2 TEST_testgroup1_test1"

stdout is "000000000041b820 <_ZN108TEST_testgroup1_test1_Test8testBodyEv>:\n"

Here is the surrounding text of that grep

000000000041b820 <_ZN108TEST_testgroup1_test1_Test8testBodyEv>: _ZN108TEST_testgroup1_test1_Test8testBodyEv(): /home/lokimon/src/testgroup1.cpp:2072

Does that help at all?

lokimon commented 2 years ago

I changed the -m 1 in https://github.com/bneumann/CppUTest-Test-Adapter/blob/d2dc2001b883abc8ed708d6204c4c368e7af9437/src/cpputest.ts#L52

to -m 2 and it no longer blows up!

The Test UI is being correctly populated and the Extension seems to be working again!