microsoft / vscode-java-test

Run and debug Java test cases in Visual Studio Code.
https://marketplace.visualstudio.com/items?itemName=vscjava.vscode-java-test
Other
295 stars 127 forks source link

Wrong Testing Result Display #1659

Open Ao-Last opened 8 months ago

Ao-Last commented 8 months ago

What happened?

Run test from the side panel and it display with green icon while the test cannot even build.

image
jdneo commented 8 months ago

Those 'dark' green check marks are the history test status.

You can run the command Test: Clear All Results to reset them.

What's your build error in the output channel?

Ao-Last commented 8 months ago

Those 'dark' green check marks are the history test status.

You can run the command Test: Clear All Results to reset them.

What's your build error in the output channel?

Thanks for pointing out the 'dark' icon's meaning. The build error is caused by an undefined symbol. I cannot reproduce it again and I will reopen the issue once I meet it again.

Ao-Last commented 8 months ago

@jdneo To reproduce,

  1. make an build error caused by a wrong usage of assert methods. E.g. assertTrue(String) image
  2. run the test from the sidebar by clicking the triangle button and wait for prompt error message image
  3. click "Continue", and the icon would be green check again. image

Build error:

image

Tested with "Test: Clear All Results", the icon does not change (in a untested statu not a error status).

jdneo commented 8 months ago

Ok, so the green check in step 3 is a history status.

A better solution might be that display the compilation error in the peek view and mark the test as failed.

Ao-Last commented 8 months ago

Yep, it shows the history status when meeting a compilation error. I suggest to mark it as a test error instead of test failed to distinguish between wrong test code and wrong source code.