search

microsoft / vscode

Visual Studio Code
https://code.visualstudio.com
MIT License
162.43k stars 28.62k forks source link

Switching to test explorer when users clicks 'run tests' #135999

Closed jdneo closed 2 years ago

jdneo commented 2 years ago

Some Java customers give feedback that they are not aware of the existence of test explorer.

I'm not sure if making VS Code automatically switching to test explorer when users clicks 'run tests' could solve this issue. @connor4312 do you think this is a good UX?

// reference https://github.com/microsoft/vscode-java-test/issues/1325

nickzhums commented 2 years ago

We did a customer interview with Morgan Stanley, and one of their pain-points is that after clicking on the 'run test' button, it doesn't take developers to the test explorer UI - Instead, they only see a green mark besides the test.

This UX leads to them thinking that we don't provide any test UI at all - Later i pointed them to the test explorer and their feedback is that we should provide better directions and interaction related to this

One potential approach is jumping to test explorer directly after user clicks 'run test'.

nickzhums commented 2 years ago

The other approach is making test explorer a more obvious part of the overall UI and not dynamic based on some triggers

connor4312 commented 2 years ago

I would very much dislike this in my own flow. It's somewhat similar to debugging, where the debug view is shown by default when debugging starts. However, when I'm debugging, it's essentially guaranteed that I'm going to use the view because that's where variables, watch expressions, call stacks, etc. are shown. For tests, my most common flow is ctrl+; c to run the test my cursor is currently in. There's no need to show the test explorer view in this case.

How are Java users running tests? Through the quickpick or custom contributed buttons? Maybe those buttons should reveal the test explorer?

jdneo commented 2 years ago

I guess that UX(toggle out the test explorer) comes from other Java IDE, for example the IntelliJ IDEA:

https://user-images.githubusercontent.com/6193897/139359408-005c89b4-2bff-402d-9d92-078bf2743495.mp4

Different part is that, in IDEA, the test explorer is a new view which won't replace its 'file explorer'. In VS Code, they are all in the left side bar. I agree that if we always toggle to test explorer when testing in VS Code, there must be some users complaining that we disturb their focus.

Or maybe we make a setting to control whether switching to test explorer? It looks like we can do a a/b test for this.

connor4312 commented 2 years ago

Yes, we can certainly add a setting for it. I would prefer it be off by default, but in that case it doesn't solve the discoverability issue you mention. Are your users only executing tests through gutter icons, or through some other way?

jdneo commented 2 years ago

Are your users only executing tests through gutter icons, or through some other way?

The Java test extension has some other integrations with different UI entries: the Java Projects explorer & the run button in the editor title area. But the usage ratio of them are really small(~4%).

So most of the users run/debug their tests via the gutter icons or the test explorer.

Trass3r commented 2 years ago

A related issue arises when the sidebar is hidden manually or by the autohide extension (which really just implements functionality that should be builtin and has been requested for years #3742).

jdneo commented 2 years ago

@connor4312 I guess this issue can be closed given https://github.com/microsoft/vscode/issues/140596 has been resolved?

connor4312 commented 2 years ago

Oh, right. I looked for a duplicate issue but missed this.