microsoft / vscode

Visual Studio Code
https://code.visualstudio.com
MIT License
162.39k stars 28.62k forks source link

Tabs API: better integration test coverage #145846

Open bpasero opened 2 years ago

bpasero commented 2 years ago

Testing #145585

I forgot to report this yesterday: I felt that a lot of the issues uncovered could have been found with just providing a very good integration test suite (see https://github.com/microsoft/vscode/issues/145773#issuecomment-1075967255 for example).

The tabs API is super nice to test actually.

A few things to ensure to cover:

We need to make sure that proper commands or API is used when altering state to prevent flaky tests as outlined in https://github.com/microsoft/vscode/issues/145773#issuecomment-1077782847

lramos15 commented 2 years ago

Isn't this actually more difficult than we thought as @jrieken pointed out without tabs API supporting modification. Any commands we use have the opportunity to cause flakiness if they don't accept an identifier. Unless we just want to test with like a single editor for example so the action is always unambiguous.

bpasero commented 2 years ago

I would try to get as much coverage done as possible with the API you have and leave the rest for later. Via API you can open editors, make them dirty, even open multiple editor groups, so a bit of coverage should be possible.