Apex Tests refresh button doesn't appear to detect test classes correctly

[laurakolkerappfire]

Summary

For the last couple of days (probably since the 58.14.2 release?) the apex tests section of the testing sidebar does not reliably load all the available classes with isTest annotation present in the project. One file's tests can be run from within it's editor, but if I want to run all tests, I can only run the ones detected in the sidebar, and it doesn't detect everything anymore. Refreshing appears to do nothing.

Steps To Reproduce:

I'm not yet sure about reliable steps to reproduce, but I've seen this in two contexts:

1. pulling from a new branch that has a new test file - then push to the default org, click refresh in the apex tests section of the sidebar. The new class is not detected as expected.
2. creating a new test file - push the new file to the default org, click refresh in the apex tests section of the new sidebar. The new class is not detected as expected.

Expected result

All current classes in the project with the isTest annotation are loaded into the apex tests section of the testing sidebar when the refresh tests button is clicked.

Actual result

Some classes (usually older ones) are detected and displayed but not all of them. Sometimes a class will be visible in the sidebar, and then disappear after a vscode restart.

Additional information

Example from today: When I grep the force-app directory for @isTest there are 8 files detected:

When I refresh the Apex Tests section, it detects 5 files:

It can't be configuration. One of the files that's missing today, was visible yesterday. No configuration changes have occurred since then.

Salesforce Extension Version in VS Code: v58.14.2

SFDX CLI Version: I'm using an older sf because our project depends on cci to setup new scratch orgs. @salesforce/cli/1.86.7-legacy.0 darwin-x64 node-v18.15.0 sfdx-cli/7.209.6 darwin-x64 node-v18.15.0

OS and version: Mac Ventura 13.5.1

[CristiCanizales]

Hi @laurakolkerappfire! Do those new test files actually contain methods to test? If a file doesn't contain any method with @isTest annotation, it won't show up in the Apex Tests section

[laurakolkerappfire]

Yes, they have methods with @isTest annotations

[daphne-sfdc]

Hi @laurakolkerappfire, Do any of the @isTest methods share the same name? If there are multiple methods with the same name, only one of those methods will show up in the Apex Tests sidebar, even if the methods have different content inside them.

[laurakolkerappfire]

Hi @daphne-sfdc The missing classes are functional and their tests can be run from within the editor with the Run Test or Run All Tests buttons visible under the isTest annotations. There are no samed named methods or samed named classes. If the class is detected by the sidebar, the methods are detected correctly in the sidebar. BUT whole classes just aren't detected predictably when the refresh button is clicked. (Which seems new. I used to be able to rely on that button responding to the current state predictably.)

I'm starting to wonder if the deployment process itself might be related? I've mostly been using my terminal to manage deployments to my scratch orgs, but I noticed when I use the VSCode triggered SFDX: push Source to Default Org command, sometimes a missing class will mysteriously re-appear. Perhaps the sidebar is now depending on some state that a deployment or git operation triggered outside of vscode wouldn't typically update? That's just speculation based on what I'm seeing.

[daphne-sfdc]

Hi @laurakolkerappfire,

Regarding your question, no, this is not related to the deployment process. Even if you only create the Apex class and test methods (but do not deploy them) and just refresh the Apex Tests sidebar, you should be able to see the tests there. What appears in the Apex Tests sidebar should be the classes that are present, not the classes that are deployed.

Can you run "SFDX: Run Apex Tests" from the command palette? When prompted, choose "All Tests". Are all your tests being run? Also, do all your tests show up in that list after "SFDX: Run Apex Tests"?

[laurakolkerappfire]

Hi @daphne-sfdc I can't seem to trigger the problem today. I wish I could share the screenshare I took earlier in the week about it, but it's just got too much actual code in it for sharing.

Relevant things that have happened in the meantime: I refreshed the language server because of that other comment, but that could be a coincidence.

If I run into the problem again, I'll double check how the SFDX: Run Apex Tests command affects the situation and report back.

[CristiCanizales]

Closing this as it can't be reproduced anymore. Feel free to reopen if the issue comes back