Persisted files associated with a test run

[firelizzard18]

I'd like to persist files in such a way that they're associated with a test run. Currently I'm storing files in the extension's storage directory and I use TestRun.onDidDispose to delete files when a test run is discarded, but that cleanup only works until VSCode is closed. Test runs (can) persist across restarts and it would be valuable to retain associated files across restarts but as far as I can tell there's no way for me to inform VSCode, "When {an old test run loaded from disk} is cleared, also clear {an associated file}". If old test runs were exposed in some way, I could potentially scan those and use onDidDispose, but A) AFAIK old runs are not exposed and B) I don't have any real way of differentiating multiple unnamed runs so I don't know which run should be associated with which file.

CC @connor4312

[connor4312]

Test runs from previous VS Code sessions contain only some very limited data that we use to hydrate the state of items, e.g. they don't contain any of the original failure messages. Could you share more about the use case you're building with these files?

[firelizzard18]

I'm capturing CPU/memory profiles for tests, and those profiles are emitted as files. I'm currently storing them in an OS temp directory that vscode-go deletes when the extension is deactivated, but it would be nice to persist them.

As a user, I restart VSCode for various reasons. Maybe it crashes (probably my fault, I'm using Gentoo), maybe something weird happens (Ctrl+C occasionally stops working), or maybe vscode-go's test controller goes haywire (definitely my fault). There have been various cases where I have had to re-execute a fairly long running test because I restarted vscode and lost the profile. When feasible, I'd like to give the user the option of persisting profiles until the test run is discarded.

Specifically, go test ... has a few flags including --cpuprofile {file} and --memprofile {file} to capture profiles. vscode-go executes the test with those flags, pointing them at some path, and then uses a webview and go tool pprof ... to display the profiles to the user.

Currently, vscode-go contributes a tree view to the test panel that lists captured profiles. I've been using that for a few years now and it's not a great UX. Generally I forget that the captured profiles exist so I re-profile the test anyways (because the profiles are opened after the test finishes). I'm currently working on overhauling vscode-go's test controller and I like the idea of showing the profiles in the test item tree.

[connor4312]

I think the way we'd solve that is to have some storage URI associated with the TestResult that VS Code manages the lifecycle of.

I think this could also be nice for other cases like storing coverage output, which we currently just tell people to stick in a temp directory.

[firelizzard18]

That would be perfect, that would certainly handle my use-case

[firelizzard18]

FWIW, this is how I'm showing profiles:

It's a bit awkward, but I'm happier with it than my previous solution, which was a separate tree view.