search

ionide / proj-info

Parse and evaluate MsBuild project files
MIT License
64 stars 37 forks source link

Run tests in parallel #184

Open safesparrow opened 1 year ago

safesparrow commented 1 year ago

Is your feature request related to a problem? Please describe. Running tests takes a long time, because they run in sequence.

Describe the solution you'd like I would like the tests to be run in parallel.

Additional context I tried this and most tests work (and fast), but some are unable to load the projects when running alongside other tests. I'm trying to find the root cause of that. This also requires using unique test directories for each test (currently they are not).

safesparrow commented 1 year ago

So one type of tests that fails when run in parallel is the load project from disk everytime but only the one that uses WorkspaceLoaderProjectGraph. On the screenshot below the middle test is the one using project graph, while first and last use WorkspaceLoader (I changed the names to force a certain order of test scheduling (which might have been useless anyway).

image

@baronfel Do you have any idea what about this type of test would make it susceptible to other tests' side effects? I can't quite figure out what's different about running in parallel.

I already use unique test directories. As far as I can see we already use a single tools path for all tests, ie. we don't re-init ProjInfo for each individual proejct.

TheAngryByrd commented 1 year ago

Is there a specific exception coming through for WorkspaceLoaderProjectGraph test? I know ProjectGraph can throw exceptions if another "build" is in progress.

safesparrow commented 1 year ago

How can I see those exceptions? What I'm getting is no projects are loaded so the test fails later on. I guess I should generate a binlog in case something is wrong in msbuild.