Running part of an umbrella app in parallel

[alanvardy]

Hello again! I have been getting a great deal of value out of this tool, thank you!

I have another hopefully interesting puzzle. I am working in an umbrella app where global state is an issue, and have been running the tests sequentially rather than in parallel as I work to fix them over time. I was wondering this morning whether it might be possible to run the badly behaving apps sequentially and the rest in parallel.

apps = Mix.Project.apps_paths() |> Map.keys
non_parallel_apps = [:bad_app1, :bad_app2, :bad_app3]
[
  tools: [
    {:formatter, ["mix", "format"]},
    {:coveralls, umbrella: [parallel: true, apps: apps -- non_parallel_apps], env: %{"MIX_ENV" => "test"}, command: "mix cmd mix coveralls.html"},
    {:coveralls_non_parallel, umbrella: [parallel: false, apps: non_parallel_apps], env: %{"MIX_ENV" => "test"}, command: "mix cmd mix coveralls.html", deps: [:coveralls]},
    {:ex_unit, enabled: false},
    {:dialyzer, enabled: false},
    {:sobelow, enabled: false},
    {:npm_test, enabled: false},
  ]
]

The trouble I am encountering with this is that deps seems to be applied to the apps within the umbrella task rather than the umbrella task itself

   coveralls_non_parallel in bad_app1 skipped due to unsatisfied dependency coveralls in bad_app1
   coveralls_non_parallel in bad_app2 skipped due to unsatisfied dependency coveralls in bad_app2
   ...etc

Not sure if I am missing something, do you have a direction that you could point me in?

Thank you again!

[karolsluszniak]

Seems that you've assembled your config perfectly and it should do what you've intended. I think I know why that's not the case and instead you end up with unsatisfied deps.

It has to do with how umbrella recursive mode is implemented - by internally translating recursive tool instance into multiple instances described by {tool_name, app_name}, later formatted into coveralls_non_parallel in bad_app1 etc. Then, all dependencies in your config are remapped to to use these instances instead of the original tool. All that happens in ExCheck.Check.Compiler, in unwrap_recursive/1 and map_recursive_dependents/1 respectively. From then on, these tools are mostly treated like any other.

Now, I'm not that much in the context of this solution to identify the issue on the first sight, but I'll try to find some time to dig deeper. In the meantime, if the issue is urgent for you, feel free to start with functions mentioned above and do your own digging - hopefully you'll find a solution.

I remember for sure that deps between umbrella recursive tools was a complex scenario, with some edge cases and it seems you've stumbled upon one of them :)

---

Btw, thanks for the kind words in https://www.alanvardy.com/post/phoenix-cypress-tests - I'm glad that ex_check proves useful! :) And as to the main subject of your article, in one of my latest projects I've replaced Cypress with Playwright for Phoenix LiveView testing and it's great so far - way better than Cypress for testing things like WYSIWYG without JS hacks.

[alanvardy]

Thank you for the thoughtful response. I will dig in at the soonest opportunity.

I think that my usecase is a little bit strange. Ideally a code base never gets to the point where we have to choose which apps get to run in parallel but here we are!

I am going to check out Playwright as well, we have an admin interface that may be able to benefit from it.

Thanks and I hope you keep up the amazing work.

[alanvardy]

I have resolved enough of the test errors that I can (mostly) run it all in parallel now.

I essentially implemented Ecto Sandboxing for the caches, and switched out local adaptors for Redis etc with GenServers implementing the same behaviours.

Thank you, closing as this is a niche use-case that is no longer needed, thanks again!