saalfeldlab / render

Render transformed image tiles
GNU General Public License v2.0
34 stars 32 forks source link

Newsolver improve spark solver parallelization #158

Closed trautmane closed 11 months ago

trautmane commented 11 months ago

Hi @minnerbe - unfortunately I mixed two separate changes in this branch when I should have split things up to simplify review. The original purpose for the branch was to allow blocks for different stacks to be solved concurrently (something we need for multi-SEM). The solve changes were made in commit 3121f59.

I then appended another set of changes to merge missing edge connection detection (something we use for FIBSEM stacks) into connected cluster detection code. In the past, I've run both tools serially even though they rely upon the same source match data. This change set merges the two tools together and allows them to be run as a prerequisite step to the alignment solve in the Spark alignment pipeline.

Let's review/discuss everything when you have time.