jni / zarpaint

Paint segmentations directly to on-disk/remote zarr arrays
BSD 3-Clause "New" or "Revised" License
14 stars 8 forks source link

Allow asynchronous behaviour for long running tasks #42

Open GenevieveBuckley opened 1 year ago

GenevieveBuckley commented 1 year ago

Because zarpaint handles large datasets, some of the computations can be long running. Making these tasks asynchronous would improve useability, since the napari viewer would no longer freeze until the computation is complete.

A solution could be to use the napari @thread_worker decorator, ideally in combination with a progress bar so the user will know when the computation is complete.

The napari threading guide shows this example:

import napari
import numpy as np

from napari.qt.threading import thread_worker

@thread_worker
def average_large_image():
    return np.random.rand(1024, 512, 512).mean(0)

viewer = napari.Viewer()
worker = average_large_image()  # create "worker" object
worker.returned.connect(viewer.add_image)  # connect callback functions
worker.start()  # start the thread!
napari.run()

And Talley explains several different ways to update the progress bar from a thread worker in this thread on forum.sc:

@thread_worker(progress={"total": 10})
def my_func():
    ...

# or you can use:
def make_worker():
    n = figure_out_number_of_iterations()
    work = thread_worker(my_func, progress={"total": n})
    ...

# edit you can also use napari.qt.create_worker
from napari.qt import create_worker

def long_function(duration):
    import time
    time.sleep(duration)

worker = create_worker(long_function, 10, _progress={'total', n})
worker.start()
GenevieveBuckley commented 1 year ago

One specific example of a slow running process is slice interpolation, where writing new label data to the zarr file on disk can be time consuming https://github.com/jni/zarpaint/pull/32. Adding asynchronous behaviour to this would be a good follow up PR.

However I wanted this issue to be broader than just that one example. It would be good to identify all the places where this might be a helpful pattern to use.