mlexchange / mlex_highres_segmentation

A Dash interface for ML-based segmentation of user-annotated large multi-dimensional image data
Other
5 stars 4 forks source link

Implement clientside caching with Tiled #133

Open hannahker opened 11 months ago

hannahker commented 11 months ago

Performance improvement.

Wiebke commented 11 months ago

I don't have time to create a PR for this right now, but sharing the pieces that I have added: in data_utils.py:

In the section of module imports, add:

from tiled.client.cache import Cache
from tiled.client import show_logs

In the else statement to reach a remote Tiled server, add:

cache = Cache(
        capacity=500_000_000,  # bytes
        max_item_size=500_000,  # bytes
        filepath="data/tiled_cache.db",
        readonly=False,
    )

Add the cache to the client initaliztation:

client = from_uri(TILED_URI, api_key=API_KEY, cache=cache, timeout=httpx.Timeout(60.0))

To additionally show statistics on caching and timing of server requests etc, add:

# Show Tiled logs
if os.getenv("TILED_LOGS", False):
    show_logs()
hannahker commented 11 months ago

Note: How can we configure the timeout?