nadeemlab / SPT

Spatial profiling toolbox for spatial characterization of tumor immune microenvironment in multiplex images
https://oncopathtk.org
Other
21 stars 2 forks source link

Compressed blobs caching #354

Closed jimmymathews closed 1 month ago

jimmymathews commented 2 months ago

For this issue, let's decide on a compression system and caching system for the binary blobs with phenotype data etc.

We need to decide when compression happens and where/when the results are stored, what is served by the API, etc.

@franciscouzo did some experiments with compression ratios and algorithms, and mentioned https://github.com/google/ngx_brotli .

This is an nginx plugin that supports brotli compression (and caching, as usual?). We could insert this between the FastAPI service and clients. In the current architecture, an off-the-shelf ingress nginx is used as a load balancer, so we probably shouldn't alter the configuration of that nginx component.

But we could just add a new service/pod to the helm configuration?

Needs some research.