search

nadeemlab / SPT

Spatial profiling toolbox for spatial characterization of tumor immune microenvironment in multiplex images (https://oncopathtk.org)
https://oncopathtk.org
Other
21 stars 2 forks source link

Implement a refresh signal for the ondemand containers #318

Closed jimmymathews closed 2 months ago

jimmymathews commented 4 months ago

The ondemand containers (counts service, squidpy metrics service, etc.) currently load up their in-memory data stores on initialization. It takes about 5 minutes for our current list of ~6 datasets.

However, during those 5 minutes, these services are not available. Moreover, if datasets are added to the database incrementally over time, a lot of computational effort is wasted re-loading all datasets when only one dataset (the new one) needs to be loaded.

For this issue, implement a signal that can be received by the ondemand container TCP server that causes it to intelligently update its data store. That is, it should fork a background task to add to its cache any new datasets it finds in the database, while leaving the other datasets in memory as they are. To force an update of a specific already-existing dataset, perhaps a study name parameter can be supplied.

jimmymathews commented 4 months ago

As part of this issue, research the most sensible method of delivering this signal. Of course, it should not be triggered by an exposed API endpoint, since the API server is public. Probably there is some relatively simple mechanism that is available for kubernetes pods.

jimmymathews commented 2 months ago

This is obsolete.