Cell level data

[jimmymathews]

Create a new API endpoint, to support the virtual slide viewer, furnishing cell-level position and phenotype information.

[jimmymathews]

This is implemented in issue298. Testing now with pushed containers, doing dataset re-import (required because of recent merge of caching stuff).

[jimmymathews]

It takes anywhere from 5 to 20 seconds when the cell data payload is around 10MB (due to internal querying etc.). A simple calculation seems to show that it's around 75 to 100 MB per 1 million cells, which makes it seems reasonable to cache the payloads for all instances of the cell data queries using the simple cache wrapper used elsewhere. This would bring the response time down considerably (after the first occurrence, of course).

[jimmymathews]

There is also an issue where non-existent samples trigger crashing keyerror rather than creating an error response.

There also seems to be a payload limit at 1million characters somewhere in the chain, probably the custom tcp server.

[jimmymathews]

The above have been addressed. There is still a substantial delay, perhaps caused by FastAPI's response validator. There seems to be a delay even in handling the request parsing, as the ondemand container often will not receive the request for 10 or more seconds. I also noticed that FastAPI sometimes parses the query incorrectly the first time, failing to use the & character correctly as a delimiter. Example:

Study name invalid: "urothelial ici&sample=uc10002"

for query

curl 'http://oncopathtk.org/api/cell-data/?study=Urothelial+ICI&sample=UC10002'

[jimmymathews]

(Nevermind the delay issue, I think it is a problem with the way I'm using curl sometimes.)