Bug in squidpy metrics computation, co-occurrence

[jimmymathews]

Encountered the following triggered sometimes when computed the co-occurrence scores:

| 08-24 06:01:06 [  INFO   ] spt ondemand start: Request: b'co-occurrence\x1dBreast cancer IMC - measurement\x1dCK\x1eKRT14\x1d\x1dCK\x1eKRT7\x1d\x1d100.0'
| 08-24 06:01:06 [  INFO   ] spt ondemand start: Request: b'co-occurrence\x1dBreast cancer IMC - measurement\x1dCK\x1eKRT14\x1d\x1dCK\x1eKRT7\x1d\x1d100.0'
| 08-24 06:01:06 [  DEBUG  ] spt ondemand start:149: ['Breast cancer IMC - measurement', 'CK\x1eKRT14', '', 'CK\x1eKRT7', '', '100.0']
| 08-24 06:01:06 [  DEBUG  ] spt ondemand start:149: ['Breast cancer IMC - measurement', 'CK\x1eKRT14', '', 'CK\x1eKRT7', '', '100.0']
| 08-24 06:01:06 [  DEBUG  ] ondemand.providers.pending_provider:27: Requesting computation.
| 08-24 06:01:06 [  DEBUG  ] ondemand.providers.pending_provider:85: Number of values possible to be computed: 169
| 08-24 06:01:06 [  DEBUG  ] ondemand.providers.pending_provider:111: Actual number computed: 20
| 08-24 06:01:06 [  DEBUG  ] ondemand.providers.pending_provider:35: Already pending.
| 08-24 06:01:06 [  DEBUG  ] ondemand.providers.squidpy_provider:158: Computed feature value of 296: BaselTMA_SP43_142_X5Y1, 0.9439820051193237
production-apiserver  | INFO:     172.27.0.1:44440 - "GET /request-spatial-metrics-computation-custom-phenotypes/?study=Breast%20cancer%20IMC&feature_class=co-occurrence&positive_marker=CK&positive_marker=KRT14&negative_marker=&positive_marker2=CK&positive_marker2=KRT7&negative_marker2=&radius=100 HTTP/1.0" 200 OK
| 08-24 06:01:07 [  DEBUG  ] ondemand.providers.squidpy_provider:158: Computed feature value of 296: BaselTMA_SP42_273_X6Y7, 0.9991911053657532
| 08-24 06:01:07 [  DEBUG  ] workflow.common.cell_df_indexer:36: Some KeyError. 1
| 08-24 06:01:07 [  DEBUG  ] workflow.common.cell_df_indexer:36: Some KeyError. (1, 1)
| Exception in thread Thread-39 (have_feature_computed):
| Traceback (most recent call last):
|   File "/usr/local/lib/python3.11/threading.py", line 1038, in _bootstrap_inner
|     self.run()
|   File "/usr/local/lib/python3.11/threading.py", line 975, in run
|     self._target(*self._args, **self._kwargs)
|   File "/usr/local/lib/python3.11/site-packages/spatialprofilingtoolbox/ondemand/providers/squidpy_provider.py", line 151, in have_feature_computed
|     value = compute_squidpy_metric_for_one_sample(
|             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|   File "/usr/local/lib/python3.11/site-packages/spatialprofilingtoolbox/workflow/common/squidpy.py", line 49, in compute_squidpy_metric_for_one_sample
|     return _summarize_co_occurrence(_co_occurrence(adata, radius))
|            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|   File "/usr/local/lib/python3.11/site-packages/spatialprofilingtoolbox/workflow/common/squidpy.py", line 64, in _summarize_co_occurrence
|     return float(occurrence_ratios[0][1][0])
|                  ~~~~~~~~~~~~~~~~~~~~^^^
| IndexError: list index out of range

Clearly the return data type coming from Squidpy needs to be validated even more before this step of processing.

For this issue, add a few guards to prevent this, passing null or None if nothing can be done.

Also make sure that in all the Squidpy metrics, we are gracefully handling None values as valid responses.

[CarlinLiao]

I've added some handlers to squidpy to return None instead of throwing errors in more cases, but I'm conflicted about this resolution. The errors indicate that the parameters with which you're calling squidpy are bad in some way, e.g., the cluster vector you've provided only has a single unique value because, for example, no cell in the sample has the DAPI channel. These cases should probably be handled outside of the squidpy logic, and we would rather use errors that squidpy throws to track down these missing edge cases.

[jimmymathews]

Yeah let's just check that the example that caused this particular error can be handled gracefully enough. So we should diagnose which edge case it is and handle it.

[jimmymathews]

This exact example works fine now. Some null values are returned, no error is thrown.