This description is updated from a description in an older PR, but here should be all relevant information in one place, split into 4 packages:
1) Add new metrics to Distance
[x] mean_var_distn
[x] Mahalanobis
[ ] (ilisi, not important since doesnt work well)
For these, here is already draft implementations on the hackathon branch which you might want to copy or look into.
2) Add “three-way comparison option”
[x] Basically compute (with the metrics from Distance) prediction vs true perturbed, and prediction vs true unperturbed, and report the ratio
For this, here is already a draft implementation from the hackathon branch which you might want to copy or look into.
3) Bootstrapping
[x] Add “bootstrap” option which reports mean+stdev instead of just a point distance estimate
For this, here is already a draft PR which you might want to continue using or look into :)
4) General TODOs
[x] All metrics should be further tested for edge cases and sparse/non-sparse inputs.
[x] Some metrics would need to be inverted as they currently measure similarity not distance.
[x] Some metrics require a specific order of what is x,y,z input (control, perturbed, predicted perturbed) so that needs to be clear in the documentation.
Description of feature
The part of interest here is pertpy’s Distance metrics. The main file to be worked on is https://github.com/theislab/pertpy/tree/main/pertpy/tools/_distances
This description is updated from a description in an older PR, but here should be all relevant information in one place, split into 4 packages:
1) Add new metrics to Distance
For these, here is already draft implementations on the hackathon branch which you might want to copy or look into.
2) Add “three-way comparison option”
For this, here is already a draft implementation from the hackathon branch which you might want to copy or look into.
3) Bootstrapping
For this, here is already a draft PR which you might want to continue using or look into :)
4) General TODOs