Running metric_melt several times

[michaelbornholdt]

This is an optimization issue. When running metrics such as precision-recall or enrichment, like in the demo book: https://github.com/cytomining/cytominer-eval/blob/master/demos/CellPainting_Demo.ipynb You call evaluate several times to calculate the metric at different value of p e.g.

for p in np.arange(0.99, 0.96, -0.01):
    r = evaluate(
        profiles=df,
        features=features,
        meta_features=meta_features,
        replicate_groups=["Metadata_gene_name"],
        operation="enrichment",
        similarity_metric="pearson",
        enrichment_percentile = p,
    )
    result.append(r)

However, this calls the function cytominer_eval.transform.get_pairwise_metricseveral times (once per call of evaluate). This is not necessary since the metrics can be retrieved from the same similarity_melted_df!!

We need to adapt the function evaluate such that it either only calculates the pairs once, when called several times. Or we maybe just need to change the demos to show that when calculating several values, you must not use evaluate for them.

What are your thoughts @gwaygenomics

[gwaybio]

nice! Love this. Maybe, for example, enrichment_percentile can accept a list?

[michaelbornholdt]

mh, yea we could to that. Same goes for Precision recall then. It could accept a list of k's

You think that's the best solution?

[gwaybio]

yep

On Fri, Apr 23, 2021 at 3:40 PM Michael Bornholdt @.***> wrote:

mh, yea we could to that. Same goes for Precision recall then. It could accept a list of k's

You think that's the best solution?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/cytomining/cytominer-eval/issues/51#issuecomment-825878282, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABYADBON2NUUONUOQB4ZCKDTKHEKPANCNFSM43O7DOUA .