ML4GLand / EUGENe

Elucidating the Utility of Genomic Elements with Neural Nets
MIT License
65 stars 4 forks source link

`configure_metrics` default `kwargs` causes errors as default with certain metrics #45

Open X02cinnamondirty opened 9 months ago

X02cinnamondirty commented 9 months ago
def configure_metrics(self, metric, metric_kwargs):
    """Configure metrics

    Keeping this a function allows for the metric to be reconfigured
    in inherited classes TODO: add support for multiple metrics

    Returns:
    ----------
    torchmetrics.Metric:
        metric
    """
    metric_name = DEFAULT_TASK_METRICS[self.task] if metric is None else metric
    metric_kwargs = (
        metric_kwargs
        if metric_kwargs is not None
        else DEFAULT_METRIC_KWARGS[self.task]
    )
    metric = METRIC_REGISTRY[metric_name](
        num_outputs_=self.output_dim, **metric_kwargs
    )
    return metric, metric_kwargs, metric_name

num_outputs may have some fault

adamklie commented 8 months ago

Hi!

Thanks for raising an issue on this. This is a known problem with the way we currently calculate metrics in the SequenceModule class. We are working on a refactor of the LightningModules here (https://github.com/ML4GLand/SeqModels/tree/dev).

For now, you can workaround this by including arguments in the metric_kwargs for the SequenceModule. You can find an example of that here: https://github.com/ML4GLand/use_cases/blob/dev/Basset/train_eugene.ipynb. I'm happy to assist if you have a specific use case that's causing an error.