search

Lightning-AI / torchmetrics

Torchmetrics - Machine learning metrics for distributed, scalable PyTorch applications.
https://lightning.ai/docs/torchmetrics/
Apache License 2.0
2.1k stars 402 forks source link

Support for Torchvision Datapoints? #1846

Open edwardclem opened 1 year ago

edwardclem commented 1 year ago

🚀 Feature

Hi everyone! Curious on everyone's thoughts here - are there any plans to support the new torchvision datapoints API as input for vision-centric metrics (mostly mAP)? The alternative is just to convert datapoints back to tensors before sending them through the metrics, which certainly isn't bad.

If this is in the roadmap, happy to take a stab at it.

SkafteNicki commented 1 year ago

Hi @edwardclem, thanks for raising this issue. I was actually not aware of the new datapoints API, so thanks for making me aware of that.

My first troughs are that we are not going to support this directly, that said (without having tested this in anyway I would expect that most metrics actually should work out of box because the datapoints are just torch.tensor and will be auto unpacked into normal tensors whenever they encounter some operation.

For mAP specifically it maybe makes sense to support BoundingBox and Mask datapoints, but I would then wait until this feature is stable (and has seen some adoption), so it will first be something we want to consider in the future.

cc: @justusschock and @Borda for opinions.

Borda commented 1 year ago

For mAP specifically it maybe makes sense to support BoundingBox and Mask datapoints, but I would then wait until this feature is stable (and has seen some adoption), so it will first be something we want to consider in the future.

Sounds good to me... @edwardclem, Would you be interested in implementing it?

edwardclem commented 1 year ago

Sure, happy to! I'll hold off on implementing until the torchvision feature is a bit more mature.