HazyResearch / hyena-dna

Official implementation for HyenaDNA, a long-range genomic foundation model built with Hyena
https://arxiv.org/abs/2306.15794
Apache License 2.0
574 stars 82 forks source link

Fix moving torchmetrics to cuda device 0 #13

Closed awaelchli closed 1 year ago

awaelchli commented 1 year ago

This PR removes .to("cuda") calls that cause the metric to be forcefully placed on GPU 0, causing a memory imbalance. A torchmetrics Metric is a nn.Module, and therefore it will be automatically placed on the right device when used in a LightningModule together with the Trainer.

The best practice in Lightning is to not manipulate module or tensor placement unless there is a good reason for it.