search

openkinome / kinoml

Structure-informed machine learning for kinase modeling
https://openkinome.org/kinoml/
MIT License
51 stars 21 forks source link

Memory performance #16

Closed jaimergp closed 4 years ago

jaimergp commented 4 years ago

Converting our DatasetProviders to native torch.Dataset objects involve creating new torch.Tensors that can easily surpass available memory.

There are some strategies we can investigate to robustly minimize this issue, but it mainly involves using minibatches and compatible mechanisms. This requires Dataloader adapters, which would allow to convert to tensor only on __getindex__ access.

jaimergp commented 4 years ago

Now part of #8