search

weaviate / t2v-transformers-models

This is the repo for the container that holds the models for the text2vec-transformers module
BSD 3-Clause "New" or "Revised" License
40 stars 27 forks source link

Add support for PyTorch GPU memory limit #41

Closed kcm closed 1 year ago

kcm commented 1 year ago

Set CUDA_MEMORY_PCT between 0.0-1.0 inclusive to limit memory use in PyTorch. Defaults to 1.0.

This is useful, for example, in shared environments such as AWS's Virtual GPU device plugin.

Addresses #39

kcm commented 1 year ago

Tested on local Linux running PyTorch 1.13.0 with CUDA12. Should be similar in EKS et al.