Closed reycn closed 3 days ago
This PR supports automatic device selection supported by pytorch.
torch.cuda.is_available()
cpu
Test passed on RTX3080, using the following environment: conda create -n pdf python=3.11 pytorch torchvision transformers ipywidgets ipykernel -y -c nvidia -c conda-forge
conda create -n pdf python=3.11 pytorch torchvision transformers ipywidgets ipykernel -y -c nvidia -c conda-forge
Description
This PR supports automatic device selection supported by pytorch.
torch.cuda.is_available()
, we use GPU for inferencecpu
) yet in inference, GPU is utilized.cpu
for compatibility.Related Issue
Test
Test passed on RTX3080, using the following environment:
conda create -n pdf python=3.11 pytorch torchvision transformers ipywidgets ipykernel -y -c nvidia -c conda-forge