search

segment-any-text / wtpsplit

Toolkit to segment text into sentences or other semantic units in a robust, efficient and adaptable way.
MIT License
624 stars 36 forks source link

sat-12l-sm running on GPU #120

Closed Randwow closed 2 days ago

Randwow commented 2 weeks ago

Hi @bminixhofer,

I'm trying to use sat-12l-sm on GPU with the following code:

MODEL_NAME = "sat-12l-sm" sat = SaT(MODEL_NAME) sat.to("cuda")

However, when I run nvidia-smi in the terminal, it doesn't show any usage of the GPU, and it seems that the GPU is not being utilized. Could you please provide any guidance or suggestions on how to ensure that the model is actually using the GPU?

Thank you!

markus583 commented 2 weeks ago

Hi, this is a bit odd, and should not be the case. Is your torch properly set up? Did you check with nvidia-smi also after calling sat.split("some text")? I just tried it myself and it works as intended. The time needed is also much lower after doing sat.cuda(). I suggest you compare time needed to segment some sentences on CPU and GPU; the latter should be an order of magnitude faster.

Randwow commented 2 weeks ago

Hi, yes I will check thank you )