Add multi cpu example - Githubissues

UKPLab / sentence-transformers

State-of-the-Art Text Embeddings

https://www.sbert.net

Apache License 2.0

15.44k stars 2.5k forks source link

Add multi cpu example #3048

Open baslia opened 2 weeks ago

baslia commented 2 weeks ago

This pull request adds an example demonstrating how to use parallel computing with the tqdm package to show progress while encoding sentences using the SentenceTransformer model. The example leverages the multiprocessing module to parallelize the encoding process and tqdm to display a progress bar.

tomaarsen commented 2 weeks ago

Hello!

I believe you should also be able to do multi-CPU processing with this script: https://github.com/UKPLab/sentence-transformers/blob/master/examples/applications/computing-embeddings/computing_embeddings_multi_gpu.py Although please do let me know if your script is stronger than this one! I'm always open to approaches for faster inference.

I think perhaps the docstrings/comments in there are unclear. What do you think?

Tom Aarsen

baslia commented 2 weeks ago

Hello, Thank you for your reply. I am not sure that the code you mentioned is working on CPUs, especially on the M1 Macs. Since I often use this package, I wanted to give a working example of what I frequently use.

tomaarsen commented 2 weeks ago

I am not sure that the code you mentioned is working on CPUs, especially on the M1 Macs.

Oh, that is fair. I don't have access to a Mac, so it's a bit hard for me to test whether https://github.com/UKPLab/sentence-transformers/blob/master/examples/applications/computing-embeddings/computing_embeddings_multi_gpu.py already works. Could you perhaps give it a try?

Tom Aarsen

baslia commented 2 weeks ago

Sure, here is the error message. I am not sure about the exact root cause but I believe it is coming from not having a CUDA device.

2024-11-11 12:32:59 - Load pretrained SentenceTransformer: all-MiniLM-L6-v2
/opt/homebrew/Caskroom/miniforge/base/envs/py312/lib/python3.12/site-packages/huggingface_hub/file_download.py:1150: FutureWarning:
`resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
/opt/homebrew/Caskroom/miniforge/base/envs/py312/lib/python3.12/multiprocessing/resource_tracker.py:254: UserWarning:
resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
Process finished with exit code 139 (interrupted by signal 11:SIGSEGV)