search

opea-project / GenAIComps

GenAI components at micro-service level; GenAI service composer to create mega-service
Apache License 2.0
25 stars 64 forks source link

TEI not working on gaudi GPU #281

Closed endomorphosis closed 1 week ago

endomorphosis commented 2 weeks ago

the repo https://github.com/huggingface/tei-gaudi

does not work, I see the documents only show CPU support, but I would like a feature request to enable the GPU parts, so that I can integrate it into microsoft graphrag.

eero-t commented 2 weeks ago

Which HW you're trying to use? Gaudi is not a GPU, but AI accelerator...

endomorphosis commented 2 weeks ago

Im using a Gaudi2 system with 8 accelerators (i reflexively used the term GPU)

eduand-alvarez commented 2 weeks ago

can you try the code in this sample to see if it works for you? https://github.com/HabanaAI/Gaudi-tutorials/blob/main/PyTorch/RAG_Application/RAG-on-Intel-Gaudi.ipynb

endomorphosis commented 2 weeks ago

yes, i have already done so, that has the faulty repository

!git clone https://github.com/huggingface/tei-gaudi

On Mon, Jul 8, 2024 at 4:21 PM Eduardo Alvarez @.***> wrote:

can you try the code in this sample to see if it works for you? https://github.com/HabanaAI/Gaudi-tutorials/blob/main/PyTorch/RAG_Application/RAG-on-Intel-Gaudi.ipynb

— Reply to this email directly, view it on GitHub https://github.com/opea-project/GenAIComps/issues/281#issuecomment-2215510072, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAZ7LERBPV6KYASDVKRAMWTZLMNGPAVCNFSM6AAAAABKQBPSECVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMJVGUYTAMBXGI . You are receiving this because you authored the thread.Message ID: @.***>

louie-tsai commented 1 week ago

the repo https://github.com/huggingface/tei-gaudi

does not work, I see the documents only show CPU support, but I would like a feature request to enable the GPU parts, so that I can integrate it into microsoft graphrag.

Could you explain more what doesn't work for you in https://github.com/huggingface/tei-gaudi? Once you launch the local tei-gaudi service, do you get correct output from your curl request? image

endomorphosis commented 1 week ago

I have seen that the repository has been updated since i posted this ticket, and it is now working and stable with the migration from v1.2.2 to v1.4.0 see e.g. https://github.com/huggingface/tei-gaudi/commit/3db17966712cd17ef48b0e09e1778523239106da