search

Marker-Inc-Korea / AutoRAG

AutoML tool for RAG
https://auto-rag.com/
Apache License 2.0
2.7k stars 211 forks source link

Support GPU at passage reranker modules #138

Closed vkehfdl1 closed 9 months ago

vkehfdl1 commented 9 months ago

I wonder tart support CUDA well... And other reranker modules has not tested yet.

vkehfdl1 commented 9 months ago

problem at upr.py I got this error when I run it at gpu server.

RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument index in method wrapper_CUDA__index_select)

attention_mask_view = context_attention_mask[i: i + shard_size]                                                        │
                             │   127 │   │   decoder_tensor_view = question_tensor[i: i + shard_size]                                                               │
                             │   128 │   │   with torch.no_grad():                                                                                                  │
                             │ ❱ 129 │   │   │   logits = model(input_ids=encoder_tensor_view,                                                                      │
                             │   130 │   │   │   │   │   │      attention_mask=attention_mask_view,                                                                 │
                             │   131 │   │   │   │   │   │      labels=decoder_tensor_view).logits

Phind Answer: https://www.phind.com/search?cache=fuqrhd654zolq5sp3m1wps3e