search

infiniflow / ragflow

RAGFlow is an open-source RAG (Retrieval-Augmented Generation) engine based on deep document understanding.
https://ragflow.io
Apache License 2.0
17.68k stars 1.8k forks source link

[Question]: Why is the TSR model using GPU slower than CPU speed, but the layout model working properly? #475

Open chouife opened 5 months ago

chouife commented 5 months ago

Describe your problem

在使用t_ocr.py和t_recognizer.py时发现源码中默认使用cpu，后改成使用gpu处理，发现ocr模型和tsr模型都出现了比使用cpu处理速度还慢的情况（没有任何报错或警告出现，且gpu也有占用），但是layout模型正常，重复测试过很多pdf都是一样的问题。请问下是什么原因了？ 以下是英语翻译（由于英语不好，直接使用的百度翻译）： I found that the source code defaults to using CPU when using t-ocr. py and t-recognizer. py. Later on, I switched to using GPU for processing and found that both the OCR model and the TSR model had slower processing speeds than using CPU (no errors or warnings appeared, and GPU was also occupied), but the layout model was normal. I have repeatedly tested many PDFs and found the same problem. May I ask what the reason is?

KevinHuSh commented 5 months ago

It's onnx model. You may install onnxruntime-gpu. Have a try.

chouife commented 5 months ago

感谢你的回复！没太理解你的意思，能否具体一点了？我确实是安装的onnxruntime-gpu，所用的版本也是requirements.txt中对应的版本。Thank you for your reply! I didn't quite understand your meaning, could you be more specific? I did indeed install onnxruntime gpu, and the version used is also the corresponding version in requirements. txt. e4427e0e-0bc5-4648-a65e-3a247ec08947

chouife commented 5 months ago

目前只是修改了ocr.py和recognizer.py中的如图所示的代码，将"False and"删除以启用GPU推理，并在对应的providers中添加了"CPUExecutionProvider"。模型文件是从huggingface中下载的，其余代码并未改动，测试时使用t_ocr.py和t_recognizer.py，仅修改待解析的文件路径。 At present, only the code shown in the figure has been modified in ocr. py and recognizer. py, removing "False and" to enable GPU inference, and adding "CPUExecutionProvider" to the corresponding providers. The model file was downloaded from huggingface, and the rest of the code has not been changed. During testing, t-ocr. py and t-recognizer. py were used, and only the path to the file to be parsed was modified 21ca0bfd-6102-465e-8d7d-6c74baef0c93

KevinHuSh commented 5 months ago

I have not had other solution here yet. Maybe you can google it.

chouife commented 5 months ago

感谢回复！我看到微信群里有人和我是一样的情况，你的意思是你们本地测试是正常的吗？ Thank you for your reply! I saw someone in the WeChat group in the same situation as me. Do you mean that your local testing is normal?

Tendo33 commented 2 months ago

感谢回复！我看到微信群里有人和我是一样的情况，你的意思是你们本地测试是正常的吗？ Thank you for your reply! I saw someone in the WeChat group in the same situation as me. Do you mean that your local testing is normal?

请问你有找到解决方法吗，即使提高了batch 我用GPU也是速度变慢了许多

dafen12 commented 5 days ago

upgrade onnxruntime-gpu-1.19.2。it works！