search

qurator-spk / eynollah

Document Layout Analysis
Apache License 2.0
328 stars 26 forks source link

Question: How to enable GPU in eynollah? i dont see better performance with my gpu #123

Open FabricioTeran opened 4 months ago

FabricioTeran commented 4 months ago

Hi, how can i enable GPU mode in eynollah? i have installed all tools needed for cuda, running nvidia-smi works fine, but i dont see a better performance of eynollah... my card is a gtx 950m with 480 cuda cores, i have monitored its performance but its usage doesnt increase from 0%

The versions of the tools are: ubuntu 22.04, Python 3.9.13, tensorflow 2.12, cuda toolkit 11.8, cuDNN 8.6, and the drivers of my graphic card are version 520 and also supports cuda 12

cneud commented 4 months ago

Thank you for the question. Your GPU/CUDA/cuDNN setup seems valid and there is no other specific action needed to have Eynollah make use of the GPU. However, if I am not mistaken the gtx 950m is a notebook GPU and the speed gains in inference will likely be rather insignificant.

On the other hand, the Eynollah code is also not currently optimized to really leverage a GPU to its full extent. Several processing steps are currently implemented for CPU only and the GPU needs to wait for these, which is why nvidia-smi is probably only showing utilization for very short durations (at least in our case - see also https://github.com/qurator-spk/eynollah/issues/84). We hope to improve this over the course of the coming months, but as this requires considerable refactoring of the code base and possibly even the re-training of models, I'm afraid it will take time.

FabricioTeran commented 4 months ago

Ok thank you for the fast response, if you can make a refactoring it would be very helpfull to me to being able to contribute on some features (since im not from the AI filed, but a full stack developer)

cneud commented 4 months ago

Thanks! We are planning to make a new release in the course of next week to fix some pressing issues regarding integration of Eynollah in OCR-D - which should then also serve as a suitable basis for a more extensive refactoring process. Keep an eye on the most recently updated branches and I will also try and keep you updated here in due course.