Check picking with GPU

[DaloroAT]

Hello @kerim371

I added GPU support to pick first breaks as we discussed earlier. Could you check it pls? Installation guide is here.

[kerim371]

Hi,

I've installed CUDA TOOLKIT 11.8, cuDNN 8.9.0 and ZLIBWINAPI. I hope my drivers are uptodate. Added cuDNN/bin and ZLIBWINAPI/bin folder to Path end var. I've also tried to set CUDA_VISIBLE_DEVICES=1 or CUDA_VISIBLE_DEVICES=0 but I always get warnings:

2023-06-21 14:24:17.1904848 [E:onnxruntime:Default, provider_bridge_ort.cc:1304 onnxruntime::TryGetProviderInfo_CUDA] D:\a\_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1106 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "C:\Users\kerim\dev\first_breaks_picking\venv\lib\site-packages\onnxruntime\capi\onnxruntime_providers_cuda.dll"

2023-06-21 14:24:17.2057833 [W:onnxruntime:Default, onnxruntime_pybind_state.cc:541 onnxruntime::python::CreateExecutionProviderInstance] Failed to create CUDAExecutionProvider. Please reference https://onnxruntime.ai/docs/reference/execution-providers/CUDA-ExecutionProvider.html#requirements to ensure all dependencies are met.

I understand that most likely it can't find path to those .dll.

I still use firstbreaks app for testing.

Could you please tell me how can I launch this application from sources (without installing it with pip install -e .[gpu])? That could accelerate my reasearches.

I'm going to try something like os.add_dll_directory()

[kerim371]

By the way I use laptop so I have 2 video cards. CUDA_VISIBLE_DEVICES is used to select between them I think. Windows 11

[DaloroAT]

Thank you for your reply!

By the way I use laptop so I have 2 video cards. CUDA_VISIBLE_DEVICES is used to select between them I think. Windows 11

Are they both from NVIDIA? Usually, laptops have integrated (as part of CPU) graphic card from Intel or AMD and discrete (not a part of CPU, it's physically another chip) graphic card from NVIDIA. Two discrete GPUs are an extremely rare case for laptops. Btw you can accelerate picking only with discrete GPU. Do you have 2 discrete GPUs?

[DaloroAT]

Added cuDNN/bin and ZLIBWINAPI/bin folder to Path end var

Could you pls open PowerShell, type env:$PATH, and send me the result? Or only parts related to CUDA and CuDNN to keep other parts of your PATH private.

[kerim371]

Right, my integrated is Intel and discrete is NVIDIA.

[kerim371]

$Env:PATH                  
C:\Users\kerim\dev\first_breaks_picking\venv\Scripts;C:\Users\kerim\AppData\Local\Programs\Microsoft VS Code\bin;C:\Users\kerim\.dotnet\tools;C:\Users\kerim\AppData\Local\Microsoft\WindowsApps;;C:\Users\kerim\dev\cudnn\8.9.0\bin;C:\Users\kerim\dev\ZLIBWAPI\dll_x64

[DaloroAT]

Could you please tell me how can I launch this application from sources (without installing it with pip install -e .[gpu])? That could accelerate my reasearches.

How do you want to use the library?

You want to do your own changes:

• git clone ...
• Enter to cloned repo
• pip install -e ".[gpu]"
• first-breaks-picking app

You want to use it as a user without changes:

• pip install -U first-breaks-picking[gpu]
• first-breaks-picking app

[DaloroAT]

You want to do your own changes

for this case to avoid cloning the repo again and again, you can set a remote source for main branch as my repo

https://stackoverflow.com/a/3903835

[DaloroAT]

$Env:PATH                  
C:\Users\kerim\dev\first_breaks_picking\venv\Scripts;C:\Users\kerim\AppData\Local\Programs\Microsoft VS Code\bin;C:\Users\kerim\.dotnet\tools;C:\Users\kerim\AppData\Local\Microsoft\WindowsApps;;C:\Users\kerim\dev\cudnn\8.9.0\bin;C:\Users\kerim\dev\ZLIBWAPI\dll_x64

By some reason, CUDA 11.8 is not in your PATH. Could you please find out where is CUDA and add it?

[DaloroAT]

CUDA_VISIBLE_DEVICES is used to select between them I think

Usually, you don't need to change this env for a single GPU machine. For your setup, the only valid value is 0 if you want to use CUDA.

[kerim371]

Could you please tell me how can I launch this application from sources (without installing it with pip install -e .[gpu])? That could accelerate my reasearches.

How do you want to use the library?

You want to do your own changes:

• git clone ...
• Enter to cloned repo
• pip install -e ".[gpu]"
• first-breaks-picking app

You want to use it as a user without changes:

• pip install -U first-breaks-picking[gpu]
• first-breaks-picking app

I meant I want to make some changes and avoid pip install <> but instead I would run the application using python app.py can I?

[kerim371]

$Env:PATH                  
C:\Users\kerim\dev\first_breaks_picking\venv\Scripts;C:\Users\kerim\AppData\Local\Programs\Microsoft VS Code\bin;C:\Users\kerim\.dotnet\tools;C:\Users\kerim\AppData\Local\Microsoft\WindowsApps;;C:\Users\kerim\dev\cudnn\8.9.0\bin;C:\Users\kerim\dev\ZLIBWAPI\dll_x64

By some reason, CUDA 11.8 is not in your PATH. Could you please find out where is CUDA and add it?

That solved the problem thank you!

For the future reference: after NVIDIA Toolkit installation new env vars appeared CUDA_PATH and CUDA_PATH_V11_8 with value C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8 but in the same time I needed to manually add bin folder to the PATH env, i.e.: $env:Path += ";C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\bin"

[DaloroAT]

I meant I want to make some changes and avoid pip install <> but instead I would run the application using python app.py can I?

If you have a repo locally, you can install it with pip install -e ".[gpu]". Flag -e force pip use your local files, instead of .../site-packages/.... You can do changes to this folder without reinstalling it.

[kerim371]

It seems that it works much faster with GPU. Maybe 10 time faster.

I will do some tests on my data using jupyter notebook to compare the perfomance and estimate how fast it is.

[DaloroAT]

but in the same time I needed to manually add bin folder to the PATH

For me, it was done automatically. But for you, some problems appear. So thank you, I will extend README with what to check and how to fix it.

[DaloroAT]

It seems that it works much faster with GPU. Maybe 10 time faster.

I will do some tests on my data using jupyter notebook to compare the perfomance and estimate how fast it is.

Great!

The difference depends on the hardware, in my case difference is ~15x. You can also modify batch_size to fit your VRAM, it will decrease computation time. But try to do it gradually.

[DaloroAT]

It's out of topic, but I also added the possibility to change picks of the model by left clicks