Error when trying to track simple hand movement

[DaBue93]

Hei @all, first of all: finally there is the update to use Bonsai with tensorflow 2.x -THANK YOU! I have labeled a very simple video on which I would like to have my hand tracked. However, I now get the following error: these are my current DLC and tf versions: does anyone have any idea how I can solve the problem here?

Further information: Running System: Windows 20 Graphics card: Nvidia QuadroP400 Bonsai version 2.6.3

Thanks :)

[glopesdev]

@DaJe93 Thanks for trying the new update. We have now successfully used it to run several different models, including single- and multi-animal trained on different computers and GPU combinations, and it seems to work fine so far.

However, TensorFlowSharp support for TF 2.8.0 is highly experimental and there might be operations in which the model definition breaks.

Any chance you can share the .pb file for this small model so we can try it and inspect it to see what might be the issue?

EDIT: Also including the config.yaml file and pose_cfg.yaml would be very useful as well, so we can look at which network you used to train the model and if there are any relevant parameters for us to be aware of to reproduce this.

[glopesdev]

@DaJe93 this model seems to work fine on my setup. I suspect given the error message that there might be a mismatch between native tensor flow versions.

Just to confirm, have you updated the tensorflow.dll in Bonsai to also use TF 2.8? There have been a few changes, so please check the updated README for details, specifically the DLL name no longer needs the lib prefix and the URL for downloading the native binaries is different.

[kvnlxm]

@DaJe93 and I solved the problem. We were following the instructions of Bonsai x DLC very closely, but some extra steps were needed.

For anyone finding themselves having the same error, it was mainly tensorflow related. We used the tensorflow version 2.8.0 with CUDA 11.2 and cuDNN 8.1.0. While the Bonsai documentation specifies these versions, we had to manually add cudart64_100.dll,cufft64_100.dll, cusolver64_100.dll, curand64_100.dll and cusparse64_100.dll into the .bin folder of the CUDA installment. You can find the .dll files here.

Moreover, one needs to make sure to set the PATH variables correctly after installing CUDA and downloading cuDNN. I found this guide particularly helpful.

Finally, we also needed to update the Bonsai DLC packages. Please note that now the DLC node is not called DetectPose anymore but PredictPose. @glopesdev it would be great if this could be updated in the Bonsai x DLC repository.

Cheers, Kevin