search

bonsai-rx / deeplabcut

A Bonsai interface for real-time markerless pose estimation using DeepLabCut
MIT License
47 stars 9 forks source link

PylonCapture lags when combined with PredictPose(DLC) #16

Open kvnlxm opened 2 years ago

kvnlxm commented 2 years ago

Hi,

we are currently experiencing an issue with the PredictPose(DLC) node for real-time tracking of DLC marker (Previously DetectPose(DLC)) when combined with the PylonCapture(Pylon) node. The image output lags about 7 seconds and hence, real time pose-estimation is not possible. In Figure 1 PylonCapture(Pylon) works in real time (without PredicPose(DLC), while in the setting of Figure 2 it lags. Note that we already tried to remove the conversion node, which does not cause the problem.

We also used the CameraCapture(Vision) node (or VideoCaptureDevice(Vision) node), which works in real-time combined with PredictPose(DLC), but we cannot retrieve the pylon camera features settings, which is important.

Does anyone face the same problem and is there already a work around. My guess is that there is some sort of bug within the PylonCapture(Pylon)node and maybe someone from the Bonsai Team can help with that.

Cheers, Kevin

Bonsai_real_time Figure 1: PylonCapture works in real-time Bonsai_lag_time Figure 2: PylonCapture lags about 7 seconds

kvnlxm commented 2 years ago

While I am still unsure why this problem exist with the PylonCapture(Pylon) method, we empirically found out that after a certain amount of time (about 4 minutes with our Pylon settings), the camera feed was finally live. This is good for us at the moment, but I still would like to know why there is such a difference between CamareCapture(Vision) and PylonCapture(Pylon) .. Or is there a way to retrieve the Pylon features for the camera also with the CamareCapture(Vision) method?

Thanks, Kevin

trose-neuro commented 1 year ago

Hi all! Following up on this: Have there been any developments? Currently with the latest Pylon, PylonCapture node, Bonsai version, DLC live install this is still the case and therefore hardly usable...

Best Tobias

trose-neuro commented 1 year ago

To follow up on this: Turns out this indeed was a simple GPU load issue. Binning on the camera solved this.