bonsai-rx / deeplabcut

A Bonsai interface for real-time markerless pose estimation using DeepLabCut
MIT License
47 stars 9 forks source link

Incorrect color channel order in DetectPose #6

Closed glopesdev closed 2 years ago

glopesdev commented 3 years ago

The trained DLC network might expect color images to be in the mirrored RGB format rather than BGR which is the default for OpenCV. This has on occasion caused subtle, but persistent problems in the accuracy of the inference output. This only affects color images and is more pronounced on datasets with a high color bias.

There are a few issues with built-in color conversion though: 1) there is no way of knowing definitely whether a given image is in the RGB or BGR formats (both have 3 channels); 2) it would break the behaviour of workflows which are already passing images in the correct format; 3) if by chance your input image source is receiving images in the correct color format, then you would be forced to not only invert the channels out of the node, but pay the extra performance penalty of the built-in node conversion.

Also, one can always simply train the network with flipped color format and in principle this would produce a model which does not require color flipping.