This does not leak hugely, but anyway. In Main, initialBB is never released.
TLD::processImage(), why cvtColor(img..., CV_RGB2GRAY), instead of CV_BGR2GRAY?
In Main, cv::Mat grey is created and given to tld through TLD::selectObject(). In that function a cv::Mat header is made and the actual data "owner" is grey in Main::doWork(). This is a bit difficult to digest at first but the danger is present when using TLD in other programs and the user does not realise that the grey image must exist throughout the life of an instance of TLD.