cv::ShapeContextDistanceExtractor: platform-dependant return values?

Hello everybody,

I might have found a problem concerning the cv::ShapeContextDistanceExtractor-class. I have observed the behavior in a unittest, that for a given image and the extracted contours (via cv::findContours()), different distances are calculated upon an identity-check of the contour, depending on the platform the software is executed on. In contrast to cv::HausdorffDistanceExtractor, an identity check does not return a plain-value of 0.0d, but instead a very small difference in the range of ca. 0.0005.

The first platform is my notebook, a Lenovo ThinkPad X250 running openSUSE Leap 15.0. The second platform is my university's gitlab-runner, which runs some unittests in its CI/CD-pipeline. The baseimage for the gitlab-runner is opensuse/leap:15.0 and the versions of the installed packages are the same as on my notebook.

I am just wondering, why the same software, compiled and executed on similar but different systems makes cv::ShapeDistanceExtractor::computeDistance() return different values, when it is instantiated with a cv::ShapeContextDistanceExtractor, but all the input and preprocessing stays the same and is deterministic?

System information (version)

OpenCV => libopencv3_3: 3.3.1-lp150.4.3.1
Operating System / Platform => openSUSE Leap 15.0 x64 (uname -r: 4.12.14-lp150.12.48-default)
Compiler => g++ -v: [...] gcc version 7.4.0 (SUSE Linux)

Detailed description

(See above and comments in the unittest.)

Steps to reproduce

Here one can find a unittest which demonstrates the problem. It is written for libboost_test1_66_0. The used image can be downloaded below the unittest. Just in case github performs some kind of compression, I have added the checksum in the unittest.

In the unittest, the same image (see below) is loaded twice to different cv::Mat-objects. Then the contours of each Mat is calculated and all the defining points are merged into one big contour-vector. Afterwards, sdeptr->computeDistance() is invoked to calculate the shapecontext-distance.

Unittest

    BOOST_AUTO_TEST_CASE(BUGREPORT_SHAPECONTEXT) {
        // Load same image two times.
        // Image has checksum sha256 246cb53b310a1b51dd5f428e061990408675798ea1f71a1bf85ad41003e6ebff
        cv::Mat m1 = cv::imread("bugreport_shapecontext_skeleton.png", cv::IMREAD_GRAYSCALE);
        cv::Mat m2 = cv::imread("bugreport_shapecontext_skeleton.png", cv::IMREAD_GRAYSCALE);

        // Create vectors to store data from findContours
        std::vector<std::vector<cv::Point>> contours1, contours2;
        std::vector<cv::Vec4i> hierarchy1, hierarchy2;

        // Find contours
        cv::findContours(m1, contours1, hierarchy1, cv::RETR_LIST, cv::CHAIN_APPROX_NONE);
        cv::findContours(m2, contours2, hierarchy2, cv::RETR_LIST, cv::CHAIN_APPROX_NONE);

        // Merge all points from obtained vectors into one vector
        std::vector<cv::Point> merged_contour1, merged_contour2;
        for(auto vec1 : contours1) {
            for (auto pt1 : vec1) {
                merged_contour1.push_back(pt1);
            }
        }
        for(auto vec2 : contours2) {
            for (auto pt2 : vec2) {
                merged_contour2.push_back(pt2);
            }
        }

        // Looks strange, but there is an `==` operator for vectors.
        BOOST_REQUIRE(merged_contour1 == merged_contour2);
        BOOST_REQUIRE(merged_contour1.size() == merged_contour2.size());

        // Calculate shapecontextdistance:
        cv::Ptr<cv::ShapeContextDistanceExtractor> sdeptr = cv::createShapeContextDistanceExtractor();

        double calculated_shapecontextdistance = sdeptr->computeDistance(merged_contour1, merged_contour2);
        const double shapecontextdistance_on_my_x250 = 0.00076877244282513857;
        const double shapecontextdistance_on_gitlab_runner = 0.00054425356211140752;

        // Does not throw any notice on my X250
        BOOST_CHECK_EQUAL(calculated_shapecontextdistance, shapecontextdistance_on_my_x250);

        // Does not throw any notice on gitlab runner with image opensuse/leap:15.0
        BOOST_CHECK_EQUAL(shapecontextdistance_on_gitlab_runner, shapecontextdistance_on_gitlab_runner);

        // Depending on the machine on which the shapecontextdistance is calculated on, the result changes to a small
        // degree. But both values should be equal?
    }

Image to be loaded

bugreport_shapecontext_skeleton

Result of executing the unittest on my notebook

Testing started at 14:39 ...
/[...]/tvpa/src/cmake-build-debug/test --run_test=TVPA_TestSuite/BUGREPORT_SHAPECONTEXT --logger=HRF,all --color_output=false --report_format=HRF --show_progress=no
Running 1 test case...
Entering test module Test
/[...]/tvpa/src/tests.cpp(201): info: check merged_contour1 == merged_contour2 has passed
/[...]/tvpa/src/tests.cpp(202): info: check merged_contour1.size() == merged_contour2.size() has passed

/[...]/tvpa/src/tests.cpp(212): info: check calculated_shapecontextdistance == shapecontextdistance_on_my_x250 has passed

/[...]/tvpa/src/tests.cpp(215): error: in "TVPA_TestSuite/BUGREPORT_SHAPECONTEXT": check calculated_shapecontextdistance == shapecontextdistance_on_gitlab_runner has failed [0.00076877244282513857 != 0.00054425356211140752]

Leaving test module Test
*** 1 failure is detected in the test module "Test"
Process finished with exit code 201

Result of executing the unittest on gitlab-runner

$ ./test --run_test=TVPA_TestSuite/BUGREPORT_SHAPECONTEXT --logger=HRF,all --color_output=false --report_format=HRF --show_progress=no
Running 1 test case...
Entering test module "Test"
/builds/[...]/tvpa/src/tests.cpp(47): Entering test suite "TVPA_TestSuite"
/builds/[...]/tvpa/src/tests.cpp(168): Entering test case "BUGREPORT_SHAPECONTEXT"
/builds/[...]/tvpa/src/tests.cpp(201): info: check merged_contour1 == merged_contour2 has passed
/builds/[...]/tvpa/src/tests.cpp(202): info: check merged_contour1.size() == merged_contour2.size() has passed
/builds/[...]/tvpa/src/tests.cpp(212): error: in "TVPA_TestSuite/BUGREPORT_SHAPECONTEXT": check calculated_shapecontextdistance == shapecontextdistance_on_my_x250 has failed [0.00054425356211140752 != 0.00076877244282513857]
/builds/[...]/tvpa/src/tests.cpp(215): info: check calculated_shapecontextdistance == shapecontextdistance_on_gitlab_runner has passed
/builds/[...]/tvpa/src/tests.cpp(168): Leaving test case "BUGREPORT_SHAPECONTEXT"; testing time: 34390227us
/builds/[...]/tvpa/src/tests.cpp(47): Leaving test suite "TVPA_TestSuite"; testing time: 34390289us
Leaving test module "Test"; testing time: 34390334us

*** 1 failure is detected in the test module "Test"
ERROR: Job failed: exit code 1

It is strange, is it not, that each time the other check fails and vice-versa? Furthermore I am wondering, why an identity-check does not equal zero using the shapecontext-distance (it equals zero for the hausdorff-distance)?

This bugreport is nothing urgent. I just wanted to make the finding public. Thank you in advance for looking into this. :slightly_smiling_face:

opencv / opencv_contrib