PPA: Failing testing suite

[thopiekar]

Since we are using Jenkins for CI I took a look on the output in im PPA builds and there seems to be something wrong

make[2]: Entering directory '/<<PKGBUILDDIR>>/obj-x86_64-linux-gnu'
Running tests...
/usr/bin/ctest --force-new-ctest-process -j4
Test project /<<PKGBUILDDIR>>/obj-x86_64-linux-gnu
    Start 1: pytest
1/1 Test #1: pytest ...........................***Failed    9.46 sec
============================= test session starts ==============================
platform linux -- Python 3.5.3, pytest-3.0.6, py-1.4.32, pluggy-0.4.0
rootdir: /<<PKGBUILDDIR>>, inifile: pytest.ini
collected 254 items

../tests/TestDecorators.py .
../tests/TestSignals.py ..
../tests/Jobs/TestJobQueue.py .......
../tests/Math/TestAxisAlignedBox.py .....
../tests/Math/TestMatrix.py ..........
../tests/Math/TestPlane.py ..
../tests/Math/TestPolygon.py ssssssssssssssss
../tests/Math/TestQuaternion.py .........
../tests/Math/TestVector.py .....
../tests/MimeTypes/TestMimeTypes.py ......F.
../tests/PluginRegistry/TestPluginRegistry.py ......
../tests/SaveFile/TestSaveFile.py ..
../tests/Scene/TestSceneNode.py .........
../tests/Settings/TestContainerRegistry.py .......................
../tests/Settings/TestContainerStack.py .........................
../tests/Settings/TestDefinitionContainer.py ........................
../tests/Settings/TestInstanceContainer.py ........
../tests/Settings/TestRoundtripping.py ............
../tests/Settings/TestSettingDefinition.py ......
../tests/Settings/TestSettingFunction.py ..............................
../tests/Settings/TestSettingInstance.py .......
../tests/Settings/TestSettingRelation.py .
../tests/Settings/TestValidator.py ................................
../tests/VersionUpgrade/TestVersionUpgradeManager.py ssss

 generated xml file: /<<PKGBUILDDIR>>/obj-x86_64-linux-gnu/junit.xml 
=========================== short test summary info ============================
FAIL ../tests/MimeTypes/TestMimeTypes.py::test_getMimeTypeForFile
SKIP [6] tests/Math/TestPolygon.py:88: Incomplete tests
SKIP [3] tests/Math/TestPolygon.py:58: Incomplete tests
SKIP [1] tests/Math/TestPolygon.py:120: Incomplete tests
SKIP [6] tests/Math/TestPolygon.py:36: Incomplete tests
SKIP [4] tests/VersionUpgrade/TestVersionUpgradeManager.py:92: Failing tests due to API changes
=================================== FAILURES ===================================
___________________________ test_getMimeTypeForFile ____________________________

mime_database = <class 'UM.MimeTypeDatabase.MimeTypeDatabase'>

    def test_getMimeTypeForFile(mime_database):
        path_base = os.path.dirname(os.path.abspath(__file__))

        mime = mime_database.getMimeTypeForFile(os.path.join(path_base, "test.jpg"))
        assert mime.comment == "Custom JPEG MIME Type" # We must get the custom one, not Qt's MIME type.

        mime = mime_database.getMimeTypeForFile(os.path.join(path_base, "test.png"))
        assert mime.name == "image/png" # Getting a file type from the system.

        mime = mime_database.getMimeTypeForFile(os.path.join(path_base, "file.test"))
        assert mime.name == "application/x-test"

        mime = mime_database.getMimeTypeForFile(os.path.join(path_base, "filetest.test.test")) # Double extension should still match
        assert mime.name == "application/x-test"

        mime = mime_database.getMimeTypeForFile(os.path.join(path_base, ".test")) # Only extension should still match
        assert mime.name == "application/x-test"

        with pytest.raises(MimeTypeNotFoundError):
            mime_database.getMimeTypeForFile(os.path.join(path_base, "pink.unicorn")) # Non-existent file type.

        with pytest.raises(MimeTypeNotFoundError):
>           mime_database.getMimeTypeForFile(os.path.join(path_base, "filetest")) # File that happens to end in the extension without being an extension.
E           Failed: DID NOT RAISE <class 'UM.MimeTypeDatabase.MimeTypeNotFoundError'>

../tests/MimeTypes/TestMimeTypes.py:172: Failed
============================ pytest-warning summary ============================
W tests/benchmarks/conftest.py Skipping benchmarks because pytest-benchmark plugin was not found.
===== 1 failed, 233 passed, 20 skipped, 1 pytest-warnings in 8.55 seconds ======

0% tests passed, 1 tests failed out of 1

Total Test time (real) =   9.46 sec

The following tests FAILED:
      1 - pytest (Failed)
Errors while running CTest
Makefile:130: recipe for target 'test' failed
make[2]: *** [test] Error 8
make[2]: Leaving directory '/<<PKGBUILDDIR>>/obj-x86_64-linux-gnu'
dh_auto_test: make -j4 test ARGS+=-j4 returned exit code 2

Full log: https://launchpadlibrarian.net/307193376/buildlog_ubuntu-zesty-amd64.uranium_1%3A2.4.99-master~201702201543~rev1744~pkg137~ubuntu17.04.1_BUILDING.txt.gz

[awhiemstra]

The test works correctly on CI, so I do directly know what is wrong. Can you check whether it returns a custom mime type or a mime type from Qt's mime type database?

[thopiekar]

How shall I check for it?

[awhiemstra]

Probably need to add some logging to MimeTypeDatabase::getMimeTypeForFile

[hroncok]

I have the same failure on the Fedora package:

=================================== FAILURES ===================================
___________________________ test_getMimeTypeForFile ____________________________

mime_database = <class 'UM.MimeTypeDatabase.MimeTypeDatabase'>

    def test_getMimeTypeForFile(mime_database):
        path_base = os.path.dirname(os.path.abspath(__file__))

        mime = mime_database.getMimeTypeForFile(os.path.join(path_base, "test.jpg"))
        assert mime.comment == "Custom JPEG MIME Type" # We must get the custom one, not Qt's MIME type.

        mime = mime_database.getMimeTypeForFile(os.path.join(path_base, "test.png"))
        assert mime.name == "image/png" # Getting a file type from the system.

        mime = mime_database.getMimeTypeForFile(os.path.join(path_base, "file.test"))
        assert mime.name == "application/x-test"

        mime = mime_database.getMimeTypeForFile(os.path.join(path_base, "filetest.test.test")) # Double extension should still match
        assert mime.name == "application/x-test"

        mime = mime_database.getMimeTypeForFile(os.path.join(path_base, ".test")) # Only extension should still match
        assert mime.name == "application/x-test"

        with pytest.raises(MimeTypeNotFoundError):
            mime_database.getMimeTypeForFile(os.path.join(path_base, "pink.unicorn")) # Non-existent file type.

        with pytest.raises(MimeTypeNotFoundError):
>           mime_database.getMimeTypeForFile(os.path.join(path_base, "filetest")) # File that happens to end in the extension without being an extension.
E           Failed: DID NOT RAISE <class 'UM.MimeTypeDatabase.MimeTypeNotFoundError'>

tests/MimeTypes/TestMimeTypes.py:172: Failed

[hroncok]

Returned <MimeType name=application/x-test>.

[thopiekar]

Well, for me it seems like the issue is gone.