Deadlock in ogr2ogr tests in the unittest.sh module

[andrewcottam]

Although it creates deadlocks in the unittest.sh it doesn't in the marxan app.

Also, if you run the following sequence as a single test then it runs fine:

    def test_000_testAllatOnce(self):
        self.makeRequest('/validateUser?user=' + LOGIN_USER + '&password=' + LOGIN_PASSWORD, False) 
        self.makeWebSocketRequest('/runMarxan?user=' + LOGIN_USER + '&project=Start%20project', False)
        self.makeWebSocketRequest('/exportProject?user=' + LOGIN_USER + '&project=Start%20project', False)
        os.remove(m.EXPORT_FOLDER + LOGIN_USER + '_Start project.mxw')

But, running it as separate tests creates a deadlock:

    def test_001_validateUser(self):
        self.makeRequest('/validateUser?user=' + LOGIN_USER + '&password=' + LOGIN_PASSWORD, False) 

    def test_002_runMarxan(self):
        self.makeWebSocketRequest('/runMarxan?user=' + LOGIN_USER + '&project=Start%20project', False)

    # this causes an ogr2ogr <defunct> process
    def test_003_exportProject(self):
        self.makeWebSocketRequest('/exportProject?user=' + LOGIN_USER + '&project=Start%20project', False)
        os.remove(m.EXPORT_FOLDER + LOGIN_USER + '_Start project.mxw')

[andrewcottam]

See example 12 on this page which has some suggestions.

[andrewcottam]

The following tests cause deadlocks: exportProject importProject exportPlanningUnitGrid - timeout (GET) importFeatures exportFeature - timeout (GET) createFeaturesFromWFS createFeatureFromLinestring - timeout (GET) - uses ogr2ogr in exportToShapefile importGBIFData

[andrewcottam]

This was because of the unittest module not supporting asynchronous functions. See Ben Darnells article here.

The reason why the single test worked and the separate tests didnt is because of ogr2ogr defunct (zombie) processes that were created because the database connection was closed while ogr2ogr was running. Because the unittest module does not support coroutines, the exportProject function was called and then the tearDown method of unittest was called which closed the database connection immediately instead of waiting for the exportProject to finish. Now tearDown uses a helper which waits for the function to finish.