marxanweb / general

Issues tracker, installers, registry, test data and example developer apps for Marxan Web
European Union Public License 1.2
2 stars 2 forks source link

Deadlock in ogr2ogr tests in the unittest.sh module #232

Closed andrewcottam closed 4 years ago

andrewcottam commented 4 years ago

Although it creates deadlocks in the unittest.sh it doesn't in the marxan app.

Also, if you run the following sequence as a single test then it runs fine:

    def test_000_testAllatOnce(self):
        self.makeRequest('/validateUser?user=' + LOGIN_USER + '&password=' + LOGIN_PASSWORD, False) 
        self.makeWebSocketRequest('/runMarxan?user=' + LOGIN_USER + '&project=Start%20project', False)
        self.makeWebSocketRequest('/exportProject?user=' + LOGIN_USER + '&project=Start%20project', False)
        os.remove(m.EXPORT_FOLDER + LOGIN_USER + '_Start project.mxw')

But, running it as separate tests creates a deadlock:

    def test_001_validateUser(self):
        self.makeRequest('/validateUser?user=' + LOGIN_USER + '&password=' + LOGIN_PASSWORD, False) 

    def test_002_runMarxan(self):
        self.makeWebSocketRequest('/runMarxan?user=' + LOGIN_USER + '&project=Start%20project', False)

    # this causes an ogr2ogr <defunct> process
    def test_003_exportProject(self):
        self.makeWebSocketRequest('/exportProject?user=' + LOGIN_USER + '&project=Start%20project', False)
        os.remove(m.EXPORT_FOLDER + LOGIN_USER + '_Start project.mxw')
andrewcottam commented 4 years ago

See example 12 on this page which has some suggestions.

andrewcottam commented 4 years ago

The following tests cause deadlocks: exportProject importProject exportPlanningUnitGrid - timeout (GET) importFeatures exportFeature - timeout (GET) createFeaturesFromWFS createFeatureFromLinestring - timeout (GET) - uses ogr2ogr in exportToShapefile importGBIFData

andrewcottam commented 4 years ago

This was because of the unittest module not supporting asynchronous functions. See Ben Darnells article here.

The reason why the single test worked and the separate tests didnt is because of ogr2ogr defunct (zombie) processes that were created because the database connection was closed while ogr2ogr was running. Because the unittest module does not support coroutines, the exportProject function was called and then the tearDown method of unittest was called which closed the database connection immediately instead of waiting for the exportProject to finish. Now tearDown uses a helper which waits for the function to finish.