OPTIMIZER: datasource error

[jsfried]

Reran processor on the BlueDemo10 project. Then went to rerun a core analysis that I'd run a few hours before without incident and got:

in the first bar of the run (Validate Rule definitions). Then went through all the core tabs, saved the scenario, and tried again and it executed without incident. Why?

[lbross]

I am 90% sure this is the same problem as reported under issue #98. See e-mail dated August 22, 2017 with subject 'Potential glitch and work around'. This is caused by an MS Access DAO error: 'Too many client tasks' when the scenario_datasource table is queried. After consulting with @lpotts, I was unable to find a solution for the underlying error. Instead, we trap the error and stop processing when it occurs. Before we implemented this 'fix', this occurrence would result in an unhandled exception and crash BioSum. It sounds like you were able to continue working and eventually execute the scenario, which is an improvement. My guess is that the 'saves' you executed somehow cleared up the DAO connection.

We can leave this open as a 'someday' item, but at this time, I am unable to fix it.

[sorgtyler]

I encountered this "too many client tasks" issue again while working on my DWM additions.

Just to add more information, the error occurred for me in FIA_BIOSUM_MANAGER.Datasource.populate_datasource_array(). For my work, I added extra tables to the datasource array (DWM and REF_FOREST_TYPE*) so that temporary databases would link to them.

They way I reproduced this error was by building a project with evalid=411001 using the OR.accdb source database. I would pick the first plot in countycd=1 during my plot input phase, and then assign the variant in the FVS module. Clicking on the Rx button caused error message boxes to appear saying "The Query Command select table_type, path, file, table_name from datasource Failed," which seems more general than when running a specific scenario. After that, another error would say that the harvest methods table couldn't be found. I returned to the Database module to look at the sources, encountered another error, and all the datasources were blank. I closed the datasources window and opened it again, finding all the tables successfully located and without any error messages.

When I click on a new project's Rx button the first time, without any breakpoints in the populate_datasource_array() method, the error occurs. However, when stepping through it slowly, no errors occur. If I keep running the debugger, close the Rx window, and then click on the Rx button again, I see different errors having to do with not being able to find other tables.

https://stackoverflow.com/questions/17758604/ms-access-too-many-client-tasks This answer on StackOverflow might be helpful. It says that working with access databases programmatically requires explicit release of resources and closure of connections as soon as possible.

In the method I mentioned, there's an OleDbConnection oConn, an OleDbCommand oCommand based on oConn, an OleDbDataReader oDataReader built using oCommand.ExecuteReader(), an ado_data_acess p_ado, and a dao_data_access p_dao all interacting with project.mdb and the databases referred to in its datasource table in some capacity in a while loop. These p_ado and p_dao objects open and close for each of the ~40 tables in my datasource array during this loop. It might be the case that there needs to be some kind of delay to check that resources have been released by one of these objects. The p_ado object has a Thread.Sleep(1000), but the p_dao.m_DaoWorkspace.Close() might not have that delay.

I hope that helps whoever needs to explore this again.

[lbross]

Updated the offending function (datasource.populate_datasource_array) that seems to be at the root of the problem. This function used a dao, rather than an ado, to determine the existence of tables. We've historically had troubles with dao's disposing of their resources properly, even after Biosum restarts. I implemented @sorgtyler's "using connection" syntax to set up a separate ado and associated connection to replace the dao.

[lbross]

If the dao to ado fix doesn't resolve the issue, consider adding an OPEN count variable to both the ado and dao classes that increments/decrements when the OPEN and CLOSE routines are used. Or, if employing the USING command, increment before nested command and decrement after exiting nested command.

From @lpotts: Since the ado class is managed code, one could declare open and close trigger events that override the base event and then add a counter in the override event before actually calling the base object.

The reason for doing this is that if a connection is open and the code directly closes the connection (ie rather than calling the ado_data_access.CloseConnection routine) the close connection could still be managed and the counter decremented.