sqlalchemy.exc.OperationalError raised in the scheduler

[warownia1]

When completed and new jobs are collected/updated in the database simultaneously the sqlite3.OperationalError database is locked is raised due to concurrent transaction timeout. This exception is immediately followed by sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread.. I suppose that the latter error is raised by a parent thread trying to clean-up. Transaction is never closed and database gets locked indefinitely.

This issue seems to be specific to sqlite3 being unable to handle more than one transaction at the time.

Suggested actions:

• make multiple short transactions within a single session instead of a large one
• this error is inevitable and scheduler must be able to handle it gracefully.

[warownia1]

Probably fixed, but the result will come out during more intensive stress testing.

[warownia1]

The problem still occurrs. Try not store the list of output files in the database. Fetch it from the file system using the configuration files instead.

[warownia1]

Fixed by 79de9f58c2a5aeed54cd0f20b9d41c6c81fbeedc