Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
File ".../env/lib/python3.11/site-packages/luigi/db_task_history.py", line 271, in _upgrade_schema
for i in conn.execute('PRAGMA table_info(task_parameters);').fetchall():
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".../env/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1414, in execute
raise exc.ObjectNotExecutableError(statement) from err
sqlalchemy.exc.ObjectNotExecutableError: Not an executable object: 'PRAGMA table_info(task_parameters);'
This is because sqlalchemy requires a sqlalchemy.text object vs string in conn.execute
Have you tested this? If so, how?
Tested locally with sqlite3 and sqlalchemy 2.0.23 and luigi 3.4.0
Description
Quick fix for task history logging and sqlalchemy 2.x
Motivation and Context
Using sqlalchemy 2.x breaks luigid if task history is enabled e.g.
Results in luigid exiting
This is because sqlalchemy requires a sqlalchemy.text object vs string in conn.execute
Have you tested this? If so, how?
Tested locally with sqlite3 and sqlalchemy 2.0.23 and luigi 3.4.0