[bug] psycopg2.errors.UndefinedColumn: column "exc_msg" of relation "data_process_task_log" does not exist

[agilebot1]

Failed to process data, it seems lose data column, perhaps need upgrade postgrel database?

2024-03-27 20:46:37,930 [DEBUG] - Web Server Access post http://arcadia-dataprocess.kubeagi-system.svc.cluster.local:28888/info-by-id 200 0.0119 seconds
2024-03-27 21:13:14,273 [ERROR] - Minio Store Process The file type is not supported
The current file type is: json
2024-03-27 21:13:14,275 [DEBUG] - Minio Store Process Insert QA list for detail preview.
2024-03-27 21:13:14,277 [DEBUG] - Minio 上传文件到minio中 []
2024-03-27 21:13:14,277 [DEBUG] - Minio Store Process update dataset status
task_id: 01HSZYVEX3RXMTAT8PN4PQX13X
namespace: poemsai
version_data_set_name: generated-chat-v1
reason: process_fail
2024-03-27 21:13:14,277 [DEBUG] - Kubernetes Try to loading kubeconfig from in cluster config
2024-03-27 21:13:14,296 [ERROR] - Minio Store Process Data process fail
Traceback (most recent call last):
  File "/arcadia_app/src/database_clients/postgresql_pool_client.py", line 128, in execute_update
    cursor.execute(sql, params)
  File "/usr/local/lib/python3.11/site-packages/dbutils/steady_db.py", line 610, in tough_method
    result = method(*args, **kwargs)  # try to execute
             ^^^^^^^^^^^^^^^^^^^^^^^
psycopg2.errors.UndefinedColumn: column "exc_msg" of relation "data_process_task_log" does not exist
LINE 5:           exc_msg = '',
                  ^

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/arcadia_app/src/data_store_process/minio_store_process.py", line 355, in text_manipulate
    data_process_log_db_operate.update_status_by_id(update_log_item, pool=pool)
  File "/arcadia_app/src/database_operate/data_process_log_db_operate.py", line 105, in update_status_by_id
    res = postgresql_pool_client.execute_update(pool, sql, params)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/arcadia_app/src/database_clients/postgresql_pool_client.py", line 133, in execute_update
    conn.rollback()
    ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/dbutils/pooled_db.py", line 430, in __getattr__
    raise InvalidConnection
dbutils.pooled_db.InvalidConnection

Exception in thread execute text manipuate task:
Traceback (most recent call last):
  File "/arcadia_app/src/database_clients/postgresql_pool_client.py", line 128, in execute_update
 cursor.execute(sql, params)
  File "/usr/local/lib/python3.11/site-packages/dbutils/steady_db.py", line 610, in tough_method
    result = method(*args, **kwargs)  # try to execute
             ^^^^^^^^^^^^^^^^^^^^^^^
psycopg2.errors.UndefinedColumn: column "exc_msg" of relation "data_process_task_log" does not exist
LINE 5:           exc_msg = '',
                  ^

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/arcadia_app/src/data_store_process/minio_store_process.py", line 355, in text_manipulate
    data_process_log_db_operate.update_status_by_id(update_log_item, pool=pool)
  File "/arcadia_app/src/database_operate/data_process_log_db_operate.py", line 105, in update_status_by_id
    res = postgresql_pool_client.execute_update(pool, sql, params)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/arcadia_app/src/database_clients/postgresql_pool_client.py", line 133, in execute_update
    conn.rollback()
    ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/dbutils/pooled_db.py", line 430, in __getattr__
    raise InvalidConnection
dbutils.pooled_db.InvalidConnection

[bjwswang]

@huangqg you need to update the pg tables with https://github.com/kubeagi/arcadia/blob/main/pypi/data-processing/db-scripts/change/20240319-schema.sql

[agilebot1]

after update the pg table, the problem was gone.