njcuk9999 / apero-utils

APERO affiliated utilities and tools
MIT License
3 stars 3 forks source link

[APERO] Add column "KEYWORDS" to astrom googlesheet #193

Closed njcuk9999 closed 4 months ago

njcuk9999 commented 10 months ago

Etienne needs a KEYWORDS columns

Example: TELLURIC, VETTED_SPIROU, VETTED_NIRPS, FAST_ROT

Problem: adding new keyword breaks previous versions with an error such as:

15:31:37.960-!!|DBMGR| E[01-010-00001]: Unhandled error has occurred: Error <class 'KeyError'>
15:31:37.961-!!|DBMGR| 
15:31:37.961-!!|DBMGR| Traceback (most recent call last):
15:31:37.961-!!|DBMGR|  File "/scratch2/spirou/drs-bin/apero-drs-spirou-07XXX/apero/core/utils/drs_startup.py", line 433, in run
15:31:37.962-!!|DBMGR|    llmain = func(recipe, params)
15:31:37.962-!!|DBMGR|  File "/scratch2/spirou/drs-bin/apero-drs-spirou-07XXX/tools/bin/apero_database.py", line 119, in __main__
15:31:37.962-!!|DBMGR|    database_update.update_database(params, dbkind=dbkind)
15:31:37.962-!!|DBMGR|  File "/scratch2/spirou/drs-bin/apero-drs-spirou-07XXX/apero/tools/module/database/database_update.py", line 111, in update_database
15:31:37.963-!!|DBMGR|    manage_databases.update_object_database(params)
15:31:37.963-!!|DBMGR|  File "/scratch2/spirou/drs-bin/apero-drs-spirou-07XXX/apero/tools/module/database/manage_databases.py", line 719, in update_object_database
15:31:37.963-!!|DBMGR|    maintable = get_object_database(params, log=log)
15:31:37.963-!!|DBMGR|  File "/scratch2/spirou/drs-bin/apero-drs-spirou-07XXX/apero/tools/module/database/manage_databases.py", line 662, in get_object_database
15:31:37.964-!!|DBMGR|    maintable = _force_column_dtypes(maintable, OBJ_DATA_TYPES)
15:31:37.964-!!|DBMGR|  File "/scratch2/spirou/drs-bin/apero-drs-spirou-07XXX/apero/tools/module/database/manage_databases.py", line 997, in _force_column_dtypes
15:31:37.964-!!|DBMGR|    values = np.array(table[col]).astype(coltype[col])
15:31:37.964-!!|DBMGR| KeyError: 'KEYWORDS'

So this problem is two fold:

  1. for 0.7.289 add a way to add new columns without breaking
  2. for 0.8.xxx use this column (add to header in preprocessing?)
njcuk9999 commented 10 months ago

Added a fix to a few branches (v0.7.288-stable-test v0.7.289-live developer main) that should allow adding new columns.