sul-dlss / libsys-airflow

Airflow DAGS for migrating and managing ILS data into FOLIO along with other LibSys workflows
Apache License 2.0
5 stars 0 forks source link

Bookplates add_979_marc_tags Task Error #1306

Closed jermnelson closed 1 month ago

jermnelson commented 1 month ago

Seeing this stack-trace in each DAG run of digital_bookplate_979 task add_979_marc_tags:

*** Found local files:
***   * /opt/airflow/logs/dag_id=digital_bookplate_979/run_id=manual__2024-10-18T22:49:17.865267+00:00/task_id=add_979_marc_tags/attempt=2.log
[2024-10-18, 22:50:27 UTC] {taskinstance.py:1979} INFO - Dependencies all met for dep_context=non-requeueable deps ti=<TaskInstance: digital_bookplate_979.add_979_marc_tags manual__2024-10-18T22:49:17.865267+00:00 [queued]>
[2024-10-18, 22:50:27 UTC] {taskinstance.py:1979} INFO - Dependencies all met for dep_context=requeueable deps ti=<TaskInstance: digital_bookplate_979.add_979_marc_tags manual__2024-10-18T22:49:17.865267+00:00 [queued]>
[2024-10-18, 22:50:27 UTC] {taskinstance.py:2193} INFO - Starting attempt 2 of 2
[2024-10-18, 22:50:27 UTC] {taskinstance.py:2217} INFO - Executing <Task(_PythonDecoratedOperator): add_979_marc_tags> on 2024-10-18 22:49:17.865267+00:00
[2024-10-18, 22:50:27 UTC] {standard_task_runner.py:60} INFO - Started process 99393 to run task
[2024-10-18, 22:50:27 UTC] {standard_task_runner.py:87} INFO - Running: ['airflow', 'tasks', 'run', 'digital_bookplate_979', 'add_979_marc_tags', 'manual__2024-10-18T22:49:17.865267+00:00', '--job-id', '50996', '--raw', '--subdir', 'DAGS_FOLDER/digital_bookplates/digital_bookplate_979.py', '--cfg-path', '/tmp/tmpd1wqqrjm']
[2024-10-18, 22:50:27 UTC] {standard_task_runner.py:88} INFO - Job 50996: Subtask add_979_marc_tags
[2024-10-18, 22:50:27 UTC] {task_command.py:423} INFO - Running <TaskInstance: digital_bookplate_979.add_979_marc_tags manual__2024-10-18T22:49:17.865267+00:00 [running]> on host 577e21c64556
[2024-10-18, 22:50:27 UTC] {taskinstance.py:2513} INFO - Exporting env vars: AIRFLOW_CTX_DAG_OWNER='libsys' AIRFLOW_CTX_DAG_ID='digital_bookplate_979' AIRFLOW_CTX_TASK_ID='add_979_marc_tags' AIRFLOW_CTX_EXECUTION_DATE='2024-10-18T22:49:17.865267+00:00' AIRFLOW_CTX_TRY_NUMBER='2' AIRFLOW_CTX_DAG_RUN_ID='manual__2024-10-18T22:49:17.865267+00:00'
[2024-10-18, 22:50:27 UTC] {taskinstance.py:2731} ERROR - Task failed with exception
Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.11/site-packages/airflow/models/taskinstance.py", line 444, in _execute_task
    result = _execute_callable(context=context, **execute_callable_kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/airflow/models/taskinstance.py", line 414, in _execute_callable
    return execute_callable(context=context, **execute_callable_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/airflow/decorators/base.py", line 241, in execute
    return_value = super().execute(context)
                   ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/airflow/operators/python.py", line 200, in execute
    return_value = self.execute_callable()
                   ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/airflow/operators/python.py", line 217, in execute_callable
    return self.python_callable(*self.op_args, **self.op_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/airflow/libsys_airflow/plugins/digital_bookplates/bookplates.py", line 146, in add_979_marc_tags
    fund_name = tag_data.get('fund_name', None)
                ^^^^^^^^^^^^
AttributeError: 'str' object has no attribute 'get'
jgreben commented 1 month ago

Fixed by #1321