Closed ahafele closed 1 month ago
When I run this locally, putting txt and mrc files in a new or updates gobi directory, the files get moved to gobi/transmitted/new|updates
. They also appear correctly in the file download section:
I added some information to the log on which files are being moved:
Hmm, but I think I see what's going on. The marc files don't have the stf.
prefix so they aren't being selected to be moved:
libsys@sul-libsys-airflow-dev:~/libsys-airflow/shared/data-export-files/gobi$ ls -l marc-files/new/
total 772
-rw-rw-r-- 1 libsys root 9811 May 31 06:57 202405311349.mrc
-rw-rw-r-- 1 libsys root 742216 May 31 06:58 202405311357.mrc
-rw-rw-r-- 1 libsys root 4905 Jun 3 06:39 202406031339.mrc
-rw-rw-r-- 1 libsys root 4905 Jun 3 06:45 202406031345.mrc
-rw-rw-r-- 1 libsys root 6646 Jun 3 06:51 202406031351.mrc
-rw-rw-r-- 1 libsys root 3482 Jun 3 11:01 202406031800.mrc
libsys@sul-libsys-airflow-dev:~/libsys-airflow/shared/data-export-files/gobi$ ls -l transmitted/new/
total 40
-rw-rw-r-- 1 libsys root 74 May 29 15:58 202405292258.csv
-rw-rw-r-- 1 libsys root 4906 May 29 15:58 202405292258.mrc
-rw-rw-r-- 1 libsys root 27 May 29 15:58 stf.202405292258.txt
-rw-rw-r-- 1 libsys root 54 May 31 06:59 stf.202405311349.txt
-rw-rw-r-- 1 libsys root 5823 May 31 06:59 stf.202405311357.txt
-rw-rw-r-- 1 libsys root 27 Jun 3 06:40 stf.202406031339.txt
-rw-rw-r-- 1 libsys root 27 Jun 3 06:46 stf.202406031345.txt
-rw-rw-r-- 1 libsys root 54 Jun 3 06:51 stf.202406031351.txt
DAG is successful but files are not moved to state=transmitted
https://sul-libsys-airflow-dev.stanford.edu/dags/send_gobi_records/grid?run_id=manual__2024-06-03T15:46:22.307587%2B00:00&execution_date=2024-06-03+15:46:22.307587%2B00:00&tab=graph&dag_run_id=manual__2024-06-03T15:46:22.307587%2B00:00