Select DAG - Githubissues

sul-dlss / libsys-airflow

Airflow DAGS for migrating and managing ILS data into FOLIO along with other LibSys workflows

Apache License 2.0

5 stars 0 forks source link

Select DAG #1439

Open ahafele opened 6 days ago

ahafele commented 6 days ago

Note: account for Initial selection and ongoing (separate DAG?), should include option for date range configuration

Selection Criteria: Initial

All libraries, ~~excluding Lane~~
New Bib records
Instance status term = Cataloged
Cataloged date = real date (not empty)
Unsuppressed at Instance level
Using specified timeframe, e.g. since FOLIO migration
No deletes
No updates

Ongoing

All libraries
New cataloged Bib records since last run
Instance status term = Cataloged
Cataloged date = real date (not empty)
Unsuppressed at Instance level
No deletes
No updates

Record processing:

Exclude records with 915 |aNO EXPORT|bAUTHORITY VENDOR or 590 |a MARCit brief record.

ahafele commented 3 days ago

Add Campus as a variable so we can e.g. send only Law library records.

shelleydoljack commented 3 days ago

Do we know the schedule for the backstage selection? (weekly, daily, etc?) And are we sending marcxml files or marc files (the answer is dependent upon whether we are sending holdings and items)? I'm guessing just bib marc and no holdings info, right?

dlrueda commented 3 days ago

Just bib Marc for export. No holdings and items needed.

ahafele commented 2 days ago

Marc files and weekly schedule.

shelleydoljack commented 1 day ago

So... the exclude 915 criteria, is that correct? I was looking at what we're doing in general and realized it's not the same for this. Should it be both |bFOR SU ONLY and |bAUTHORITY VENDOR?

ahafele commented 22 hours ago

Only exclude if the 915 contains both |aNO EXPORT and |bAUTHORITY VENDOR

ahafele commented 22 hours ago

Update based on meeting with Backstage - we can leave Lane's records in the initial file.

shelleydoljack commented 15 hours ago

Since we don't need to exclude Lane from the initial file, I think we said at standup that we won't make the campus configurable, until we need to. Is that correct, or am I misremembering?