Live test for data exports

sul-dlss / FOLIO-Project-Stanford

Task management for Stanford’s analysis of FOLIO.

2 stars 0 forks source link

Live test for data exports #627

Closed jgreben closed 2 months ago

jgreben commented 6 months ago

[x] POD - full dump
[x] POD - incremental
[ ] GOBI
[ ] OCLC

shelleydoljack commented 5 months ago

Alissa wants to look at the files that would be transmitted for the full dump to make sure the processing looks right.

shelleydoljack commented 4 months ago

Josh mentioned using httpbin to fake requests sent to test POD. If we do this, then we should remove the authentication token from the pod connection info in airflow. Otherwise, we could upload the data to the test stream by simply adding url_params={"stream": "Test"} to pod_transmission.py

ahafele commented 4 months ago

In progress for pod and gobi - https://docs.google.com/document/d/1LrClfJo235NtEUR3K5aFQmH1t01sfr9smmtndt8DFy4/edit

ahafele commented 2 months ago

Still in progress. Closing ticket and making bug tickets as needed.