[ ] trigger_file_processing_DAG for Hindawi and APS
def trigger_file_processing_DAG(ids_and_articles, publisher):
for data in ids_and_articles:
if publisher == "APS":
data["article"] = json.dumps(data["article"])
trigger_dag.trigger_dag(
dag_id=data["id"],
conf={"record": data["article"]},
replace_microseconds=False,
)
return records
[ ] Rename
file
tofile_content
in trigger_file_processing[ ] trigger_file_processing_DAG for Hindawi and APS
for this change we need to adapt split_xmls
[ ] use providers instead of custom classes. For example, for s3 connection, docs: