search

CityofPittsburgh / data-rivers

Apache Airflow and Beam ETL scripts for the City of Pittsburgh's data analysis pipelines
10 stars 1 forks source link

Rewrite OTRS R script in Python #75

Open jtotoole opened 3 years ago

jtotoole commented 3 years ago

At present, the OTRS DAG utilizes this old R script of Geoff's, launched via GKEPodOperator. It would be better to rewrite this script in Python and launch it via a BashOperator, as we do with most GCS and Dataflow scripts. That way we don't have to pay to run a GKE cluster just to handle this step, and it improves the overall coherence of the system (better to have everything in Python if possible).

jtotoole commented 3 years ago

Update: The Service Desk is swapping OTRS out for another ticket-management system, so this may not be worth prioritizing