hubmapconsortium / airflow-dev

Configuration, workflows, and DAGs for the development instance of Airflow on HIVE infrastructure
MIT License
1 stars 0 forks source link

Create TEST instance of Airflow on HuBMAP infrastructure #16

Closed pdblood closed 3 years ago

pdblood commented 5 years ago

Summary: Enable secure, reliable use of Airflow workflow manager running Docker containers from users outside PSC

Airflow will run as a local 'airflow' user with tightly controlled privileges that will allow airflow to execute sudo to run jobs as actual users (for now, our collaborators in HuBMAP).

Review any security issues associated with airflow sudo'ing to users.

Done When: Airflow running on HuBMAP infrastructure with monitoring, regular backups, documented security controls, and anything else required for robust production deployment