aws / amazon-mwaa-docker-images

Apache License 2.0
27 stars 11 forks source link

Implement `safe-pip-install` command #18

Closed rafidka closed 5 months ago

rafidka commented 10 months ago

Overview

PyPI package installation should be safe with regards to Airflow, i.e. it should respect its constraints and not break it in anyway. Typically, this would involve:

  1. Applying Airflow's constraints file using the --constraint command argument.
  2. Prefixing the installation with Airflow package each time, e.g.
pip installation --constraint <Airflow constraints file> apache-airflow==<x.y.z> <the required package>

The reason for repeatedly specifying Airflow packages even though they are already install is to make pip break if the required package has conflicting dependencies.

Acceptance Criteria