[RFC] use pip sync instead of pip install for pinned dependencies

[cnolanminich]

This PR introduces a way to separate the python packages you need vs. the pinned dependencies.

The motivating factors to adopt this:

• we can control when we upgrade Dagster versions for Hooli (so if there is a breaking change we're not quite ready to adopt, we can accept other PRs in the meantime).
• We can see all of the python packages that are used (so if there are weird dependency issues we can see them in the requirements.txt, which acts as lockfile of sorts.
• The Dockerfile builds deterministically (so if it built once it will always build).
• It's useful to have a demonstration of how to do this in case customers want to see how we would use pinned dependencies
• (later) we could likely skip the "build steps" if these files aren't updated, which would allow build to be much faster (< 1 minute vs. 8 minutes)

Why we might not want to

• it's another step if you want to upgrade anything in Dagster+
• you need Docker installed on your computer because uv pip sync is platform-specific (this is fine for every package except for Sling, which requires amd64 arch to be specified.
• it's annoying if your setup.py would work across Python versions but are set to different versions (as we have in some of our Dockerfiles). The only way to solve this permanently would be multiple dockerfiles with all of our python distros.

[github-actions[bot]]

Your pull request at commit d55b0dd838da0fe8722f3fab84671522158a3785 is automatically being deployed to Dagster Cloud.

Location	Status	Link	Updated
demo_assets		Building...	Apr 19, 2024 at 02:25 PM (UTC)
batch_enrichment		Building...	Apr 19, 2024 at 02:25 PM (UTC)
data-eng-pipeline		Building...	Apr 19, 2024 at 02:25 PM (UTC)
basics		Building...	Apr 19, 2024 at 02:25 PM (UTC)
snowflake_insights		Building...	Apr 19, 2024 at 02:25 PM (UTC)

[github-actions[bot]]

Your pull request at commit 734f8d1e15a213f92d88164bfa30a14a6ba6929f is automatically being deployed to Dagster Cloud.

Location	Status	Link	Updated
demo_assets		Deploy failed	Apr 19, 2024 at 04:29 PM (UTC)
batch_enrichment		Deploy failed	Apr 19, 2024 at 04:29 PM (UTC)
data-eng-pipeline		Deploy failed	Apr 19, 2024 at 04:29 PM (UTC)
basics		Deploy failed	Apr 19, 2024 at 04:29 PM (UTC)
snowflake_insights		Deploy failed	Apr 19, 2024 at 04:29 PM (UTC)

[github-actions[bot]]

Your pull request at commit a1b5eacfca0ec05832322f499af095d8c412fdd9 is automatically being deployed to Dagster Cloud.

Location	Status	Link	Updated
demo_assets		Deploy failed	Apr 19, 2024 at 05:23 PM (UTC)
batch_enrichment		Deploy failed	Apr 19, 2024 at 05:23 PM (UTC)
data-eng-pipeline		Deploy failed	Apr 19, 2024 at 05:23 PM (UTC)
basics		Deploy failed	Apr 19, 2024 at 05:23 PM (UTC)
snowflake_insights		Deploy failed	Apr 19, 2024 at 05:23 PM (UTC)

[github-actions[bot]]

Your pull request at commit 342356224a519bd61a918b6b3a671130fb162417 is automatically being deployed to Dagster Cloud.

Location	Status	Link	Updated
demo_assets		Deploy failed	Apr 19, 2024 at 05:36 PM (UTC)
batch_enrichment		Deploy failed	Apr 19, 2024 at 05:36 PM (UTC)
data-eng-pipeline		Deploy failed	Apr 19, 2024 at 05:36 PM (UTC)
basics		Deploy failed	Apr 19, 2024 at 05:36 PM (UTC)
snowflake_insights		Deploy failed	Apr 19, 2024 at 05:36 PM (UTC)

[github-actions[bot]]

Your pull request at commit ecdf90fd0558d85f3437c835644d21043228f218 is automatically being deployed to Dagster Cloud.

Location	Status	Link	Updated
demo_assets		View in Cloud	Apr 19, 2024 at 05:49 PM (UTC)
batch_enrichment		View in Cloud	Apr 19, 2024 at 05:49 PM (UTC)
data-eng-pipeline		View in Cloud	Apr 19, 2024 at 05:49 PM (UTC)
basics		View in Cloud	Apr 19, 2024 at 05:49 PM (UTC)
snowflake_insights		View in Cloud	Apr 19, 2024 at 05:49 PM (UTC)

[github-actions[bot]]

Your pull request at commit 612609ba387a7409b30b70439cb31de7e46fc087 is automatically being deployed to Dagster Cloud.

Location	Status	Link	Updated
demo_assets		View in Cloud	Apr 19, 2024 at 06:01 PM (UTC)
batch_enrichment		View in Cloud	Apr 19, 2024 at 06:01 PM (UTC)
data-eng-pipeline		View in Cloud	Apr 19, 2024 at 06:01 PM (UTC)
basics		View in Cloud	Apr 19, 2024 at 06:01 PM (UTC)
snowflake_insights		View in Cloud	Apr 19, 2024 at 06:01 PM (UTC)