search

DAGWorks-Inc / hamilton

Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage/tracing and metadata. Runs and scales everywhere python does.
https://hamilton.dagworks.io/en/latest/
BSD 3-Clause Clear License
1.7k stars 103 forks source link

Make docker container contain all environments for all examples #75

Open skrawcz opened 1 year ago

skrawcz commented 1 year ago

Is your feature request related to a problem? Please describe. The docker container created for the examples does not enable one to run all the examples. The script to build environments that is used in the build process, excludes building environments for examples that have multiple requirements files, e.g.requirements-ray.txt is required to run run_ray.py in the data_quality example. This means that people can only run one variant in these examples.

Describe the solution you'd like The build script should prepare all the environments in an example. E.g.

Describe alternatives you've considered No doing this. Leaving things as is.

Additional context Script in question: https://github.com/DAGWorks-Inc/hamilton/blob/main/examples/make_python_virtualenv.sh

It is used here: https://github.com/DAGWorks-Inc/hamilton/blob/main/examples/Dockerfile

skrawcz commented 1 year ago

cc @CharityKithaka

CharityKithaka commented 1 year ago

Noted. Let me take it up @skrawcz

CharityKithaka commented 1 year ago

Hey, @skrawcz feeling better this week. Working on it.

skrawcz commented 1 year ago

Hey, @skrawcz feeling better this week. Working on it.

let me know when you want to sync.

skrawcz commented 10 months ago

@CharityKithaka let me know if you're still working on this.