numaproj / numaflow-python

Numaflow Python SDK
Apache License 2.0
50 stars 17 forks source link

Optimize example Dockerfiles #181

Open ayildirim21 opened 1 month ago

ayildirim21 commented 1 month ago

Summary

The example images are quite large in size, and take a while to build. We should optimize this to reduce both size and build time. Considered solution: Build dependencies in a virtualenv in the builder layer, and copy over to the runner.

Use Cases

When building example images


Message from the maintainers:

If you wish to see this enhancement implemented please add a 👍 reaction to this issue! We often sort issues this way to know what to prioritize.

vigith commented 1 month ago

@ab93 any other suggestions ?

ab93 commented 1 month ago

@ab93 any other suggestions ?

Yes we should do something along the lines of: https://github.com/ab93/dockerdemo/blob/main/Dockerfile

vigith commented 1 month ago

Yes we should do something along the lines of: https://github.com/ab93/dockerdemo/blob/main/Dockerfile

lol, that is too much for examples :)

ab93 commented 1 month ago

I think you got overwhelmed there 🙂 It's not hard, and the dockerfile will be very short for our use case. Numalogic dockerfile is also a good example.

kohlisid commented 1 month ago

Did optimize docker file a bit for the recent intuit builds! Will do some further research and go through the examples provided by @ab93 too.

@ayildirim21 if you're interested in working on this as well let me know!