search

Azure / draft-classic

A tool for developers to create cloud-native applications on Kubernetes.
https://draft.sh
MIT License
3.92k stars 395 forks source link

Streaming build context for faster incremental build #566

Open mumoshu opened 6 years ago

mumoshu commented 6 years ago

As far as I could see from the current codebase, draftd sends the whole project files as the build context to the docker daemon. This, "in theory", slow down docker build by unnecessary sending unchanged files from draftd to docker daemon.

Similarly as I've noted in https://github.com/GoogleCloudPlatform/skaffold/issues/195, I guess we could enhance draft to use streaming to send only diffs from draftd to docker daemon.

Just for clarification, this is for the "2" in the below diagram:

draft --1--> draftd --2--> docker daemon

Honestly, I'm not sure how the path "1" works today in draft. It would make sense to address it in an another github issue if necessary, too.

bacongobbler commented 6 years ago

Yes! We would love to have this in place. I know @fibonacci1729 has played with a few ideas in this space and might be able to share some of this thoughts.

squillace commented 6 years ago

@mumoshu the new draft .12 removes the daemon completely, removing the problem. Rerun tests, and see if there are still optimizations here we should work on.

bacongobbler commented 6 years ago

While we have removed draftd from the system and thereby eliminated the draft --1--> draftd cycle, technically speaking we are still sending the entire build context to the docker daemon.

The architecture looks more like this:

draft --1--> docker daemon --> container registry (if configured) --> kubernetes

Docker has a relatively new feature that allows clients to send incremental build contexts to the daemon. We could take advantage of that during long-running sessions such as a draft up with the local file watcher feature enabled.

Other than that, you're completely right. For the most part this issue is moot.