search

tilt-dev / tilt

Define your dev environment as code. For microservice apps on Kubernetes.
https://tilt.dev/
Apache License 2.0
7.59k stars 298 forks source link

feat: parallel sync with live update #4511

Open MasayaAoyama opened 3 years ago

MasayaAoyama commented 3 years ago

Describe the Feature You Want

tilt sync files in parallel using the same image.

Current Behavior

tilt sync files in serial using the same image.

Why Do You Want This?

In our environment, a container image is used for several deployments (e.g. 1. single binary for multi apps, 2. each-namespace apps). We use tilt with live sync, but tilt sync files in serial to several deployments using the same image. Do you have any idea or plan to sync files in parallel using the same image?

test result

Run Deployments test1 and test2 with the same image nginx:1.18 by tilt.

cd tilttest
tilt up &

And then, trigger live sync for both Deployments test1 test2.

touch ./testdata

and then, show live sync progress from the logfile.

# kubectl exec -it test1-5786f6b9d6-vgwc5 -- cat /root/log
Tue May  4 17:37:28 UTC 2021
0
Tue May  4 17:37:29 UTC 2021
9
Tue May  4 17:37:30 UTC 2021
55
Tue May  4 17:37:31 UTC 2021
90
Tue May  4 17:37:32 UTC 2021
132
Tue May  4 17:37:33 UTC 2021
175
Tue May  4 17:37:34 UTC 2021
211
Tue May  4 17:37:35 UTC 2021
251
Tue May  4 17:37:36 UTC 2021
255
Tue May  4 17:37:37 UTC 2021
255
# kubectl exec -it test2-56ccc6b89f-bbjs6 -- cat /root/log
Tue May  4 17:37:28 UTC 2021
0
Tue May  4 17:37:29 UTC 2021
0
Tue May  4 17:37:30 UTC 2021
0
Tue May  4 17:37:31 UTC 2021
0
Tue May  4 17:37:32 UTC 2021
0
Tue May  4 17:37:33 UTC 2021
0
Tue May  4 17:37:34 UTC 2021
0
Tue May  4 17:37:35 UTC 2021
0
Tue May  4 17:37:36 UTC 2021
43
Tue May  4 17:37:37 UTC 2021
91
Tue May  4 17:37:39 UTC 2021
136
Tue May  4 17:37:40 UTC 2021
184
Tue May  4 17:37:41 UTC 2021
228
Tue May  4 17:37:42 UTC 2021
243
Tue May  4 17:37:43 UTC 2021
255
maiamcc commented 3 years ago

Thanks for the request and thorough report @MasayaAoyama ! The fact that we don't live update the same image in parallel is largely an artifact of our logic around not doing the same docker build in parallel (which would be a waste of CPU, as if we build the image once, all subsequent "builds" would just be a matter of hitting the cache). I bet there's room for optimization here! But we'd need to think through behaviors in the failure case, e.g. if for some reason we fail to live update both of these images and fall back to a full docker build, we're in the position of building the same image twice in parallel, instead of building once and then hitting the cache.

Anyway, appreciated! I'll run this request up the flagpole.

MasayaAoyama commented 3 years ago

Hello @maiamcc ! Thank you for your rapid response! oh, I understood the status... if we can, I think it is good the live update is called synchronously, and the process is doing asynchronously with error handling. Now the live update is in serial, so we are feeling it is nearly the same as building the same image repeatedly without cache.

We'll be tracking the issues :) Thank you for always your great product!