search

apache / buildstream

BuildStream, the software integration tool
https://buildstream.build/
Apache License 2.0
85 stars 30 forks source link

_stream: Only query cache for the requested elements #1965

Open abderrahim opened 3 weeks ago

abderrahim commented 3 weeks ago

query_cache() used to query the cache for the elements passed and all their (build and runtime) dependencies. This is wasteful since most of the time, we don't need all this cache querying. And for the cases we need, it is the job of the query_cache() caller to include the dependencies they need.

juergbi commented 2 weeks ago

I think this makes sense. However, do I see this right that it doesn't make a difference for typical builds as we always load all elements to support dynamic planning (unless --deps build is specified)? So this is mainly an optimization for non-build commands, or am I missing something?

abderrahim commented 2 weeks ago

I think this makes sense. However, do I see this right that it doesn't make a difference for typical builds as we always load all elements to support dynamic planning (unless --deps build is specified)? So this is mainly an optimization for non-build commands, or am I missing something?

That's correct. The main optimisation here is for artifact checkout.

The commit message here hasn't been updated to cover my most recent findings. I'll try to update it later today.