apache / buildstream

BuildStream, the software integration tool
https://buildstream.build/
Apache License 2.0
85 stars 30 forks source link

Implement `pip` and `junction` origin loading for SourceMirror plugins #1931

Closed gtristan closed 3 months ago

gtristan commented 4 months ago

This adds support for loading SourceMirror plugins via both pip and junction plugin origins.

This tests essential mirror behavior when loading the plugins with pip or junction plugin origins, and also tests the case where the mirror itself is used to define the mirror required to obtain itself across a junction boundary, succeeding when the mirror is not used and asserting the failure mode when mirroring is forced.

gtristan commented 4 months ago

Note about remaining imperfections.

We do not lazy load the SourceMirror plugins, which means that we load them in a loop, while loading the mirrors configuration in the project configuration, rather than delaying the load until the SourceMirror is actually used.

This can still result in some situations where:

This is a bit tricky to resolve, and I suspect the current functionality to satisfy the majority of use cases, but we will most likely run into this issue.

juergbi commented 3 months ago

The project will not load properly depending on the order in which the SourceMirrors are declared in project.conf

As long as the behavior is deterministic and this results in a clear error message, I don't think we need to let that issue block this PR. It may be worth a note in the documentation. However, I agree that we likely want to eliminate this limitation at some point.