search

bazelbuild / bazel

a fast, scalable, multi-language and extensible build system
https://bazel.build
Apache License 2.0
23.2k stars 4.06k forks source link

Allow for multiple endpoints for Bazel Event Service #10908

Open konste opened 4 years ago

konste commented 4 years ago

Description of the problem / feature request:

Bazel allows configuring endpoint for the Bazel Event Service through --bes_backend command line option. We need it to allow for multiple endpoints, so that events stream is sent to all of them.

Feature requests: what underlying problem are you trying to solve with this feature?

While in the past BES was underutilized now there are more and more services that use it to provide some kind of visibility into the build process. Examples:

Unfortunately at this time only one endpoint can be configured for BES, which (probably) means we only can use one of those tools at the time, while we would definitely prefer for several of them to work at the same time. Unless there is a workaround I don't know about it would be great if Bazel would allow configuring multiple BES endpoints, so that all interested listeners could be plugged in simultaneously.

What's the output of bazel info release?

2.2.0

github-actions[bot] commented 1 year ago

Thank you for contributing to the Bazel repository! This issue has been marked as stale since it has not had any activity in the last 2+ years. It will be closed in the next 14 days unless any other activity occurs or one of the following labels is added: "not stale", "awaiting-bazeler". Please reach out to the triage team (@bazelbuild/triage) if you think this issue is still relevant or you are interested in getting the issue resolved.

brentleyjones commented 1 year ago

@bazelbuild/triage Not stale.

coeuvre commented 1 year ago

cc @michaeledgar

michaeledgar commented 1 year ago

This seems feasible enough - we already stream events to an arbitrary number of BuildEventTransports. Bazel has complicated rules for blocking on these transports to ensure BES streams are completely uploaded. The net effect would be waiting for the slowest BES server to complete between Bazel commands.

The workaround would be reading the BEP stream from one BES server and writing it to the others.

michaeledgar commented 1 year ago

I have a prototype of this here: https://bazel-review.googlesource.com/c/bazel/+/231672. No OSS tests yet but it does appear to work.

pauliusjwix commented 9 months ago

I have a use case that could leverage this functionality - providing BES to the RBE + sending events to internal system for post-build processing of outputs. When it will be available I'm glad to help with testing it out.

fitzner commented 6 months ago

I have a use case that could leverage this functionality - providing BES to the RBE + sending events to internal system for post-build processing of outputs. When it will be available I'm glad to help with testing it out.

The support of multiple BES endpoints would be very helpful. One small remark would be the support of multiple credentials e.g. --tls_client_certificate and --tls_client_key) for the different endpoints.

sanav33 commented 3 months ago

Any updates on this?

asomerville-latai commented 2 months ago

@michaeledgar what does it mean that there are no OSS tests yet?

Any chance of this being opened as a PR here? or is that link you provided a preferred internal thing?