Build & ship "BDK" (bazel development kit) which will allow developers to use familiar programming languages to write code for bazel

[beingminimal]

Description of the feature request:

Bazel Development Kit will allow developers to use familiar programming languages to write bazel code. This gives you access to the entire bazel ecosystem without learning StarLark and lets you leverage the power of your existing toolchain for testing, dependency management, etc.

First support typescript for this but eventually we can give support for other languages like Python, Java, C#, and Go etc.

What underlying problem are you trying to solve with this feature?

Currently not all developers are familiar with bazel and starlark language. And bazel is such a powerful tool which can save tons of time in building and shipping softwares, But most of the developers are familiar with other commonly used languages like TypeScript, Python, Java, C#, and Go. So if we can support this feature for bazel then adoption of bazel can be skyrocketed and millions of money can be saved which are currently being wasted in CI-CD pipelines.

@janakdr @ulfjack @lberki @katre @cushon @laszlocsomor @laurentlb @damienmg @aehlig @c-parsons Consider this as a most important DX improvement and give some priority to it. Please make sure this feature request will not be vanished like other issues created in open source projects as usual. This is a really needed feature for all of us. Thank You Everyone.

This concept is inspired by Terraform CDK see here. https://github.com/hashicorp/terraform-cdk https://www.terraform.io/cdktf

[lberki]

No. We will not allow Bazel to run user code in its own address space implemented in any other programming language than Starlark.

Starlark has a number of properties other languages don't guarantee (lack of Turing completeness, built-in sandboxing, caching modules separately from each other, accounting of execution time and soon RAM use) and which Bazel requires.

In addition, it would increase the API surface of Bazel by quite a bit; this would result in both an unacceptably high maintenance burden and user support load, in addition to creating unnecessary and useless choices. I'd much rather there be one "well lit path" than multiple choices without a clear answer as to which choice to take when.

Please do not reopen this bug; this decision will not be re-considered.

[hicksjoseph]

Hi codecycleteam, I wanted to thank you for using Bazel and suggesting features for it. While lberki's technical direction is the direction we are going on this request, I wanted to reach out to you to let you know that some thought was given to your request, but that it was decided to not pursue it. We hope that you will continue to make suggestions for Bazel in the future. Thanks much, Joe

[beingminimal]

@hicksjoseph @lberki Thanks a lot for responding to this issue. I appreciate @hicksjoseph's supportive response.

It's my humble request to consider following content as feedback to increase the impact of this open source project only. Nothing personal with anyone.

• We believe an open source project is driven by the community and not from a single person's decision.
• Let's measure the coding burden with the Pareto principle (The 80/20 rule) If coding burden of 20% on the community can solve bazel adoption problems up to 80% for all developers in the world then don't you think it's advisable to support it?
• This will not help only existing bazel users but also it will open a gate for bazel to adopt in small companies and startups also. Currently only large scale companies are able to adopt it due to bazel's learning curve and starlark dependency. See here
• We should evaluate first how we can take advantage of other programming languages like typescript and not only be reliable on starlark for this. And then we should decide what can be further implemented and how.
• And as of today the bazel project has 781 contributors who are committing code on a regular basis and thousands of users are using this right, so it's advisable to ask them to fill survey to conclude such feature implementation.
• Even projects like terraform and kubernetes have also implemented such features then there should be something impactful for developers then only they may have taken such decisions. Check the following links. https://kubernetes.io/docs/reference/using-api/client-libraries/ https://aws.amazon.com/blogs/containers/introducing-cdk-for-kubernetes/ https://www.cncf.io/projects/cdk-for-kubernetes-cdk8s/ https://www.npmjs.com/package/cdk8s (Check weekly downloads here) https://www.cncf.io/online-programs/end-yaml-engineering-with-cdk8s/ https://github.com/hashicorp/terraform-cdk https://www.terraform.io/cdktf

Expectation of this message from the community.

1. Such feature requests should be discussed in the open and concluded with validated pros & cons,
2. We should roll out a well explained survey on bazel website home page for 1 or 2 month for this feature request and on insights of that survey results we should decide further,

We hope bazel's contributors and such a helpful community will support this initiative for existing and new coming developers to make developers life easy.

Awaiting a positive response. Thanks a lot everyone.

[lberki]

Whereas you are correct that I am just a single person, I do consider myself knowledgeable enough to state with confidence that implementing what you demand would be a large net negative for Bazel and its community for the reasons stated in my comment above.

It might be possible that the benefits are commensurate in the sense that it might be possible that I get hit by a meteorite while sitting here before the end of the day. Given that I have not heard significant demand for configuring Bazel in any language other than Starlark (in fact, I don't remember any instance other than this thread), I highly doubt so and given the high cost, the technical complexity and the dubious benefits, I don't think it's worthwhile to spend our time on validating this. We have all sorts of ideas all the time, we must choose which ones to pursue!

If you feel strongly about this, feel free to reach out to bazel-discuss@googlegroups.com and start a thread; however, given the difficulty of integrating probably any language other than Starlark into Bazel the burden of proof is very high.

Why are you asking us to do this at this point in time? I appreciate your interest in the well-being and success of Bazel. Maybe if I had some more background, I could suggest an approach that benefits both you and Bazel more than the one you suggested?

[beingminimal]

@lberki Thanks a lot for the reply. Here is what i think about this.

Whereas you are correct that I am just a single person, I do consider myself knowledgeable enough to state with confidence that implementing what you demand would be a large net negative for Bazel and its community for the reasons stated in my comment above.

I agree and am happy that someone knowledgeable enough is replying. I agree you are saying it will be a burden for the bazel community but let's first validate it enough that in the future is it really going to be helpful or not? Obviously we are not going to build and ship over the night. So, i think it will be helpful to validate efforts vs impact first and then decide. Here are a few things we can do to validate this

1. Create new discussion in github discussion and let's communicate there OR here by keeping issue open
2. Create a well explained survey and pin it as an issue for awareness and wait for 4 to 6 weeks to get more survey results.
3. You can suggest anything

It might be possible that the benefits are commensurate in the sense that it might be possible that I get hit by a meteorite while sitting here before the end of the day. Given that I have not heard significant demand for configuring Bazel in any language other than Starlark (in fact, I don't remember any instance other than this thread), I highly doubt so and given the high cost, the technical complexity and the dubious benefits, I don't think it's worthwhile to spend our time on validating this. We have all sorts of ideas all the time, we must choose which ones to pursue!

I agree we have all sorts of ideas all the time but we should build only impactful things, but the point is let's validate it enough and let it be decided by the community.

• It might be possible that due to diff framework's build support most of the developers are not aware about bazel.
• Integrate some analytics with bazel cli and check its usage based on location, you will have an idea of location based adoption of bazel, which is near to 0 in many countries.
• It's very easy for large organisations to increase team size and give training and assign dedicated people for bazel implementation but it's not possible for small companies and startups
• Check bazel issue analytics most of the issues have been created by either large organisation's employees or individual contributors.
• And this is the main reason you haven't heard about features requests like this because developers are not enough aware about bazel globally, and even they are aware then they are not able to enforce bazel in CI-CD due to it's long learning curve and learning dependency of starlark
• Like you are knowledgeable enough to answer with technical things, also we have experienced enough pain of enforcing bazel in small team in our company, it's very hard to make developers motivated to adopt new things.
• You can only support typescript officially and the rest of the languages can be supported by the community based on API, what do you say?

Even projects like terraform and kubernetes have also implemented such features then there should be something impactful for developers then only they may have taken such decisions. Check the following links. https://kubernetes.io/docs/reference/using-api/client-libraries/ https://aws.amazon.com/blogs/containers/introducing-cdk-for-kubernetes/ https://www.cncf.io/projects/cdk-for-kubernetes-cdk8s/ https://www.npmjs.com/package/cdk8s (Check weekly downloads here) https://www.cncf.io/online-programs/end-yaml-engineering-with-cdk8s/ https://github.com/hashicorp/terraform-cdk https://www.terraform.io/cdktf

See cdk8s npm module weekly download here.

Only by assuming things we should not validate the need of such features for existing and future developers. And bazel is such a powerful tool which can save a ton of time for developers if we can remove all friction for adoption.

Awareness and 0% friction in adoption of bazel can really create significant impact for developers. Think once and please join the community in this validation and game changing feature for the community.

I hope you will support us in this initiative. Awaiting your response. Thanks a lot for everything.

[sventiffe]

@codecycleteam Bazel's eng lead from Google here. I fully support lberki's points who is the overall Tech Lead on our end both for Bazel and our internal build system, so he does speak indeed with some authority. His single person's voice is representing ~10 years of Bazel and Blaze development. Which does not mean that I agree to everything he'd say :), but I hope it provides some context.

Starlark has been designed for Bazel's use case and it's part of the core design. You are in fact not suggesting a feature only but a massive design change of Bazel.

All that being said, everything can be discussed openly, there are multiple channels such as bazel-discuss, this issue or even Slack.

What I suggest you to do though is to discuss it via a design review. As you are suggesting a design change, I'd expect you to not only identify potential value of the proposal, but also addressing design differences (to Starlark's scope, for example, but also to Bazel's overall design principles like hermeticity).

I'd advice not to base your proposal upon a comparison with Terraform or Kubernetes. Bazel is a build system and has been designed for this use case and your proposal needs to match Bazel's (design) goals, not others' design goals. I have some trouble following your reasoning there.

[beingminimal]

@lberki @sventiffe haha alright guys. I got the whole picture now. but in the future if you guys are considering redesigning bazel architecture and design then please consider such features first.

I appreciate your support and humbleness for this conversation. Thanks a lot for everything.

[beingminimal]

If anyone is looking for a similar approach OR solution then here is the open sourced project wireit by google. Please have a look. Thanks.