search

VirtusLab / besom

Besom - a Pulumi SDK for Scala. Also, incidentally, a broom made of twigs tied round a stick. Brooms and besoms are used for protection, to ward off evil spirits, and cleansing of ritual spaces.
https://virtuslab.github.io/besom/
Apache License 2.0
124 stars 7 forks source link

Scala Language Plugin for Pulumi should support all the executors that Java Language Plugin does #303

Closed dubinsky closed 8 months ago

dubinsky commented 11 months ago

Getting started creates an impression that Scala CLI and Scala Language Plugin are required to use Besom. In fact, if the project uses one of the execution methods supported by the Pulumi Java Language Plugin (Gradle, Maven, JAR, JBang or SBT), Besom SDK can be used with runtime: java in Pulumi.yaml, just as Pulumi Java SDK can be used from Scala code that way. Indeed, if the project uses Gradle, Maven or JBang, Scala Language Plugin must not be used: while retaining support for JAR and SBT execution, it drops support for Gradle etc. in favour of Scala CLI. Scala Language Plugin seems to be required only if the project uses Scala CLI, and although this setup is recommended, I think the ability to use Besom with - say - Gradle should be at least mentioned ;)

lbialy commented 11 months ago

That is definitely true (in fact, before we've had language plugin we used runtime: java). I see no reason not to document that and I didn't include that fact in the first draft of docs only because I strived for a lean, uncluttered and frictionless experience (as all DX should be!). I think we can add sections regarding the use of other build tools available on the platform. In fact, we should also support mill on Scala language plugin's side if we don't do it yet.

On Thu 16. Nov 2023 at 21:06, Leonid Dubinsky @.***> wrote:

Getting started https://virtuslab.github.io/besom/docs/getting_started/ creates an impression that Scala CLI and Scala Language Plugin are required to use Besom. In fact, if the project uses one of the execution methods supported by the Pulumi Java Language Plugin (Gradle, Maven, JAR, JBang or SBT), Besom SDK can be used with runtime: java in Pulumi.yaml, just as Pulumi Java SDK can be used from Scala code that way. Indeed, if the project uses Gradle, Maven or JBang, Scala Language Plugin must not be used: while retaining support for JAR and SBT execution, it drops support for Gradle etc. in favour of Scala CLI. Scala Language Plugin seems to be required only if the project uses Scala CLI, and although this setup is recommended, I think the ability to use Besom with - say - Gradle should be at least mentioned ;)

— Reply to this email directly, view it on GitHub https://github.com/VirtusLab/besom/issues/303, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACBVNUR5MS4ZVYZVNOMJCI3YEZW2LAVCNFSM6AAAAAA7OXAXUWVHI2DSMVQWIX3LMV43ASLTON2WKOZRHE4TONRXGY4DKMQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>

dubinsky commented 11 months ago

@lbialy one way to remove some friction is to copy from the Java language plugin to the Scala one not just the JAR and SBT executors but all of them (Gradle, Maven and JBang too); this way, by switching from the Java runtime to the Scala one one would gain the ability to use Scala CLI - but would not lose the ability to use - say - Gradle...

dubinsky commented 11 months ago

I am not sure what the Scala language plugin does, but I suspect that it provides more than just the executors. I can easily see how Besom might require the use of the Scala language plugin, and I am more than willing to switch to runtime: scala - as long as I do not lose the ability to use the build tool of my choosing that I can use now with runtime: java.

lbialy commented 11 months ago

Yeah, so the long game plan for language plugin is to support way more features as planned by Pulumi, programmatic code generation on demand on user's machine for starters. Current impl is just the same go app as pulumi-java with changes so we could backport gradle etc very easily I think. In the long run we want to rewrite it to Scala and compile with native-image.

On Thu 16. Nov 2023 at 21:33, Leonid Dubinsky @.***> wrote:

I am not sure what the Scala language plugin does, but I suspect that it provides more than just the executors. I can easily see how Besom might require the use of the Scala language plugin, and I am more than willing to switch to runtime: scala - as long as I do not lose the ability to use the build tool of my choosing that I can use now with runtime: java.

— Reply to this email directly, view it on GitHub https://github.com/VirtusLab/besom/issues/303#issuecomment-1815270280, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACBVNUU4IQDJLI7DIMRL6GTYEZ2CHAVCNFSM6AAAAAA7OXAXUWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMJVGI3TAMRYGA . You are receiving this because you were mentioned.Message ID: @.***>

dubinsky commented 11 months ago

plan for language plugin is to support way more features

Cool!

we could backport gradle etc very easily

From what I see, JAR and SBT support were just copied, so it's just more copying ;) Ability to switch to runtime: scala without breaking Gradle is going to greatly improve DX ;)

pawelprazak commented 10 months ago

The fact that those language plugins are mostly interchangeable right now is not intentional.

The Java SDK and it's language plugin have different lifecycle and ownership - it is maintained by Pulumi The Scala SDK a.k.a. Besom is maintained by VirtusLab.

This is the first reason why we cannot guarantee the interchangeability of language plugins in the future.

The second, and more important fact is, that the language plugin has more responsibilities then just build tools. Please have a look at the gRPC definition for reference. This is not implemented in both mentioned plugins at the moment of writing, but it will be implemented in Besom, rather sooner than later. This will make the interchangeability impossible by design.

Another internal goal of Besom is to rewrite the language plugin to Scala 3 with Graalvm static binary, this will allow us to embed the codegen module and maybe even the scala-cli itself, it sounds tempting.

As for the tooling support, I've always thought the Pulumi way of doing language plugins is fundamentally incompatible with the "JVM-way", and the dependency should be IMO reversed.

Once we have Automation API implemented, every build tool can easily create a plugin for Besom, just like most JVM people like it (it was highly requested by Java SDK users to have a gradle and maven plugins).

So to summarize, I'm in favor of the following plan:

Also we'd like to prioritize delivery of feature parity with upstream - this is the main goal, so we must be laser focused on the core of Besom.

pawelprazak commented 10 months ago

As for something immediately actionable, we probably should document that Java SDK and Besom are completely separate and can interact in unpredictable and "interesting" ways when used together.

dubinsky commented 10 months ago

@pawelprazak I understand that to continue to be able to use Besom I need to use runtime: scala, and I am happy - eager even! - to do it. I also understand that when the Automation API becomes available, build-tool integration using it will become possible, and I am happy to look at the possibility of developing a Gradle plugin for such integration.

I do not quite understand the impact of this statement:

I've always thought the Pulumi way of doing language plugins is fundamentally incompatible with the "JVM-way", and the dependency should be IMO reversed.

Does this mean that at some point the ability to use Besom via the pulumi CLI delegating to the Besom's Scala Language Plugin is going to get removed completely?

Before all of the above transpires, what is the way for me right now to use Besom properly, with the Scala Language Plugin, if I am using Gradle as a build tool and the language plugin does not (and will not) support it? Should I point to my executable JAR in Pulumi.yaml's runtime.option.binary or is there a better way? Or am I out of luck and not a part of the target audience of the early adopters? ;)

Thanks!

pawelprazak commented 10 months ago

I'm sorry about my confusing remarks, I was just thinking out loud. There are no specific decisions made yet regarding language plugin, other than that we'd like to find a way to give the community the best support for build tools possible, and the exact shape of it, I hope, will be made clear by user feedback, like yours :)

This is a very good point you are making, about early adoption, and I see no reason no to port Java lang host Gradle support, @lbialy WDYT?

dubinsky commented 9 months ago

Will this be in the next release? ;)

golosegor commented 9 months ago

it would be really nice to have this resolved...

lbialy commented 9 months ago

Ok, I'll take a look at this. I think we can include this in the upcoming 0.2.0.

On Wed 3. Jan 2024 at 18:37, Egor Goloshchapov @.***> wrote:

it would be really nice to have this resolved...

— Reply to this email directly, view it on GitHub https://github.com/VirtusLab/besom/issues/303#issuecomment-1875731394, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACBVNUUQLS3GCCGUUFXZVGDYMWJONAVCNFSM6AAAAAA7OXAXUWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNZVG4ZTCMZZGQ . You are receiving this because you were mentioned.Message ID: @.***>

pawelprazak commented 8 months ago

Reopening until release

lbialy commented 8 months ago

Release of 0.2.0 which brings support for Maven and Gradle is underway. Closing the issue.