Problem with `ScalablyTypedConverterExternalNpmPlugin`

[FloWi]

Hi!

I was coding along with Sébastian Doeraene's great talk and wanted learn more about laminar.

Unfortunately I ran into issues with the ScalablyTypedConverterExternalNpmPlugin when trying to use ScalablyTyped. I learned from your docs that externalNpm needs to be set up but I can't get it to work without exceptions.

I tried both settings in build.sbt

    externalNpm := {
      Process("npm install", baseDirectory.value).!
      baseDirectory.value
    },

and

    externalNpm := {
      Process("yarn", baseDirectory.value).!
      baseDirectory.value
    },

I always get this error:

[error] (stImport) java.lang.NoSuchMethodError: 'scala.collection.mutable.WrappedArray os.list$.apply(os.Path)'

Do you have an idea on what might be wrong?

I stripped down my project into a minimal setup in this repo that also includes more details about the steps to reproduce and the stracktrace in this markdown file

Let me know if you have further questions.

[FloWi]

Just rediscovered the repo to this video, which works fine.

He uses this setting:

https://github.com/sjrd/scalajs-sbt-vite-laminar-chartjs-example/blob/6a015abb0151ab2b8541650fdd300a77ab90326d/build.sbt#L28

I also tried that, but I got the same error. Can someone tell me what's wrong the setup of my project?

[oyvindberg]

Hey there's thanks for the report.

This is incompatible versions of os-lib on the classpath - sbt global classloader strikes again.

If you disable the other SBT plugins and renewable them one by one you'll find the conflict. Workaround is to disable that, the fix is to bump os-lib in ST

[FloWi]

Wow - I would have never found that bug. Thanks a lot! I upgraded the scalajs plugin from 1.13.1 to 1.14.0 and now everything works.

[FloWi]

I upgraded the scalajs plugin from 1.13.1 to 1.14.0 and now everything works.

In case anyone reads this, it's actually not true. I don't really know, why it worked and now stopped working again. I just added a new js-dependency and now I get the same error.

Have to test this in a clean project to try to reproduce it. Soo annoying what sbt does here. 😭

[FloWi]

Ah, I found the incompatible plugin.

addSbtPlugin("com.disneystreaming.smithy4s" % "smithy4s-sbt-codegen" % "0.18.3")

I created a repo with a minimal example and a README to reproduce. Would you mind taking a look, @oyvindberg and ideally give me some advice on what I can do to get both to run?

I don't know how I can find out the dependencies of a plugin and which one needs to be upgraded. The authors of the other plugin are very helpful on their discord channel. I'm sure they can provide a fix in case something can be done on their side. But I actually don't know what to tell them 🙈, since I'm a plugin-n00b.

[FloWi]

I think I found the root cause. I wanted to see what an sbt-plugin looks like on disk and found the coursier cache folders.

It looks like your plugin requires sbt version 1.9.6 and the smithy4s plugin requires sbt version 1.8.3.

scalablytyped converter

cat ~/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scalablytyped/converter/sbt-converter_2.12_1.0/1.0.0-beta43/sbt-converter-1.0.0-beta43.pom | grep -B1 -A4 org.scala-sbt

        <dependency>
            <groupId>org.scala-sbt</groupId>
            <artifactId>sbt</artifactId>
            <version>1.9.6</version>
            <scope>provided</scope>
        </dependency>

smithy4s

cat ~/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/disneystreaming/smithy4s/smithy4s-sbt-codegen_2.12_1.0/0.18.3/smithy4s-sbt-codegen-0.18.3.pom | grep -B1 -A4 org.scala-sbt

        <dependency>
            <groupId>org.scala-sbt</groupId>
            <artifactId>sbt</artifactId>
            <version>1.8.3</version>
            <scope>provided</scope>
        </dependency>

Is that the correct place to look for inconsistencies?

[FloWi]

Looks like my findings were not the cause of my problems.

I asked on the smithy4s discord and @armanbilge provided some insights.

It looks like scalablytyped is indeed using an old version of Haoyi Li's os-lib. I didn't know about that library and assumed that you were talking about sbt when you mentioned "os-lib".

~I think I can work around the issue by using my minimal-example project to execute stImport with the new js-dependency and then go to my other project and be happy with the generated jar.~ Didn't work.

I don't know how automated your release process is. If it's not a big deal I'd appreciate a new release with an updated os-lib.

[oyvindberg]

Thanks for your work here. I'll go ahead and merge it.

These dependency updates in sbt are a mess - once a library without binary compatibility guarantees is used by an sbt plugin, all other sbt plugins need to be on that same version. Once one plugin updates their version, all other plugins need to update as well. It's madness, really.

[FloWi]

It's madness, really.

100% agree! It was deep and dark in the rabbit hole. There's a lot of hope involved ;-)

A big scalathankyou for your work! 🙏

Your plugin makes coding a lot easier. After I got it going again in my project it only took ~1h to build my shiny new feature with the help of the js library!