Scala.js linking to support multiple modules

[shishkin]

Hi! I'm running into an issue trying to compile two top-level exports @JSExportTopLevel(moduleID = "foo", name = "handler") and @JSExportTopLevel(moduleID = "bar", name = "handler").

Here is the error I see:

1 targets failed
lambda.fastOpt The linker produced a result not supported by the pre v1.3.0 legacy API. Call the overload taking an OutputDirectory instead. Linking returned more than one public module. Full report:
Report(
  publicModules = [
    Module(
  moduleID      = foo,
  jsFileName    = foo.js,
  sourceMapName = Some(foo.js.map),
  moduleKind    = CommonJSModule,
),
Module(
  moduleID      = bar,
  jsFileName    = bar.js,
  sourceMapName = Some(bar.js.map),
  moduleKind    = CommonJSModule,
)
  ],
)

I've already set the CommonJS ModuleKind:

  def scalaVersion = "3.1.0"
  def scalaJSVersion = "1.8.0"
  def moduleKind = ModuleKind.CommonJSModule
  def esFeatures = ESFeatures.Defaults.withESVersion(ESVersion.ES2020)

This seems to come from https://github.com/scala-js/scala-js/blob/v1.7.1/linker-interface/shared/src/main/scala/org/scalajs/linker/interface/Linker.scala#L39-L57. While Mill worker assumes the output to be a single file: https://github.com/com-lihaoyi/mill/blob/0.10.0/scalajslib/worker/1/src/ScalaJSWorkerImpl.scala#L122-L125.

I could try to make the change that scala-js linker is proposing, but would need some guidance about the potential impact of the change. Thanks in advance for any help here.

[lolgab]

Fixed in #1714. Now fastOpt and fullOpt emit other modules as well. Available since version 0.10.0-30-0f1a37 There is a single caveat which is that fastOpt will continue to link a main out.js file and fastOpt return type points to that file. So cache is not invalidated properly when the out.js doesn't change but one of the modules does.

[shishkin]

Thanks @lolgab, this is awesome! Now compilation works. I'm still having an issue running tests when top-level export is present in the project:

[96/97] lambda.test.fastOptTest
Found multiple public modules but module support is disabled: [hello, out]
  exported to JavaScript with @JSExport
Linker: Compute reachability: 293622 us
Linker: 309311 us
1 targets failed
lambda.test.fastOptTest There were linking errors

(No additional logs about linking errors and out/lambda/test/fastOptTest.dest is empty)

This is the build.sc:

object lambda extends ScalaJSModule {
  def scalaVersion = "3.1.0"
  def scalaJSVersion = "1.8.0"
  def moduleKind = ModuleKind.ESModule
  def esFeatures = ESFeatures.Defaults.withESVersion(ESVersion.ES2020)

  object test extends Tests with TestModule.Munit {
    def ivyDeps = Agg(ivy"org.scalameta::munit::0.7.29")
  }
}

[lolgab]

@shishkin I think that Tests trait doesn't inherit the moduleKind of the main module. The same probably applies for esFeatures. This should work:

object lambda extends ScalaJSModule { outer =>
  def scalaVersion = "3.1.0"
  def scalaJSVersion = "1.8.0"
  def moduleKind = ModuleKind.ESModule
  def esFeatures = ESFeatures.Defaults.withESVersion(ESVersion.ES2020)

  object test extends Tests with TestModule.Munit {
    def ivyDeps = Agg(ivy"org.scalameta::munit::0.7.29")
    override def moduleKind = outer.moduleKind
    override def esFeatures = outer.esFeatures
  }
}

[shishkin]

Thanks @lolgab, that makes sense. Although setting module kind on the test module directly results in another error. Running ESModule on Node fails due to output files having .js extensions instead of .mjs. And changing it to CommonJSModule fails with this:

[96/96] lambda.test.test
Starting process: node
node:internal/modules/cjs/loader:936
  throw err;
  ^

Error: Cannot find module './internal-785f0ca7a63c1c87ea4ee85dba7faee4d2482140.js'
Require stack:
- /Users/serega/code/dojo/scalajs/scalajs-lambda/[stdin]
    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:933:15)
    at Function.Module._load (node:internal/modules/cjs/loader:778:27)
    at Module.require (node:internal/modules/cjs/loader:1005:19)
    at require (node:internal/modules/cjs/helpers:102:18)
    at /Users/serega/code/dojo/scalajs/scalajs-lambda/out/lambda/test/fastOptTest.dest/out.js:2:64
    at Script.runInThisContext (node:vm:129:12)
    at Object.runInThisContext (node:vm:305:38)
    at [stdin]:14:25
    at Script.runInThisContext (node:vm:129:12)
    at Object.runInThisContext (node:vm:305:38) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [ '/Users/serega/code/dojo/scalajs/scalajs-lambda/[stdin]' ]
}
1 targets failed
lambda.test.test org.scalajs.testing.common.RPCCore$ClosedException: org.scalajs.testing.adapter.JSEnvRPC$RunTerminatedException
org.scalajs.testing.adapter.JSEnvRPC$RunTerminatedException
org.scalajs.jsenv.ExternalJSRun$NonZeroExitException: exited with code 1

Guessing from the error, test runner runs Node from the project root directory passing the main module JS via STDIN, so node is not able to locate module dependencies from the out/lambda/test/fastOptTest.dest directory.