swc-project / swc

Rust-based platform for the Web
https://swc.rs
Apache License 2.0
31.28k stars 1.23k forks source link

`swc_core@4.0.3` and `@swc/core@1.8.x` cannot run Wasm plugins that uses `SourceFile` #9711

Closed fpapado closed 2 weeks ago

fpapado commented 3 weeks ago

Describe the bug

Follow the reproduction at https://github.com/fpapado/swc-1.8.0-and-plugin-styled-components-4.0.0.

Using @swc/core@1.8.0 together with @swc/plugin-styled-components@4.0.0 and then attempting to transpile a file leads to a panic. This seems contrary to https://plugins.swc.rs/versions/range/19, which shows @swc/core@1.8.0 and @swc/plugin-styled-components@4.0.0 as compatible.

SWC_DEBUG=1 pn swc --sync index.js

thread '<unnamed>' panicked at /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/swc_plugin_proxy-3.0.0/src/memory_interop/read_returned_result_from_host.rs:110:10:
Returned value should be serializable: wasm plugin bytecheck failed "check bytes error: check failed for tuple struct member 0: check failed for struct member source_file: check failed for enum tuple variant Some: check failed for tuple struct member 0: check failed for struct member lazy: invalid tag for enum: 131"
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Panic: PanicHookInfo { payload: Any { .. }, location: Location { file: "/Users/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/swc-4.0.0/src/plugin.rs", line: 169, col: 14 }, can_unwind: true, force_no_backtrace: false }
Backtrace:    0: <unknown>
   1: <unknown>
   2: <unknown>
   3: _napi_register_module_v1
   4: _napi_register_module_v1
   5: _napi_register_module_v1
   6: _napi_register_module_v1
   7: _wasmer_vm_imported_memory32_atomic_notify
   8: _wasmer_vm_imported_memory32_atomic_notify
   9: _napi_register_module_v1
  10: _napi_register_module_v1
  11: _napi_register_module_v1
  12: _napi_register_module_v1
  13: _napi_register_module_v1
  14: _napi_register_module_v1
  15: _napi_register_module_v1
  16: <unknown>
  17: <unknown>

failed to handle: failed to invoke plugin: failed to invoke plugin on 'Some("index.js")'

Caused by:
    0: failed to invoke `@swc/plugin-styled-components` as js transform plugin at @swc/plugin-styled-components
    1: failed to run Wasm plugin transform. Please ensure the version of `swc_core` used by the plugin is compatible with the host runtime. See the documentation for compatibility information. If you are an author of the plugin, please update `swc_core` to the compatible version.

                       Note that if you want to use the os features like filesystem, you need to use `wasi`. Wasm itself does not have concept of filesystem.

                       https://swc.rs/docs/plugin/selecting-swc-core

                       See https://plugins.swc.rs/versions/from-plugin-runner/3.0.0 for the list of the compatible versions.

                       Build info:
                           Date: 2024-11-04
                           Timestamp: 2024-11-04T01:54:05.821856000Z

                       Version info:
                           swc_plugin_runner: 3.0.0
                           Dependencies: anyhow 1.0.92,codspeed-criterion-compat 2.7.2,criterion 0.5.1,enumset 1.1.5,futures 0.3.31,once_cell 1.20.2,parking_lot 0.12.3,serde 1.0.214,serde_json 1.0.132,swc_atoms 2.0.0,swc_common 3.0.0,swc_css_ast 3.0.0,swc_css_parser 3.0.0,swc_ecma_ast 3.0.0,swc_ecma_loader 3.0.0,swc_ecma_parser 4.0.0,swc_ecma_visit 3.0.0,swc_malloc 1.0.0,swc_plugin_proxy 3.0.0,testing 3.0.0,tokio 1.41.0,tracing 0.1.40,vergen 9.0.1,virtual-fs 0.16.0,wasmer 4.3.7,wasmer-cache 4.3.7,wasmer-compiler-cranelift 4.3.7,wasmer-wasix 0.27.0

    2: RuntimeError: unreachable
Error: Failed to compile 1 file with swc.
    at Object.assertCompilationResult (/Users/fotis/swc-1.8.0-and-plugin-styled-components-4.0.0/node_modules/.pnpm/@swc+cli@0.5.0_@swc+core@1.8.0/node_modules/@swc/cli/lib/swc/util.js:165:15)
    at files (/Users/fotis/swc-1.8.0-and-plugin-styled-components-4.0.0/node_modules/.pnpm/@swc+cli@0.5.0_@swc+core@1.8.0/node_modules/@swc/cli/lib/swc/file.js:205:19)
    at async _default (/Users/fotis/swc-1.8.0-and-plugin-styled-components-4.0.0/node_modules/.pnpm/@swc+cli@0.5.0_@swc+core@1.8.0/node_modules/@swc/cli/lib/swc/file.js:224:9)

Input code

import {styled} from 'styled-components';

const MyComponent = styled.div`
  color: purple;
`

Config

{
  "$schema": "https://swc.rs/schema.json",
  "jsc": {
    "experimental": {
      "plugins": [
        ["@swc/plugin-styled-components", {}]
      ]
    }
  }
}

Playground link (or link to the minimal reproduction)

https://github.com/fpapado/swc-1.8.0-and-plugin-styled-components-4.0.0

SWC Info output

/Users/fotis/Library/Application Support/fnm/node-versions/v20.18.0/installation/lib/node_modules/corepack/dist/lib/corepack.cjs:22373 throw new UsageError(This project is configured to use ${result.spec.name} because ${result.target} has a "packageManager" field); ^

UsageError: This project is configured to use pnpm because /Users/fotis/swc-1.8.0-and-plugin-styled-components-4.0.0/package.json has a "packageManager" field at Engine.findProjectSpec (/Users/fotis/Library/Application Support/fnm/node-versions/v20.18.0/installation/lib/node_modules/corepack/dist/lib/corepack.cjs:22373:21) at async Engine.executePackageManagerRequest (/Users/fotis/Library/Application Support/fnm/node-versions/v20.18.0/installation/lib/node_modules/corepack/dist/lib/corepack.cjs:22404:24) at async Object.runMain (/Users/fotis/Library/Application Support/fnm/node-versions/v20.18.0/installation/lib/node_modules/corepack/dist/lib/corepack.cjs:23096:5) { clipanion: { type: 'usage' } }

Node.js v20.18.0

Operating System:
    Platform: darwin
    Arch: arm64
    Machine Type: arm64
    Version: Darwin Kernel Version 24.0.0: Mon Aug 12 20:51:54 PDT 2024; root:xnu-11215.1.10~2/RELEASE_ARM64_T6000
    CPU: (10 cores)
        Models: Apple M1 Pro

Binaries:
    Node: 20.18.0
    npm: 10.8.2
    Yarn: N/A
    pnpm: 9.12.3

Relevant Packages:
    @swc/core: 1.8.0
    @swc/helpers: N/A
    @swc/types: N/A

SWC Config:
    output: N/A
    .swcrc path: N/A

Next.js info:
    output: N/A

Expected behavior

I expected swc to transpile the code without errors around the plugin's compatibility with core, as listed in https://plugins.swc.rs/versions/range/19.

Actual behavior

SWC exits with a panic

Version

1.8.0

Additional context

Please let me know if there is any other information that I can provide, and I will get back to you promptly 😌

kdy1 commented 3 weeks ago

I think there's a UB in the code related to rkyv. When I run it via cargo test, it works.

If I add logging code and run it via node.js, it hangs.

Log ``` ❯ npx swc index.js try_serialize: [0, 0, 0, 0, 9, 0, 0, 0, 15, 0, 0, 0, 9, 0, 0, 0, 15, 0, 0, 0, 2, 0, 0, 0, 115, 116, 121, 108, 101, 100, 0, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 115, 116, 121, 108, 101, 100, 45, 99, 111, 109, 112, 111, 110, 101, 110, 116, 115, 39, 115, 116, 121, 108, 101, 100, 45, 99, 111, 109, 112, 111, 110, 101, 110, 116, 115, 39, 22, 0, 0, 0, 41, 0, 0, 0, 17, 0, 0, 0, 212, 255, 255, 255, 1, 0, 0, 0, 19, 0, 0, 0, 217, 255, 255, 255, 77, 121, 67, 111, 109, 112, 111, 110, 101, 110, 116, 0, 14, 0, 0, 0, 64, 0, 0, 0, 70, 0, 0, 0, 2, 0, 0, 0, 115, 116, 121, 108, 101, 100, 0, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 8, 0, 0, 0, 64, 0, 0, 0, 74, 0, 0, 0, 180, 255, 255, 255, 0, 0, 0, 0, 71, 0, 0, 0, 74, 0, 0, 0, 100, 105, 118, 0, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 9, 99, 111, 108, 111, 114, 58, 32, 112, 117, 114, 112, 108, 101, 59, 10, 10, 9, 99, 111, 108, 111, 114, 58, 32, 112, 117, 114, 112, 108, 101, 59, 10, 0, 0, 75, 0, 0, 0, 92, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 17, 0, 0, 0, 204, 255, 255, 255, 17, 0, 0, 0, 213, 255, 255, 255, 74, 0, 0, 0, 93, 0, 0, 0, 180, 255, 255, 255, 0, 0, 0, 0, 208, 255, 255, 255, 1, 0, 0, 0, 0, 0, 0, 0, 17, 0, 0, 0, 64, 0, 0, 0, 93, 0, 0, 0, 0, 0, 0, 0, 80, 255, 255, 255, 0, 0, 0, 0, 0, 0, 0, 0, 200, 255, 255, 255, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 50, 0, 0, 0, 93, 0, 0, 0, 0, 0, 0, 0, 50, 0, 0, 0, 61, 0, 0, 0, 2, 0, 0, 0, 11, 0, 0, 0, 188, 254, 255, 255, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 144, 255, 255, 255, 0, 0, 0, 0, 44, 0, 0, 0, 93, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 180, 255, 255, 255, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 42, 0, 0, 0, 228, 253, 255, 255, 1, 0, 0, 0, 76, 254, 255, 255, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 17, 0, 0, 0, 2, 0, 0, 0, 164, 255, 255, 255, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 93, 0, 0, 0, 140, 255, 255, 255, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] try_serialize: [99, 111, 109, 109, 111, 110, 95, 112, 108, 117, 103, 105, 110, 95, 116, 114, 97, 110, 115, 102, 111, 114, 109, 44, 101, 99, 109, 97, 95, 97, 115, 116 ```

If I run it without logging code, it crashes with the error message in the issue body.

@kwonoj What do you think? Should we update rkyv (and wasmer) first?

kwonoj commented 2 weeks ago

Do we have exact change caused this regression? If we revert those what we'll lose?

I'm in for rkyv / wasmer update and it should happen, but rkyv had major breaking so update would require some code changes - also we are't 100% sure if update will fix this issue.

kdy1 commented 2 weeks ago

The problematic patch is https://github.com/swc-project/swc/pull/9696, which makes source file analysis of swc_common lazy, improving the total runtime by 2% ~ 5%. I introduced CacheCell with the PR, and it seems like it's problematic. Maybe code for rkyv-impl at https://github.com/swc-project/swc/blob/499c8034133417dd01e864c1e11844f6ce9215dc/crates/swc_common/src/cache.rs#L35-L115 is problematic, but I failed to debug.

kwonoj commented 2 weeks ago

Isn't rspack attempt to upgrade to latest swc in the next release? Blocking all plugin sounds like a deal breaker; maybe cost those perf improvement and revert, then retry after rkyv / wasmer update?

kdy1 commented 2 weeks ago

It's not all Wasm plugin, but I think we should revert it anyway. I left a question on rkyv Discord, so if it's a real UB, I'll revert the PR and reapply it after updating rkyv

kdy1 commented 2 weeks ago

I triggered the publish pipeline for @swc/core@v1.9.0

sleyland commented 2 weeks ago

Using @swc/core@v1.9.1 and various plugins, e.g. @swc/plugin-styled-components@v5.0.0 has now resolved the issue mentioned above for our environment.

fpapado commented 2 weeks ago

Can confirm that this works for us as well. Thank you for the quick fix 🎉