denoland / deno

A modern runtime for JavaScript and TypeScript.
https://deno.com
MIT License
97.44k stars 5.37k forks source link

Deno panics when some npm/node feature is used with invalid UTF-8 #23792

Closed lambdalisue closed 5 months ago

lambdalisue commented 6 months ago

Version: Deno 1.43.3

Using msgpackr by using npm: flag with invalid UTF-8 string (\x80\xfd...) panic Deno like


============================================================
Deno has panicked. This is a bug in Deno. Please report this
at https://github.com/denoland/deno/issues/new.
If you can reliably reproduce this panic, include the
reproduction steps and re-run with the RUST_BACKTRACE=1 env
var set and include the backtrace in your report.

Platform: macos aarch64
Version: 1.43.3
Args: ["/Users/alisue/.local/share/mise/installs/deno/latest/bin/deno", "run", "--allow-all", "--unstable-ffi", "--unstable-temporal", "--unstable-kv", "/Users/alisue/ogh/vim-denops/deno-denops-std/test.ts"]

thread 'main' panicked at cli/napi/js_native_api.rs:867:33:
called `Result::unwrap()` on an `Err` value: Utf8Error { valid_up_to: 0, error_len: Some(1) }
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

I'm not familiar with but the library seems to use some node's extra feature to accelerate performance and that cause this panic when used with invalid UTF-8 string.

How to reproduce

Run the following script.

// Adding following environment variable fix the issue
// https://github.com/kriszyp/msgpackr/blob/b306efd2a576911619ebc873ab04de494caeeb4f/node-index.js#L11
//Deno.env.set("MSGPACKR_NATIVE_ACCELERATION_DISABLED", "true");

// I use dynamic import here but panic occurred with static import as well.
const { Unpackr } = await import("npm:msgpackr");

const unpackr = new Unpackr({
  useRecords: false,
  structures: [],
});

// The following Uint8Array is for the following strucuter.
// I think the "\x80ýS" part cause the panic.
// [
//   [
//     1,
//     7,
//     null,
//     {
//       lnum: 7,
//       script: 0,
//       mode: " ",
//       silent: 0,
//       noremap: 0,
//       lhs: "<Plug>(test-denops-std-map)",
//       lhsraw: "\x80ýS(test-denops-std-map)",
//       nowait: 0,
//       expr: 0,
//       sid: 34,
//       rhs: "Hello",
//       buffer: 0
//     }
//   ]
// ]
const chunk = new Uint8Array([
  148,
  1,
  7,
  192,
  140,
  164,
  108,
  110,
  117,
  109,
  7,
  166,
  115,
  99,
  114,
  105,
  112,
  116,
  0,
  164,
  109,
  111,
  100,
  101,
  161,
  32,
  166,
  115,
  105,
  108,
  101,
  110,
  116,
  0,
  167,
  110,
  111,
  114,
  101,
  109,
  97,
  112,
  0,
  163,
  108,
  104,
  115,
  187,
  60,
  80,
  108,
  117,
  103,
  62,
  40,
  116,
  101,
  115,
  116,
  45,
  100,
  101,
  110,
  111,
  112,
  115,
  45,
  115,
  116,
  100,
  45,
  109,
  97,
  112,
  41,
  166,
  108,
  104,
  115,
  114,
  97,
  119,
  184,
  128,
  253,
  83,
  40,
  116,
  101,
  115,
  116,
  45,
  100,
  101,
  110,
  111,
  112,
  115,
  45,
  115,
  116,
  100,
  45,
  109,
  97,
  112,
  41,
  166,
  110,
  111,
  119,
  97,
  105,
  116,
  0,
  164,
  101,
  120,
  112,
  114,
  0,
  163,
  115,
  105,
  100,
  34,
  163,
  114,
  104,
  115,
  165,
  72,
  101,
  108,
  108,
  111,
  166,
  98,
  117,
  102,
  102,
  101,
  114,
  0,
]);

// It panic Deno
console.log(unpackr.unpackMultiple(chunk));
devsnek commented 5 months ago

This should be fixed in https://github.com/denoland/deno/pull/24101, let us know if you run into any other issues.