Encoding difference in Middleware and Edge API Route

dferber90 commented 1 year ago

Verify canary release

[X] I verified that the issue exists in the latest Next.js canary release

Provide environment information

Operating System:
  Platform: darwin
  Arch: arm64
  Version: Darwin Kernel Version 22.2.0: Fri Nov 11 02:03:51 PST 2022; root:xnu-8792.61.2~4/RELEASE_ARM64_T6000
Binaries:
  Node: 17.4.0
  npm: 8.3.1
  Yarn: 1.22.15
  pnpm: 7.15.0
Relevant packages:
  next: 13.1.3-canary.4
  eslint-config-next: N/A
  react: 18.2.0
  react-dom: 18.2.0

Which area(s) of Next.js are affected? (leave empty if unsure)

Middleware / Edge (API routes, runtime)

Link to the code that reproduces this issue

https://github.com/uncurated-tests/debug-statsig

To Reproduce

pnpm install
pnpm build

cat .next/server/pages/api/exp.js
cat .next/server/middleware.js

Describe the Bug

I’m running into what seems to be an encoding difference between middleware and Edge API Routes

Specifically I’m importing a module which contains Buffer.from("someString") and this value of someString is encoded differently in .next/server/pages/api/exp.js and .next/server/middleware.js. The actual value can be seen here.

Original value in third-party package is a binary encoded in a buffer:

Buffer.from("/~--USCNAUJPTHINMYKRSGHKTWKHPHVNNOESFRNLCZGBDEATCHBRITGRPLBEIEDKPTSEGHTRRUCMZAFIAEHUJOROLUARUGAMTZBIUYCLBGUAEGCAILQAMDHRSCIQLVEEUZSKKZGEALPSSACYMTLTCRIRBHMXCOSYLBAZZWZMOMRSISSIMKLIJEBAPEKGIMGGGILYYEBYREJMGPMQKWLKSZCDBTBNPMPALAGUMPDOIDNGNZECVEPRBOVIBDPKPGTLSBVUFJCKTONPKEMOTTLSMAVGKNAGVCKYLCMMGDCWBBBSPYGTUMDMTMTKMVAFNCMNWFSMMESVADMCGLBZTJFOHTMFLRMUBWTNMGAONACISDMWGAMLBJTDCVRWCGMZGMGNBFSOSLNECFTGSSGQSNDZASMRDJKMIOYTNRWSFMPFHNNIBMGFNUTVPWMHKIKPAWCUSRGYVASTETERGWFKBLAITCSXAXAQNFBQMSGS**ó��\b @>

Value in .next/server/middleware.js (same as above)

Buffer.from("/~--USCNAUJPTHINMYKRSGHKTWKHPHVNNOESFRNLCZGBDEATCHBRITGRPLBEIEDKPTSEGHTRRUCMZAFIAEHUJOROLUARUGAMTZBIUYCLBGUAEGCAILQAMDHRSCIQLVEEUZSKKZGEALPSSACYMTLTCRIRBHMXCOSYLBAZZWZMOMRSISSIMKLIJEBAPEKGIMGGGILYYEBYREJMGPMQKWLKSZCDBTBNPMPALAGUMPDOIDNGNZECVEPRBOVIBDPKPGTLSBVUFJCKTONPKEMOTTLSMAVGKNAGVCKYLCMMGDCWBBBSPYGTUMDMTMTKMVAFNCMNWFSMMESVADMCGLBZTJFOHTMFLRMUBWTNMGAONACISDMWGAMLBJTDCVRWCGMZGMGNBFSOSLNECFTGSSGQSNDZASMRDJKMIOYTNRWSFMPFHNNIBMGFNUTVPWMHKIKPAWCUSRGYVASTETERGWFKBLAITCSXAXAQNFBQMSGS**ó��\b @>

Value in .next/server/pages/api/exp.js (wrong encoding)

Buffer.from("/~--USCNAUJPTHINMYKRSGHKTWKHPHVNNOESFRNLCZGBDEATCHBRITGRPLBEIEDKPTSEGHTRRUCMZAFIAEHUJOROLUARUGAMTZBIUYCLBGUAEGCAILQAMDHRSCIQLVEEUZSKKZGEALPSSACYMTLTCRIRBHMXCOSYLBAZZWZMOMRSISSIMKLIJEBAPEKGIMGGGILYYEBYREJMGPMQKWLKSZCDBTBNPMPALAGUMPDOIDNGNZECVEPRBOVIBDPKPGTLSBVUFJCKTONPKEMOTTLSMAVGKNAGVCKYLCMMGDCWBBBSPYGTUMDMTMTKMVAFNCMNWFSMMESVADMCGLBZTJFOHTMFLRMUBWTNMGAONACISDMWGAMLBJTDCVRWCGMZGMGNBFSOSLNECFTGSSGQSNDZASMRDJKMIOYTNRWSFMPFHNNIBMGFNUTVPWMHKIKPAWCUSRGYVASTETERGWFKBLAITCSXAXAQNFBQMSGS**\xf3\0\0\x01\x01\x01\x02\x03\x03\x04\x04\b\x03\x10\x05 \x03@\x05\x80\x06\x01\x03\x01\x02>\x03@\x05\x80\x06\x03\x03\x01\x04|\x03\x80\x06\xf2\0\x01\x03\x01\x04\x7f\x03\x80\x06\xf2\0\x01\x05\xf2\0\x02\x07\xf2\0\x01\x03\xf2\0\x01\b\n\x03\x01\x04u\x03\x80\x06\xf2\0\x01 \xf2\0\x04\x03@ \xf24\x02\x01\xf2\x8c\x01   \xf2\0\x01\x06\xf2\0\x01\x05\xf2\0\x02\x07\xf2\0\b\x03\x80\b@\n\b\v\x05\n\x01\v\x02\n\b\f\x02   \x01\v\x02  \x01\v\x02

Note the \x escaped values like \xf3\0\0\x01\x01\x01\x02\x03\x03\x04\x04\b\x03\x10\x05 \x03@\x05\x80\x06\x01\x03\x01\x02 in the last one which you see when you scroll right. They do not appear in the first two.

This screenshot makes the encoding difference obvious

Expected Behavior

Edge API Routes should produce the same output for the Buffer as Middleware.

Which browser are you using? (if relevant)

No response

How are you deploying your application? (if relevant)

No response

dferber90 commented 1 year ago

This was identified to be a problem of SWC. It can be reproduced by typing "\x000" into https://swc.rs/playground, which mistakenly produces \00.

github-actions[bot] commented 1 year ago

This closed issue has been automatically locked because it had no new activity for a month. If you are running into a similar issue, please create a new issue with the steps to reproduce. Thank you.

vercel / next.js