TypeError: Expected input type is Uint8Array (got object)

[zygopleural]

Been trying to upgrade to cuid2 from cuid on-off for a while now but never been able to solve this issue. Hoping maintainers or community can help.

I installed with the following:

yarn remove cuid

yarn add @paralleldrive/cuid2

And my diffs are essentially

- import cuid from "cuid"
+ import cuid2 from "@paralleldrive/cuid2"

- cuid()
+ cuid2.createId()

But I'm always seeing the error:

TypeError: Expected input type is Uint8Array (got object)

> 1 | import cuid2 from "@paralleldrive/cuid2"

Same with:

TypeError: Expected input type is Uint8Array (got object)

> 1 | import { createId } from "@paralleldrive/cuid2"

Only seems to be a problem on my web package, my api is not complaining.

[ericelliott]

Can you tell me more about your environment?

[miguel-osuna]

Hi @ericelliott, I encountered the same issue while running some test cases with Jestand @paralleldrive/cuid2. It seems that Jest adds a large global object which is not properly converted into a Buffer object, but instead, it is converted into a string with all the object keys. I suspect this happens in the createFingerprint function.

Ideally, sha3 should receive a Uint8Array as an argument, but in this case, it is receiving a string. To provide more context, I've included some logs from the callstack at createFingerprint:

I hope this clarifies the issue. Thank you! 😄

[kisaiev]

@miguel-osuna

It seems that Jest adds a large global object which is not properly converted into a Buffer object, but instead, it is converted into a string with all the object keys

String of object keys is exactly what was requested in createFingerprint() method:

const globals = Object.keys(globalObj).toString();

According to the screenshot you shared, it's a string of keys from window object.

Ideally, sha3 should receive a Uint8Array as an argument, but in this case, it is receiving a string.

The error itself is coming from @noble/hashes/src/utils.ts:98.

export type Input = Uint8Array | string;
export function toBytes(data: Input): Uint8Array {
  if (typeof data === 'string') data = utf8ToBytes(data);
  if (!(data instanceof Uint8Array))
    throw new TypeError(`Expected input type is Uint8Array (got ${typeof data})`);
  return data;
}

As you can see Input type allows both Uint8Array and string. You pass it a string of object keys so it rewrites data = utf8ToBytes(data) first and then it fails, because whatever is returned from utf8ToBytes() function is not a Uint8Array instance.

I would suggest to dig into this utf8ToBytes() method:

export function utf8ToBytes(str: string): Uint8Array {
  if (typeof str !== 'string') {
    throw new TypeError(`utf8ToBytes expected string, got ${typeof str}`);
  }
  return new TextEncoder().encode(str);
}

I wasn't able to replicate this problem on my local, running this logic within the Jest test suite, but the bottom line seems to be that there is no problems with @paralleldrive/cuid2. The issue is coming from util.TextEncoder.encode([input]) which should return instance of Uint8Array, but doesn't seem to be working as expected for your test case. It could be due to TypeScript or Jest manipulations.

[ericelliott]

Hi everybody.

In order to move forward with this, I'd need a minimal demonstration that reliably reproduces the problem. Can somebody put one together in CodeSandbox or similar or a GitHub repo we can clone, and share the link?

[ericelliott]

If we can validate the bug, we may be able to wrap a try/catch around it, and if it fails, maybe we could just count the globals instead of read their names, and pad with random entropy to make up for the lost entropy?

[AlaricWhitney]

I'm running into the exact same issue. I'm using React/Redux with Jest.

It seems simple enough to reproduce, as I have a file that uses the createId() method, but Jest fails when trying to import createId()

[AlaricWhitney]

I've created a quick repo to show how easy it is to reproduce. https://github.com/AlaricWhitney/cuid2-issue. just run npm run test and the issue will show up.

[ericelliott]

Any ideas on root cause? What is jest doing that makes the list of global names fail?

[ericelliott]

Can anybody try filtering out keys that are not typeof === 'string' before trying to encode? Maybe Jest is inserting weird key types into the global environment (IMO, that would be a bug in Jest, but one we could work around without much fuss)?

[kisaiev]

JSDOM environment in Jest

This seems to be related to the use of jsdom as it builds global object which doesn't match the one provided by Node.js. There is a known issue in Jest when jsdom environment is used and results of new TextEncoder().encode() and new Uint8Array() are different, refer to https://github.com/jestjs/jest/issues/9983.

The easy fix for the example provided by @AlaricWhitney would be to use custom environment which overwrites Uint8Array provided by jsdom:

1. Install jest-environment-jsdom. Make sure to use the same version as your jest (in cuid2-issue example it was v27), ref. https://stackoverflow.com/a/72124554

   ❯ npm i jest-environment-jsdom@27

2. Create jsdom-env.js file in the root

   const JSDOMEnvironmentBase = require('jest-environment-jsdom');
   
   Object.defineProperty(exports, '__esModule', {
     value: true
   });
   
   class JSDOMEnvironment extends JSDOMEnvironmentBase {
     constructor(...args) {
         const { global } = super(...args);
   
         global.Uint8Array = Uint8Array;
     }
   }
   
   exports.default = JSDOMEnvironment;
   exports.TestEnvironment = JSDOMEnvironment;

3. Update scripts to use the custom environment:

   {
     // ...
     "scripts": {
         // ...
         "test": "react-scripts test --env=./jsdom-env.js",
         // ...
     },
   }

   P.S. Change will make tests in the example fail, since expectations were not updated to match results from createId.

JSDOM lacks features

Also it is known that jsdom doesn't support TextEncoder and TextDecoder, refer https://github.com/jsdom/jsdom/issues/2524. The workaround would be similar, see this comment.

---

The bottom line is that when Unit Tests are executed via Jest, features like Uint8Array/TextEncoder/TextDecoder may be available in jsdom environment but will produce results different from those expected in Node.js or browser. It might be resolved by jsdom at some point, but there is no clear ETA on the issue. I personally don't think it's on cuid2 to adjust and use something else, but owner of the package may fill differently 🙃

[ericelliott]

@kisaiev Thank you for the root cause analysis. It sounds like there would be no trivial fix to get this working with Jest. This is a bug in Jest, not a bug in Cuid, and the bug in Jest is caused by reliance on the outdated JSDOM implementation.

Note: This would not just break Cuid2 - it also breaks other things that would rely on new, unsupported DOM APIs.

Yet another reason to use Riteway instead of Jest.

Could we document a workaround for Jest users?

e.g., before loading Jest, __backupUint8Array__ = Uint8Array Then in your jest code, before calling cuid2: global.Uint8Array = __backupUint8Array__;. Does that work?

[ericelliott]

Can anybody confirm that this solution works?

If not, does this solution work? I think we should add a troubleshooting section to the doc on how to get this working with Jest.

[AlaricWhitney]

The bottom line is that when Unit Tests are executed via Jest, features like Uint8Array/TextEncoder/TextDecoder may be available in jsdom environment but will produce results different from those expected in Node.js or browser. It might be resolved by jsdom at some point, but there is no clear ETA on the issue. I personally don't think it's on cuid2 to adjust and use something else, but owner of the package may fill differently 🙃

I have validated that the solution that @kisaiev pointed out does resolve the TypeError: Expected input type is Uint8Array (got object) issue.

e.g., before loading Jest, backupUint8Array = Uint8Array Then in your jest code, before calling cuid2: global.Uint8Array = backupUint8Array;. Does that work?

I have validated that this does not work.

https://github.com/jsdom/jsdom/issues/2524#issuecomment-1480930523 also does not work.

[ericelliott]

Since it doesn't look like this is something we can easily fix with a code change, I have updated the Readme Troubleshooting section.

[coryasilva]

@AlaricWhitney, @ericelliott I have validated that this works, hopefully it works for you too.

jest.config.mjs

const config = {
  ...
  // testEnvironment: 'jest-environment-jsdom',
  testEnvironment: './jest-environment-jsdom.js', // had to extend; see https://github.com/jsdom/jsdom/issues/2524
}

jest-environment-jsdom.js

const { TextEncoder, TextDecoder } = require('util')
const { default: $JSDOMEnvironment, TestEnvironment } = require('jest-environment-jsdom')

Object.defineProperty(exports, '__esModule', {
  value: true,
})

class JSDOMEnvironment extends $JSDOMEnvironment {
  constructor(...args) {
    const { global } = super(...args)
    global.TextEncoder = TextEncoder
    global.TextDecoder = TextDecoder
    global.Uint8Array = Uint8Array
  }
}

exports.default = JSDOMEnvironment
exports.TestEnvironment = TestEnvironment === $JSDOMEnvironment ? JSDOMEnvironment : TestEnvironment

This was a frustrating issue...

[ericelliott]

@coryasilva Did the instructions from the readme not work for you?

[coryasilva]

@ericelliott No not exactly, the instructions from the readme were missing the TextEnconder and TextDecoder

[VladimirMikulic]

Here's the ESM version of @coryasilva's snippet for packages of type module/ESM.

import { TextEncoder, TextDecoder } from 'util';
import jestEnvironmentJsDom from 'jest-environment-jsdom';

// jest-environment-jsdom attaches properties to exports object so it needs to be imported like this
const { default: $JSDOMEnvironment, TestEnvironment: JSDOMTestEnvironment } = jestEnvironmentJsDom;

class JSDOMEnvironment extends $JSDOMEnvironment {
  constructor(...args) {
    const { global } = super(...args);
    global.Uint8Array = Uint8Array;
    if (!global.TextEncoder) global.TextEncoder = TextEncoder;
    if (!global.TextDecoder) global.TextDecoder = TextDecoder;
  }
}

export const TestEnvironment = JSDOMTestEnvironment === $JSDOMEnvironment ? JSDOMEnvironment : JSDOMTestEnvironment;

export default JSDOMEnvironment;

[ericelliott]

please feel free to open a PR