browsers support?

[cainlevy]

A promise based streaming tokenizer for Node.js and browsers.

strtok3 says that it supports browsers but depends on core node modules. This makes it incompatible without polyfills which have become quite a struggle in create-react-app v5.

Any advice? How is this intended to work?

[Borewit]

It's a bit of a challenge to use this kind of modules in a browser environment. There are 2 types of modules available:

• CommonJS (CJS) version 6
• ECMAScript Module (ESM) version 7

With CJS you need to includes strtok3/lib/core, example of strtok3 running in the browser.

With ESM it should automatically switch to core is browser context, but you can also import strttok3/core.

I think you need to polyfill:

1. buffer
2. readable-stream

Buffer has the required functions to parse binary files and do some text decoding functions. There is nothing like that in the browser, so you simply need this. The internal logic is still standardized in Node.js streams, in the future this may be replaces by web streams (#542) as both worlds slowly start to support that.

[Borewit]

Maybe ESM has room for improvement, it's fairly new. Ideally it automatically depends on the pollyfill libraries, so you don't have to pollyfill them via config files.

[Borewit]

PR, aiming to get rid of the Buffer dependency: #1074