Migrating to V4 - React Query Devtools module parse failed

[TommyNT]

Describe the bug

I have migrated to React Query V4. When I include @tanstack/react-query-devtools it displays error like below:

unitless.browser.esm.js:50 Uncaught Error: Module parse failed: Unexpected token (18:1274)
You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders
|    * @author Kent C. Dodds <me@kentcdodds.com> (https://kentcdodds.com)
|    */
> const o={CASE_SENSITIVE_EQUAL:7,EQUAL:6,STARTS_WITH:5,WORD_STARTS_WITH:4,CONTAINS:3,ACRONYM:2,MATCHES:1,NO_MATCH:0};function s(e,n,t){return e=a(e,t),(n=a(n,t)).length>e.length?o.NO_MATCH:e===n?o.CASE_SENSITIVE_EQUAL:(e=e.toLowerCase())===(n=n.toLowerCase())?o.EQUAL:e.startsWith(n)?o.STARTS_WITH:e.includes(` ${n}`)?o.WORD_STARTS_WITH:e.includes(n)?o.CONTAINS:1===n.length?o.NO_MATCH:function(e){let n="";return e.split(" ").forEach((e=>{e.split("-").forEach((e=>{n+=e.substr(0,1)}))})),n}(e).includes(n)?o.ACRONYM:function(e,n){let t=0,r=0;function s(e,n,r){for(let o=r,s=n.length;o<s;o++){if(n[o]===e)return t+=1,o+1}return-1}function a(e){const r=1/e,s=t/n.length;return o.MATCHES+s*r}const i=s(n[0],e,0);if(i<0)return o.NO_MATCH;r=i;for(let t=1,a=n.length;t<a;t++){r=s(n[t],e,r);if(!(r>-1))return o.NO_MATCH}return a(r-i)}(e,n)}function a(e,t){let{keepDiacritics:o}=t;return e=`${e}`,o||(e=e.replace(r,(e=>n[e]))),e}function i(e,n){let t=n;"object"==typeof n&&(t=n.accessor);const r=t(e);return null==r?[]:Array.isArray(r)?r:[String(r)]}const u={maxRanking:1/0,minRanking:-1/0};function c(e){return"function"==typeof e?u:{...u,...e}}e.compareItems=function(e,n){return e.rank===n.rank?0:e.rank>n.rank?-1:1},e.rankItem=function(e,n,t){if((t=t||{}).threshold=t.threshold??o.MATCHES,!t.accessors){const r=s(e,n,t);return{rankedValue:e,rank:r,accessorIndex:-1,accessorThreshold:t.threshold,passed:r>=t.threshold}}const r=function(e,n){const t=[];for(let r=0,o=n.length;r<o;r++){const o=n[r],s=c(o),a=i(e,o);for(let e=0,n=a.length;e<n;e++)t.push({itemValue:a[e],attributes:s})}return t}(e,t.accessors),a={rankedValue:e,rank:o.NO_MATCH,accessorIndex:-1,accessorThreshold:t.threshold,passed:!1};for(let e=0;e<r.length;e++){const i=r[e];let u=s(i.itemValue,n,t);const{minRanking:c,maxRanking:l,threshold:A=t.threshold}=i.attributes;u<c&&u>=o.MATCHES?u=c:u>l&&(u=l),u=Math.min(u,l),u>=A&&u>a.rank&&(a.rank=u,a.passed=!0,a.accessorIndex=e,a.accessorThreshold=A,a.rankedValue=i.itemValue)}return a},e.rankings=o,Object.defineProperty(e,"__esModule",{value:!0})}));
| //# sourceMappingURL=index.production.js.map
| 
    at ./node_modules/@tanstack/match-sorter-utils/build/umd/index.production.js (unitless.browser.esm.js:50:29)
    at __webpack_require__ (bootstrap:63:1)
    at ./node_modules/@tanstack/react-query-devtools/build/lib/devtools.js (devtools.js:8:24)
    at __webpack_require__ (bootstrap:63:1)
    at ./node_modules/@tanstack/react-query-devtools/build/lib/index.js (index.js:5:16)
    at __webpack_require__ (bootstrap:63:1)
    at ./formEfficiencyReport/src/App.tsx (App.tsx:1:1)
    at __webpack_require__ (bootstrap:63:1)
    at ./formEfficiencyReport/src/index.tsx (index.tsx:1:1)
    at __webpack_require__ (bootstrap:63:1)

Your minimal, reproducible example

https://codesandbox.io/s/dry-lake-cyly62?file=/src/App.js

Steps to reproduce

In package.json:

"@tanstack/react-query": "4.3.3",
    "@tanstack/react-query-devtools": "4.3.3",

In webpack.config.js:

const babelConfig = {
  presets: [
    [
      '@babel/preset-env',
      {
        targets: {
          browsers: [
            '> 1%',
            'last 2 chrome versions',
            'last 2 firefox versions',
            'last 2 safari versions',
            'last 2 opera versions',
            'not ie 11',
            'not op_mini all',
          ],
        },
      },
    ],
    '@babel/preset-react',
    '@babel/preset-typescript',
  ],
  plugins: [
    '@babel/plugin-proposal-optional-chaining',
    '@babel/plugin-transform-runtime',
    'styled-components',
    '@babel/plugin-proposal-class-properties',
  ],
};

rules: [
      {
        test: /\.(ts|tsx|style.ts|style.tsx|js|jsx)$/,
        resolve: {
          extensions: ['.ts', '.tsx', '.style.ts', '.style.tsx', '.js', '.jsx'],
        },
        exclude: [/node_modules/, /tsconfig.json/],
        use: {
          loader: 'babel-loader',
          options: { ...babelConfig },
        },
      },
      {
        test: /\.(mjs)$/,
        include: /node_modules/,
        type: 'javascript/auto',
      }]

Expected behavior

I expect to display React Query Devtools, but it displays error. When I'm not including devtools it works fine.

How often does this bug happen?

Every time

Screenshots or Videos

No response

Platform

• OS: Mac OS 12.3.1
• Node 14.17.0
• NPM 6.14.13

react-query version

4.3.3

TypeScript version

No response

Additional context

No response

[TkDodo]

show a runnable reproduction please. There were many issues already (search the closed issues!) and all were related to bundlers doing something wrong. Please also use the latest version of react-query

[DamianOsipiuk]

@TkDodo I guess it might be connected with the fact that match-sorter-utils are living inside @tanstack/table repo and have previous setup in regards to ems module handling. We removed the browser entry from package.json in query because of this issue that some bundlers were picking umd build instead of esm

[TkDodo]

true - we are not using a fixed version dependency:

'@tanstack/match-sorter-utils': ^8.1.1

so are you saying this is an upstream issue with react-table 😅 ?

[TommyNT]

I've added link to Codesandbox but it works. I think it is problem in my webpack.config.js (see in codesandbox)

[DamianOsipiuk]

@TkDodo I would say yes. I guess we could fix it either by reworking @tanstack/table monorepo esm support 😨, or moving match-sorter-utils out of it.

@TommyNT codesandbox is using webpack 5. Which one do you use in your project?

[TommyNT]

@DamianOsipiuk I use webpack ^4.43.0 (in codesandbox is package.json with my dependencies and versions)

[TkDodo]

I guess we could fix it either by reworking @tanstack/table monorepo esm support

here we go again 😆. Can we reproduce the setup we have in query?

[TommyNT]

When I am using esm-loader in

module: {
    rules: [
      {
        test: /\.(ts|tsx|style.ts|style.tsx|js|jsx)$/,
        resolve: {
          extensions: [".ts", ".tsx", ".style.ts", ".style.tsx", ".js", ".jsx"]
        },
        exclude: [/node_modules/, /tsconfig.json/],
        use: {
          loader: ["babel-loader", "esm-loader"],
          options: { ...babelConfig }
        }
      }

It displays other error but it also relates to /node_modules/@tanstack/match-sorter-utils/build/umd/index.production.js

unitless.browser.esm.js:50 Uncaught Error: Module build failed (from ./node_modules/esm-loader/esm-loader.mjs):
Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: /Users/xxxxxx/node_modules/esm-loader/esm-loader.mjs
    at Module.load (:10090/pl/formlib/formsReports/internal/modules/cjs/loader.js:931:11)
    at Function.Module._load (:10090/pl/formlib/formsReports/internal/modules/cjs/loader.js:774:14)
    at Module.require (:10090/pl/formlib/formsReports/internal/modules/cjs/loader.js:957:19)
    at require (:10090/Users/xxxxxxnode_modules/v8-compile-cache/v8-compile-cache.js:159:20)
    at loadLoader (:10090/Users/xxxxxx/node_modules/loader-runner/lib/loadLoader.js:18:17)
    at iteratePitchingLoaders (:10090/Users/xxxxxx/node_modules/loader-runner/lib/LoaderRunner.js:169:2)
    at runLoaders (:10090/Users/xxxxxx/node_modules/loader-runner/lib/LoaderRunner.js:365:2)
    at NormalModule.doBuild (:10090/Users/xxxxxx/node_modules/webpack/lib/NormalModule.js:295:3)
    at NormalModule.build (:10090/Users/xxxxxx/node_modules/webpack/lib/NormalModule.js:446:15)
    at Compilation.buildModule (:10090/Users/xxxxxx/node_modules/webpack/lib/Compilation.js:739:10)
    at :10090/Users/xxxxxx/node_modules/webpack/lib/Compilation.js:981:14
    at :10090/Users/xxxxxx/node_modules/webpack/lib/NormalModuleFactory.js:409:6
    at :10090/Users/xxxxxx/node_modules/webpack/lib/NormalModuleFactory.js:155:13
    at AsyncSeriesWaterfallHook.eval [as callAsync] (eval at create (:10090/Users/xxxxxx/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:6:1)
    at :10090/Users/xxxxxx/node_modules/webpack/lib/NormalModuleFactory.js:138:29
    at :10090/Users/xxxxxx/node_modules/webpack/lib/NormalModuleFactory.js:346:9
    at processTicksAndRejections (:10090/pl/formlib/formsReports/internal/process/task_queues.js:77:11)
    at ./node_modules/@tanstack/match-sorter-utils/build/umd/index.production.js (unitless.browser.esm.js:50:29)
    at __webpack_require__ (bootstrap:63:1)
    at ./node_modules/@tanstack/react-query-devtools/build/lib/devtools.js (devtools.js:8:24)
    at __webpack_require__ (bootstrap:63:1)
    at ./node_modules/@tanstack/react-query-devtools/build/lib/index.js (index.js:5:16)
    at __webpack_require__ (bootstrap:63:1)
    at ./formEfficiencyReport/src/App.tsx (App.tsx:1:1)
    at __webpack_require__ (bootstrap:63:1)
    at ./formEfficiencyReport/src/index.tsx (index.tsx:1:1)
    at __webpack_require__ (bootstrap:63:1)

[DamianOsipiuk]

I guess we could fix it either by reworking @tanstack/table monorepo esm support

here we go again 😆. Can we reproduce the setup we have in query?

Yes i think we should be able to. Unless there are some quirks that we do not know of. @TanStack/react-table-maintainers Any thoughts on that?

[TommyNT]

The same issue is in #4278

[Bre77]

I am completely new to React Query, trying to add it to my project, and having the same issue as @TommyNT with 4.13.4 and 4.13.5. It seems React Query 3 works just fine with the devtools, and React Query 4.x works fine without it.

[TommyNT]

I am completely new to React Query, trying to add it to my project, and having the same issue as @TommyNT with 4.13.4 and 4.13.5. It seems React Query 3 works just fine with the devtools, and React Query 4.x works fine without it.

Hi, I actually work on React Query v3 and it's fine, but I would like to migrate to v4. It seems that RQ v4 works on Webpack v5

[peterkrieg]

I also had this issue when migrating from v3 => v4. This issue is for webpack 4 environments I think, which use acorn 6, and doesn't support nullish coalescing yet.

I had the exact same error as the original author - if you look at the error message @TommyNT you'll see the place of code referenced: Unexpected token (18:1274) which if you open up node_modules, you can find the exact place of code, which traces to this line in match-sorter-utils: https://github.com/TanStack/table/blob/main/packages/match-sorter-utils/src/index.ts#L85

  options.threshold = options.threshold ?? rankings.MATCHES

If I just replace the ?? with || locally in my node_modules, I was able to see the problem resolved, just to test. It's worth trying that to confirm there's nothing else going on.

As far as resolution, I was able to fix by forcing a resolution for acorn 7, which does support nullish coalescing. See the webpack issue with someone mentioning that fix: https://github.com/webpack/webpack/issues/10227. In your package.json, if your config supports resolutions, you can do:

  "resolutions": {
      "acorn": "npm:acorn-with-stage3"
  }

EDIT ^ don't actually recommend this, it led to other issues in my setup/ other people as well.

Instead, you could try running the match-sorter-utils library through babel with some config like:

  test: /\.js$/,
  include: /node_modules\/@tanstack\/match-sorter-utils/,
  use: ['babel-loader'],

And make sure to use the @babel/plugin-proposal-nullish-coalescing-operator babel plugin in your config

[peterkrieg]

Just noting, I actually had other issues related to upgrading acorn (which I'm pretty confused about) so I ended up just targeting match-sorter-utils to be run through our webpack babel-loader.

Something like

  test: /\.js$/,
  include: /node_modules\/@tanstack\/match-sorter-utils/,
  use: ['babel-loader'],

And then make sure to include @babel/plugin-proposal-nullish-coalescing-operator in your babel config

[globalmatt]

As far as resolution, I was able to fix by forcing a resolution for acorn 7, which does support nullish coalescing. See the webpack issue with someone mentioning that fix: webpack/webpack#10227. In your package.json, if your config supports resolutions, you can do:

  "resolutions": {
      "acorn": "npm:acorn-with-stage3"
  }

Otherwise, you could probably apply @babel/plugin-proposal-nullish-coalescing-operator or something similar to process the ?? of this file.

@peterkrieg Thanks for this! I ran into this same issue when trying to install and use downshift. Your solution worked a treat :)

Just to add that, after changing my config, I needed to delete my node_modules dir and then yarn install for it to pick up the change.

[globalmatt]

Hmm, might have spoken too soon - that resolutions fix seems to have broken my React Styleguidist install:

main.bundle.js:19505 Uncaught TypeError: plugins[i] is not a function
    at Function.extend (main.bundle.js:19505:62)
    at ./node_modules/acorn-stage3/index.js.module.exports (main.bundle.js:25591:17)
    at ./node_modules/acorn/index.js (main.bundle.js:25723:16)
    at __webpack_require__ (main.bundle.js:790:30)
    at fn (main.bundle.js:101:20)
    at ./node_modules/buble/dist/buble-browser.es.js (main.bundle.js:29785:63)
    at __webpack_require__ (main.bundle.js:790:30)
    at fn (main.bundle.js:101:20)
    at ./node_modules/react-styleguidist/lib/client/utils/compileCode.js (main.bundle.js:154943:63)
    at __webpack_require__ (main.bundle.js:790:30)

Maybe I'll try upgrading to Webpack 5 and hope that fixes the issue.

Edit: Using this instead seems to have got React Styleguidist working too:

    "resolutions": {
        "acorn": "8.0.1"
    },

[TommyNT]

Just noting, I actually had other issues related to upgrading acorn (which I'm pretty confused about) so I ended up just targeting match-sorter-utils to be run through our webpack babel-loader.

Something like

  test: /\.js$/,
  include: /node_modules\/@tanstack\/match-sorter-utils/,
  use: ['babel-loader'],

And then make sure to include @babel/plugin-proposal-nullish-coalescing-operator in your babel config

@peterkrieg thank you! This solution works

[peterkrieg]

Hey @globalmatt yeah I think that original solution was flawed, perhaps webpack 4 isn't compatible with updated acorn after all. I edited my original comment to instead recommend parsing the match-sorter-utils through babel to convert the problematic ??. Does that work for you?

I do wonder if there's a bit of a mismatch in js target version here - the rest of react-query code gets converted to older version of JS features (aka, nulllish coalescing will get converted out for example). I feel like to keep consistent with rest of library the match-sorter-utils should match the same transpile target as tanstack/query - what do you think @TkDodo ?

I tried looking through the rollup config but couldn't find exactly where that config was being set in either @tanstack/query or @tanstack/table (where the match-sorter-utils package lives)

[TkDodo]

match-sorter-utils is referenced here, like all other dependencies:

https://github.com/TanStack/query/blob/667efe37a3c2fcf82aa37fb44aa046687be04501/rollup.config.ts#L117-L129

Again, this is an upstream issue. match-sorter-utils doesn't define exports in package.json, and webpack4 also has an issue to pick up the "wrong stuff", so this kind of confuses bundlers.

I don't see much action going on over in the table repo, so the 3 possible solutions are to me:

1) make a PR to match-sorter-utils that fixes esm exports 2) copy the code of match-sorter-utils over to the query devtools and we treat it like any other source code that we own 3) set the match-sorter-utils dependency fixed to 8.1.1, _not: ^8.1.1 - because that version was still working.

Thoughts @DamianOsipiuk ?

[DamianOsipiuk]

I found a couple of issues opened in the table repo regarding bundlers picking up UMD bundle and failing to parse it.

make a PR to match-sorter-utils that fixes esm exports

I think we would only need to remove browser field from package.json and it should solve the problem for bundlers. If we do not add exports field, newer bundlers should fall back to module or main

copy the code of match-sorter-utils over to the query devtools and we treat it like any other source code that we own

It needs to be deployed as a separate package, cause both reac-query-devtools and vue-query use it. Not mentioning table (and possibly router?). Maybe in this case it would be better to extract it from table monorepo and make it as a separate standalone repository?

set the match-sorter-utils dependency fixed to 8.1.1, _not: ^8.1.1 - because that version was still working.

I do not see any changes related to this after 8.1.1. Are you sure this will help? If it works, this could be a temporary fix until the problem will be fixed in table.

[TkDodo]

I think we would only need to remove browser field from package.json and it should solve the problem for bundlers.

maybe, but then they wouldn't support ESM. I think the proper fix would be to add exports, no ?

I do not see any changes related to this after 8.1.1. Are you sure this will help? If it works, this could be a temporary fix until the problem will be fixed in table.

8.5.14 is the latest release, and things definitely worked some time ago. There is no other release between 8.5.14. and 8.1.1 so my guess is that is the culprit.

[TkDodo]

@TommyNT @peterkrieg @globalmatt @Bre77 can you please try it out with the preview build from the attached PR to see if it fixes the issue?

"@tanstack/react-query": "https://pkg.csb.dev/TanStack/query/commit/e8fc0458/@tanstack/react-query",
"@tanstack/react-query-devtools": "https://pkg.csb.dev/TanStack/query/commit/e8fc0458/@tanstack/react-query-devtools",

thanks

[peterkrieg]

@TkDodo that fixed it for me, thank you!