Module build failed with MDX files: Unexpected token

Benwick91 commented 4 years ago

Describe the bug My mdx files fail after upgrade to Storybook version 5.3.14. In Storybook 5.3.9 everything works fine. I have no empty lines in my MDX files.

WARNING in ./src/****/Test.mdx
Module build failed (from ./node_modules/react-scripts/node_modules/babel-loader/lib/index.js):
SyntaxError: /src/****/Test.mdx: Unexpected token (13:9)

  11 | const makeShortcode = name => function MDXDefaultShortcode(props) {
  12 |   console.warn("Component " + name + " was not imported, exported, or provided by MDXProvider as global scope")
> 13 |   return <div {...props}/>
     |          ^
  14 | };
  15 |

Expected behavior The building of my mdx files.

Code snippets Here my main.js and my webpack.config.js. I get it only work in 5.3.9 with both files. perhaps here is the problem.

My main.js:

module.exports = {
  stories: ['../src/**/*.stories.mdx', '../src/**/*.stories.js', '../src/**/*.stories.ts', '../src/**/*.stories.tsx'],
  addons: [
    '@storybook/preset-create-react-app',
    '@storybook/addon-actions',
    '@storybook/addon-links',
    '@storybook/addon-storysource',
    '@storybook/addon-knobs/register',
    '@storybook/addon-docs',
  ],
};

My webpack.config.js:

const path = require('path');
const webpack = require("webpack");

module.exports = async ({ config, mode }) => {
    config.module.rules.push({
        test: /\.(ts|tsx)$/,
        include: [path.join(__dirname, '../src'), path.join(__dirname, '../stories')],
        use: [
            {
                loader: require.resolve('awesome-typescript-loader'),
                options: {
                    configFileName: './.storybook/tsconfig.json'
                }
            },
            {
                loader: require.resolve('react-docgen-typescript-loader'),
            },
        ]  
    });

    config.resolve.extensions.push('.ts', '.tsx');

    // Return the altered config
    return config;
};

System: Environment Info:

  System:
    OS: macOS Mojave 10.14.6
    CPU: (8) x64 Intel(R) Core(TM) i5-8279U CPU @ 2.40GHz
  Binaries:
    Node: 12.16.1 - ~/.nvm/versions/node/v12.16.1/bin/node
    Yarn: 1.21.1 - /usr/local/bin/yarn
    npm: 6.14.1 - ~node_modules/.bin/npm
  Browsers:
    Chrome: 80.0.3987.122
    Firefox: 73.0.1
    Safari: 13.0.5
  npmPackages:
    @storybook/addon-actions: ^5.3.9 => 5.3.14 
    @storybook/addon-docs: ^5.3.9 => 5.3.14 
    @storybook/addon-info: ^5.3.9 => 5.3.14 
    @storybook/addon-knobs: ^5.3.9 => 5.3.14 
    @storybook/addon-links: ^5.3.9 => 5.3.14 
    @storybook/addon-storysource: ^5.3.9 => 5.3.14 
    @storybook/addons: ^5.3.9 => 5.3.14 
    @storybook/preset-create-react-app: ^1.5.2 => 1.5.2 
    @storybook/react: ^5.3.9 => 5.3.14

Additional context Thank you!

shilman commented 4 years ago

@Benwick91 can you bisect to figure out exactly which version broke it? From there I can more easily help diagnose exactly what's going wrong.

Benwick91 commented 4 years ago

@shilman Thank for your answer. There happens something weird. If I update from a version, that runs without problems, the update works (5.3.9 til 6.0.0.alpha20). But if I delete the node-modules folder and make a new yarn install, the MDX error occurs. And I can install another version (even 5.3.9) and it doesn't work anymore. One strange thing is that in the yarn.lock there are two node-logger versions (5.3.9 and 5.3.14) after the update

stale[bot] commented 4 years ago

Hi everyone! Seems like there hasn't been much going on in this issue lately. If there are still questions, comments, or bugs, please feel free to continue the discussion. Unfortunately, we don't have time to get to every issue. We are always open to contributions so please send us a pull request if you would like to help. Inactive issues will be closed after 30 days. Thanks!

jgarplind commented 4 years ago

Not sure if this is helpful, but for me the difference between the error above and a working config was this:

use: [
+          {
+            loader: 'babel-loader',
+            // may or may not need this line depending on your app's setup
+            options: {
+              plugins: ['@babel/plugin-transform-react-jsx'],
+            },
+          },
          {
            loader: '@mdx-js/loader',
            options: {
              compilers: [createCompiler({})],
            },
          },
        ],

I can see it doesn't resemble your config at all, but maybe it would help someone knowledgable such as @shilman to understand where the error comes from.

stale[bot] commented 4 years ago

Hi everyone! Seems like there hasn't been much going on in this issue lately. If there are still questions, comments, or bugs, please feel free to continue the discussion. Unfortunately, we don't have time to get to every issue. We are always open to contributions so please send us a pull request if you would like to help. Inactive issues will be closed after 30 days. Thanks!

jacklimwenjie commented 4 years ago

Same thing happened on version 5.3.18, even without webpack configuration.

shilman commented 4 years ago

Have you tried the docs preset with configureJSX: true?

module.exports = {
  stories: ['../src/**/*.stories.(js|mdx)'],
  addons: [
    {
      name: '@storybook/addon-docs',
      options: { configureJSX: true },
    },
  ],
};

stale[bot] commented 4 years ago

Hi everyone! Seems like there hasn't been much going on in this issue lately. If there are still questions, comments, or bugs, please feel free to continue the discussion. Unfortunately, we don't have time to get to every issue. We are always open to contributions so please send us a pull request if you would like to help. Inactive issues will be closed after 30 days. Thanks!

jucian0 commented 4 years ago

I've the same issue

alexnitta commented 4 years ago

I'm seeing the Unexpected token error for MDX files on 5.3.18, even with configureJSX: true. My setup is a bit unconventional, though - I'm using babel-loader to transpile TypeScript, instead of ts-loader.

Here is my main.js config:

const path = require('path');

module.exports = {
    stories: ['../../**/*.story.(ts|tsx|js|jsx|mdx)'],
    addons: [
        '@storybook/addon-actions',
        '@storybook/addon-links',
        '@storybook/preset-create-react-app',
        {
            name: '@storybook/addon-docs',
            options: {
                configureJSX: true,
            },
        },
    ],
    webpackFinal: async config => {
        config.module.rules.push({
            test: /\.(ts|tsx)$/,
            loader: require.resolve('babel-loader'),
            options: {
                presets: [['react-app', { flow: false, typescript: true }]],
            },
        });

        config.resolve.extensions.push('.ts', '.tsx');

        config.module.rules.push({
            test: /\.css$/,
            use: [
                {
                    loader: 'postcss-loader',
                    options: {
                        sourceMap: true,
                        config: {
                            path: './.storybook/',
                        },
                    },
                },
            ],

            include: path.resolve(__dirname, '../..'),
        });

        return config;
    },
};

nevolgograd commented 4 years ago

Getting this error when trying to add second story in one .mdx file:

<Preview>
  <Story name="sequent">
    <Component type="sequent" {...props} />
  </Story>

  <Story name="instant">
    <Component type="instant"  {...props} />
  </Story>
</Preview>

Config is almost the same as @alexnitta has.

shilman commented 4 years ago

@nevolgograd delete the whitespace between <Story> blocks. hopefully this whitespace sensitivity gets fixed in MDX 2.0

nevolgograd commented 4 years ago

@shilman would never have thought of this, thanks!

Yama-Tomo commented 4 years ago

same error occurs when updating @storybook/preset-create-react-app to 3.0.1

update

Still have error in @storybook/preset-create-react-app@3.1.0 Downgrading the @storybook/preset-create-react-app to 3.0.0 will work

Maybe this is related 🤔 ? https://github.com/storybookjs/presets/pull/147

update:2

My problem solved upgrade storybook v6.0.0-beta.37 to v6.0.0-beta.38 relate: https://github.com/storybookjs/storybook/pull/11333 (5ff06d2)

jpmarra commented 4 years ago

Hit this same issue when running through the Controls walkthrough https://gist.github.com/shilman/69c1dd41a466bae137cc88bd2c6ef487#:~:text=Storybook%20Controls%20w%2F%20CRA%20%26%20TypeScript,ergonomic%20way%20to%20write%20stories.

sethburtonhall commented 4 years ago

I am having the same issue when migrating from version 5.3.x to 6.0.x via the migration guide here https://github.com/storybookjs/storybook/issues/9311

daniel-norris commented 4 years ago

I'm getting the same issue after following a basic example on Storybook tutorials.

Tried the following:

removing spaces and line breaks (https://github.com/storybookjs/storybook/issues/9598#issuecomment-578452772)
upgrading storybook to latest rc
downgrading @storybook/preset-create-react-app to 3.0.0.

I'm getting a module build failed and unexpected token error.

main.js

module.exports = {
    stories: ['../src/**/*.stories.(js|mdx)'],
    addons: [
        '@storybook/preset-create-react-app',
        '@storybook/addon-actions',
        '@storybook/addon-links',
        {
            name: '@storybook/addon-docs',
            options: {
                configureJSX: true,
            },
        },
    ],
};

test.stories.mdx

import { Meta, Story } from '@storybook/addon-docs/blocks';
import { Test } from './components/Test';
<Meta title="Components|Test" component={Test} />
# Test
## Displays a test component
test text goes here
<Story name="standard">
<Test>Test</Test>
</Story>

Test.js

import React from 'react';

const Test = ({ children }) => (
    <button>{ children }</button>

)

export default Test;

package.json

...
"devDependencies": {
    "@storybook/addon-actions": "^6.0.0-beta.41",
    "@storybook/preset-create-react-app": "^3.0.0",
    "@storybook/addon-docs": "^6.0.0-beta.41",
    "@storybook/addon-links": "^6.0.0-beta.41",
    "@storybook/addons": "^6.0.0-beta.41",
    "@storybook/react": "^6.0.0-beta.41",
    "babel-loader": "^8.1.0",
    "react": "^16.13.1",
    "react-is": "^16.13.1"
  }
...

shilman commented 4 years ago

@daniel-norris whitespace is important in MDX. extra blank lines can cause problems, per the issue you referenced. but missing blank lines can too. for instance i'm pretty sure you are missing required blank lines around the JSX snippets and headings.

MDX 2.0 will hopefully be released in the next few weeks, in concert with Storybook 6.0, and hopefully that will address some of these issues, or at least give new clarity on the rules.

daniel-norris commented 4 years ago

@daniel-norris whitespace is important in MDX. extra blank lines can cause problems, per the issue you referenced. but missing blank lines can too. for instance i'm pretty sure you are missing required blank lines around the JSX snippets and headings.

MDX 2.0 will hopefully be released in the next few weeks, in concert with Storybook 6.0, and hopefully that will address some of these issues, or at least give new clarity on the rules.

@shilman thanks for the reply. Is there an example repo or boilerplate I can look at for correct mdx usage with storybook?

stale[bot] commented 4 years ago

Hi everyone! Seems like there hasn't been much going on in this issue lately. If there are still questions, comments, or bugs, please feel free to continue the discussion. Unfortunately, we don't have time to get to every issue. We are always open to contributions so please send us a pull request if you would like to help. Inactive issues will be closed after 30 days. Thanks!

shilman commented 4 years ago

Sorry for the slow reply on this. Lots of valid Storybook MDX files here: https://github.com/storybookjs/storybook/tree/next/examples

stale[bot] commented 4 years ago

Hi everyone! Seems like there hasn't been much going on in this issue lately. If there are still questions, comments, or bugs, please feel free to continue the discussion. Unfortunately, we don't have time to get to every issue. We are always open to contributions so please send us a pull request if you would like to help. Inactive issues will be closed after 30 days. Thanks!

stale[bot] commented 4 years ago

Hey there, it's me again! I am going close this issue to help our maintainers focus on the current development roadmap instead. If the issue mentioned is still a concern, please open a new ticket and mention this old one. Cheers and thanks for using Storybook!

coreybruyere commented 3 years ago

This is a long shot, but I'm hoping someone who's more familiar with MDX's intricacies might be able to help me pinpoint my error. I've gone line by line and manually checked by deleting code and checking for output and have only been able to successfully get my MDX story to display when I remove everything but the final return. I can't share my private repo, but as noted hoping someone might be able to pinpoint whats going on..

I tried adding configureJsx: true to storybook addon docs option but that just hangs up my entire Storybook with a blank sidebar and blank page.

Here's my entire warning:

SyntaxError: Unexpected token, expected "," (6:47)
    at Object._raise (/Users/corey.bruyere/Sites/haiku/node_modules/@babel/parser/src/parser/error.js:60:45)
    at Object.raiseWithData (/Users/corey.bruyere/Sites/haiku/node_modules/@babel/parser/src/parser/error.js:55:17)
    at Object.raise (/Users/corey.bruyere/Sites/haiku/node_modules/@babel/parser/src/parser/error.js:39:17)
    at Object.unexpected (/Users/corey.bruyere/Sites/haiku/node_modules/@babel/parser/src/parser/util.js:139:16)
    at Object.expect (/Users/corey.bruyere/Sites/haiku/node_modules/@babel/parser/src/parser/util.js:116:28)
    at Object.parseCallExpressionArguments (/Users/corey.bruyere/Sites/haiku/node_modules/@babel/parser/src/parser/expression.js:864:14)
    at Object.parseCoverCallAndAsyncArrowHead (/Users/corey.bruyere/Sites/haiku/node_modules/@babel/parser/src/parser/expression.js:725:29)
    at Object.parseSubscript (/Users/corey.bruyere/Sites/haiku/node_modules/@babel/parser/src/parser/expression.js:622:19)
    at Object.parseSubscripts (/Users/corey.bruyere/Sites/haiku/node_modules/@babel/parser/src/parser/expression.js:579:19)
    at Object.parseExprSubscripts (/Users/corey.bruyere/Sites/haiku/node_modules/@babel/parser/src/parser/expression.js:560:17)
    at Object.parseUpdate (/Users/corey.bruyere/Sites/haiku/node_modules/@babel/parser/src/parser/expression.js:534:21)
    at Object.parseMaybeUnary (/Users/corey.bruyere/Sites/haiku/node_modules/@babel/parser/src/parser/expression.js:518:17)
    at Object.parseExprOps (/Users/corey.bruyere/Sites/haiku/node_modules/@babel/parser/src/parser/expression.js:318:23)
    at Object.parseMaybeConditional (/Users/corey.bruyere/Sites/haiku/node_modules/@babel/parser/src/parser/expression.js:273:23)
    at Object.parseMaybeAssign (/Users/corey.bruyere/Sites/haiku/node_modules/@babel/parser/src/parser/expression.js:226:21)
    at Object.parseVar (/Users/corey.bruyere/Sites/haiku/node_modules/@babel/parser/src/parser/statement.js:1008:26)
./packages/core/node_modules/tsdx/templates/react-with-storybook/stories/Thing.stories.tsx 18:30
Module parse failed: Unexpected token (18:30)
File was processed with these loaders:
 * ./node_modules/@storybook/source-loader/dist/index.js
You may need an additional loader to handle the result of these loaders.
| // By passing optional props to this story, you can control the props of the component when
| // you consume the story in a test.
> export const Default = (props?: Partial<Props>) => <Thing {...props} />;
| 
| 
./packages/icons/node_modules/tsdx/templates/react-with-storybook/stories/Thing.stories.tsx 18:30
Module parse failed: Unexpected token (18:30)
File was processed with these loaders:
 * ./node_modules/@storybook/source-loader/dist/index.js
You may need an additional loader to handle the result of these loaders.
| // By passing optional props to this story, you can control the props of the component when
| // you consume the story in a test.
> export const Default = (props?: Partial<Props>) => <Thing {...props} />;
|

Along with the code that is causing it from theme.stories.mdx:

import { Meta, Props, Preview, Story, Source, Subtitle } from '@storybook/addon-docs/blocks';

import { Box, Flex, Heading, useTheme } from '../src';

<Meta title="Design/Theme Spec"></Meta>

<Preview withToolbar withSource="open">
  <Story name="Colors">
    {() => {
      const theme = useTheme();
      const { colors } = theme;
      const topLevelColors = Object.keys(colors!).filter((color) => typeof colors![color] === "string");
      const topColorsSection = (
        <>
          <Heading>Aliases</Heading>
          {topLevelColors.map((color, idx) => (
            <Flex mb={1} key={`${color}-${idx}`} {...rest}>
              <Box
                sx={{
                  width: 24,
                  height: 24,
                  bg: colors![color],
                  borderWidth: "1px",
                  borderColor: "gray",
                  borderStyle: "solid"
                }}
              />
              <Box>{colors![color]}</Box>
            </Flex>
          ))}
        </>
      );
      const subColorsSection = Object.keys(colors!).map(
        (label, idx) =>
          typeof colors![label] === "object" && (
            <Box key={`${label}-${idx}`}>
              {label && <Heading>{label}</Heading>}
              {Object.values(colors![label]!).map((subColor, idx) => {
                let hex = subColor;
                return (
                  <Flex key={`${hex}-${idx}`} {...rest}>
                    <Box
                      sx={{
                        mb: 1,
                        width: 24,
                        height: 24,
                        bg: hex,
                        borderWidth: "1px",
                        borderColor: "gray",
                        borderStyle: "solid"
                      }}
                    />
                    <Box>{hex}</Box>
                  </Flex>
                );
              })}
            </Box>
          )
      );
      return (
        <>
          {topColorsSection}
          {subColorsSection}
        </>
      );
    }};
  </Story>
</Preview>

shilman commented 3 years ago

@coreybruyere a couple different thoughts:

we're also passing the MDX through our own compiler, so it's possible that there's a bug there (tho I think it's fairly battle tested by now?)
do you have a custom babel config in your project root? if so, try removing it for debugging purposes

coreybruyere commented 3 years ago

I tried removing my babel config and was still seeing the same issue.

If you get a chance could you possibly take a look at a simplified version of my library? It contains the same config as my private repo. Stories break when the code is present inside of packages/core/stories/0-c-theme-spec.stories.mdx.

Any pointers or help would be greatly appreciated

puzant commented 3 years ago

Hello everyone, I'm also facing the same issue, & I can't figure out what's wrong I'm basically using hooks inside the input.stories.mdx

export const handleChange = (e: React.ChangeEvent<HTMLInputElement>) => {
  const [value, setValue] = useState<string>("")
  setValue(e.target.value)
}

& just rendering it inside a Canvas like so

<Canvas>
  <Story 
    name="Text" 
    args={{
      label: "Search",
      variant: "outlined",
      type: "text",
      color: "primary",
      height: 40,
      placeholder: "placeholder",
      onChange: handleChange,
      value: value
    }}
  >
    {Template.bind({})}
  </Story>
</Canvas>

but if I remove the onChange line & the handleChange function the input shows on storybook,
any idea why this is happening?

Sandstedt commented 2 years ago

Sorry for writing in and old closed thread. Just had this issue, but solved it by replacing:

@storybook/addon-essentials with:

'@storybook/addon-links',
'@storybook/addon-actions',
'@storybook/addon-viewport',
{
  name: '@storybook/addon-docs',
  options: {
    configureJSX: true,
    babelOptions: {},
    sourceLoaderOptions: null,
    transcludeMarkdown: true,
  },
},
'@storybook/addon-controls',
'@storybook/addon-backgrounds',
'@storybook/addon-toolbars',
'@storybook/addon-measure',
'@storybook/addon-outline',

as described in the docs here.

Now when you start the storybook server and open it (usually on http://localhost:6006/), you can now see the full error in the console, not just "Unexpected token" (the full error will not be displayed in your terminal). You can also run build-storybook --debug-webpack to allow webpack to show more error info. In my case there was an old story that used the old way of writing MDX in a regular .md file. That confused the compiler at ton :D.

Other errors was:

There was also some incorrect indentation and to space/line-breaks between some JSX code in another .mdx file that the compiler didn't like either (as stated above).
There was an import React from "react" statement in the top that caused an error.
I had two Stories using a <Story name="Button"> that had the same name.

Another error was that I had comments in a jsx block, like this:

<Box
color="danger"
className="anim-error-reveal"
{/* className="anim-out" */}
/>

When I removed all those. It compiled fine.

Hope this fixed can help someone else too.

storybookjs / storybook

Module build failed with MDX files: Unexpected token #9996

update

update:2