search

gatsbyjs / gatsby

The best React-based framework with performance, scalability and security built in.
https://www.gatsbyjs.com
MIT License
55.29k stars 10.31k forks source link

[gatsby-transformer-remark] query on markdownRemark no longer returns added fields #13814

Closed neonguru closed 5 years ago

neonguru commented 5 years ago

Description

When doing a query on markdownRemark, fields added using createNodeFields are not present. This occurs only after running npm audit fix, updating the gatsby package from 2.0.91 to 2.3.36 - gatsby-transformer-remark was unchanged from 2.2.0. With the previous version of gatsby, it worked fine.

Code is adding the fields in the setFieldsOnGraphQLNodeType method with createNodeField. console.log the nodes after calling createNodeField shows the fields correctly placed.

I checked if the fields are present on the allMarkdownRemark query in createPages and they were not.

They were also not present in /src/templates/post.jsx, in the following query:

export const pageQuery = graphql`
  query BlogPostBySlug($slug: String!) {
    markdownRemark(fields: { slug: { eq: $slug } }) {
      html
      timeToRead
      excerpt
      frontmatter {
        title
        cover
        date
        category
        tags
      }
      fields {
        nextTitle
        nextSlug
        prevTitle
        prevSlug
        slug
        date
      }
    }
  }
`;

Steps to reproduce

gatsby new blog https://github.com/Vagr9K/gatsby-material-starter cd blog npm install npm audit fix (works fine if you leave this out) npm run build

Expected result

Should build as before npm audit fix

Actual result

error GraphQL Error Encountered 1 error(s):

It is looking for nextTitle in the query above. nextTitle field was added at line 27 of gatsby-node.js in the setFieldsOnGraphQLNodeType method with createNodeField.

Environment

Run gatsby info --clipboard in your project directory and paste the output here.

$ gatsby info --clipboard

System: OS: Windows 10 CPU: (12) x64 Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz Binaries: Yarn: 1.12.3 - D:\Program Files (x86)\Yarn\bin\yarn.CMD npm: 6.5.0 - D:\Program Files\nodejs\npm.CMD Languages: Python: 2.7.15 Browsers: Edge: 42.17134.1.0 npmPackages: gatsby: ^2.3.36 => 2.3.36 gatsby-image: 2.0.26 => 2.0.26 gatsby-plugin-catch-links: 2.0.9 => 2.0.9 gatsby-plugin-feed: 2.0.11 => 2.0.11 gatsby-plugin-google-analytics: 2.0.9 => 2.0.9 gatsby-plugin-lodash: 3.0.3 => 3.0.3 gatsby-plugin-manifest: 2.0.13 => 2.0.13 gatsby-plugin-netlify-cms: 3.0.9 => 3.0.9 gatsby-plugin-nprogress: 2.0.7 => 2.0.7 gatsby-plugin-offline: 2.0.21 => 2.0.21 gatsby-plugin-react-helmet: 3.0.5 => 3.0.5 gatsby-plugin-sass: 2.0.7 => 2.0.7 gatsby-plugin-sharp: ^2.0.35 => 2.0.35 gatsby-plugin-sitemap: 2.0.4 => 2.0.4 gatsby-plugin-twitter: 2.0.8 => 2.0.8 gatsby-remark-autolink-headers: 2.0.12 => 2.0.12 gatsby-remark-copy-linked-files: 2.0.8 => 2.0.8 gatsby-remark-images: 3.0.1 => 3.0.1 gatsby-remark-prismjs: 3.2.0 => 3.2.0 gatsby-remark-relative-images: ^0.2.2 => 0.2.2 gatsby-remark-responsive-iframe: 2.0.8 => 2.0.8 gatsby-source-filesystem: ^2.0.33 => 2.0.33 gatsby-transformer-remark: 2.2.0 => 2.2.0 gatsby-transformer-sharp: 2.1.10 => 2.1.10

error The system cannot find the path specified.

Error: The system cannot find the path specified.

error UNHANDLED REJECTION

Error: The system cannot find the path specified.

jonniebigodes commented 5 years ago

@neonguru a lot has changed since that version of gatsby and the present one. And it looks like some work needs to be done with that starter. If you don't mind the wait i'll take a look at this and try to come up with a alternative or solution for your issue, do you mind waiting a bit?

neonguru commented 5 years ago

I am not in a rush, just interested in how adding fields is done now.

sidharthachatterjee commented 5 years ago

@neonguru This shouldn't happen since my minor versions don't intend to have any breaking changes. Can you please link to a minimal reproduction of this?

Edit: You already linked to one and I missed it 🤦‍♂

neonguru commented 5 years ago

Yes, it is easy to reproduce. 🤦‍♂

jonniebigodes commented 5 years ago

@neonguru i'm currently finishing up a reproduction for you. I'll post back soon...Just doing some tweaks. Mind waiting just a little longer?

neonguru commented 5 years ago

That's fine. Thanks for checking on this.

jonniebigodes commented 5 years ago

@neonguru sorry for the wait, when i was about to write this comment i was called away to take care of some errands.

Below is detailed the steps i took to make this work:

exports.createPages = ({ graphql, actions }) => { const { createPage } = actions; const postPage = path.resolve("src/templates/post.jsx"); const tagPage = path.resolve("src/templates/tag.jsx"); const categoryPage = path.resolve("src/templates/category.jsx"); return graphql( { allMarkdownRemark(sort: { fields: [frontmatter___date], order: DESC }) { edges { node { fields { slug } frontmatter { title tags category } } } } } ).then(result => { if (result.errors){ console.log(result.errors); throw result.errors } const tagSet = new Set(); const categorySet = new Set();

const posts= result.data.allMarkdownRemark.edges;

posts.forEach((edge,index) => {

  if (edge.node.frontmatter.tags) {
    edge.node.frontmatter.tags.forEach(tag => {
      tagSet.add(tag);
    });
  }

  if (edge.node.frontmatter.category) {
    categorySet.add(edge.node.frontmatter.category);
  }

  const previous = index === posts.length - 1 ? posts[ index - 1].node : posts[index + 1].node
  const next = index === 0 ? posts[posts.length - 1].node : posts[index - 1].node

  createPage({
    path: edge.node.fields.slug,
    component: postPage,
    context: {
      slug: edge.node.fields.slug,
      nexttitle:next.frontmatter.title,
      nextslug:next.fields.slug,
      prevtitle:previous.frontmatter.title,
      prevslug:previous.fields.slug
    }
  });
});

tagSet.forEach(item=>{
  createPage({
    path: `/tags/${_.kebabCase(item)}/`,
    component: tagPage,
    context: {
      tag: item
    }
  });
})
categorySet.forEach(item=>{
  createPage({
    path: `/categories/${_.kebabCase(item)}/`,
    component: categoryPage,
    context: {
      category:item
    }
  });
})

});

};

Key thing to take from this, pagination that was done through graphql field, is now made possible through Gatsby's special prop `context`. It's a trade off, but one of little consequence and which yelds the exact same result.

- With the adjustment done here, some changes had to be made to the template, namely `./src/templates/post.jsx` and one extra component, that being `./src/components/PostSuggestions/index.jsx`.

Starting with the template `./src/templates/post.jsx`, i tried to maintain the strucure as much as possible to prevent confusion, also the code is commented out, so that when you're reading it know what is happening at every step, leaving out the imports to keep the code shorter.
```jsx
export default class PostTemplate extends React.Component {

  constructor(props) {
    super(props);
    this.state = {
      mobile: true
    };
  }
  /**
    * it's best apply the check on both lifecycle methods or some errors
    * will pop up, as all the pages/routes are generated in node in gatsby, so the access to those apis don't exist like window for instance
    */
  componentDidMount(){

    if (typeof window!=='undefined'){
      this.handleResize();
      window.addEventListener("resize", this.handleResize);
    }
  }
  componentWillUnmount(){
    if (typeof window!=='undefined'){
      window.removeEventListener('resize',this.handleResize)
    }
  }

  handleResize=()=>{
    if (window.innerWidth >= 640) {
      this.setState({ mobile: false });
    } else {
      this.setState({ mobile: true });
    }
  }

  render(){
    // the variable extracted through destructuring from the state
    const {mobile}= this.state
    const expanded = !mobile;
    //
    const postOverlapClass = mobile ? "post-overlap-mobile" : "post-overlap";
    /**
     * all of the data passed in gatsby.node inside context will be retrieved through the prop pageContext
     * then retrived what is needed
     *  location is a internal Gatby routing prop
     */
    const {pageContext,location}= this.props
    const {nexttitle,nextslug,prevtitle,prevslug,slug}=pageContext
    //

    // data retrieved from query
    const postNode = this.props.data.markdownRemark;
    const post = postNode.frontmatter;
    if (!post.id) {
      post.id = slug;
    }
    if (!post.category_id) {
      post.category_id = config.postDefaultCategoryID;
    }
    //
    const coverHeight = mobile ? 180 : 350;
    return (
      <Layout location={location}>
        <div className="post-page md-grid md-grid--no-spacing">
          <Helmet>
            <title>{`${post.title} | ${config.siteTitle}`}</title>
            <link rel="canonical" href={`${config.siteUrl}${post.id}`} />
          </Helmet>
          <SEO postPath={slug} postNode={postNode} postSEO />
          <PostCover
            postNode={postNode}
            coverHeight={coverHeight}
            coverClassName="md-grid md-cell--9 post-cover"
          />
          <div
            className={`md-grid md-cell--9 post-page-contents mobile-fix ${postOverlapClass}`}
          >
            <Card className="md-grid md-cell md-cell--12 post">
              <CardText className="post-body">
                <h1 className="md-display-2 post-header">{post.title}</h1>
                <PostInfo postNode={postNode} />
                <div dangerouslySetInnerHTML={{ __html: postNode.html }} />
              </CardText>
              <div className="post-meta">
                <PostTags tags={post.tags} />
                <SocialLinks
                  postPath={slug}
                  postNode={postNode}
                  mobile={this.state.mobile}
                />
              </div>
            </Card>
            <UserInfo
              className="md-grid md-cell md-cell--12"
              config={config}
              expanded={expanded}
            />
            <Disqus postNode={postNode} expanded={expanded} />
          </div>
          {/* old component <PostSuggestions postNode={postNode} /> */}
          <PostSuggestions postNode={{next:{title:nexttitle,slug:nextslug},previous:{title:prevtitle,slug:prevslug}}} />
        </div>
      </Layout>
    );
  }

}

export const pageQuery = graphql`
  query BlogPostBySlug($slug: String!) {
    markdownRemark(fields: { slug: { eq: $slug } }) {
      html
      timeToRead
      excerpt
      frontmatter {
        title
        cover
        date
        category
        tags
      }
      fields {
        slug
        date
      }
    }
  }
`;

neon2

Feel free to provide feedback, so that we can close this issue, or continue to work on it till we find a solution. Also if you need i'm more than happy to push the code to repository so that you can take your time looking at it. And before i forget, sorry for the extra long comment.

neonguru commented 5 years ago

@jonniebigodes thanks so much, that works awesome! Using the context to pass the data is the key I was missing. Tried using the nodes passed into setFieldsOnGraphQLNodeType but it still wouldn't work, so I agree that the nodes modified are different in the newer gatsby releases. Excellent solution, thanks again!

jonniebigodes commented 5 years ago

@neonguru no need to thank, glad i was able to help out with a solution to your issue.

Vagr9K commented 5 years ago

Hey everyone! Starter's author here.

Just wanted to say thank you to everyone involved in reporting/resolving this issue.

Back in Aug. 2017 (pre Gatsby 1.0), when the code was written, there were a couple of issues:

Hence, I'd also like to thank @jonniebigodes for motivating me to finally refactor that ancient piece of code!

jonniebigodes commented 5 years ago

@Vagr9K no need to thank, i'm glad that i was able to help out in solving this issue.

alxiong commented 4 years ago

@jonniebigodes thank you for your excellent, detailed solution.

Do you mind briefly explain to me what onCreateNode is doing?

Also what do I need to change if I want those posts to be under the path of /blog/${slug} instead of directly /${slug}? Thanks

jonniebigodes commented 4 years ago

@alxiong first and foremost, no need to thank, glad that it's helping you aswell.

This particular case the onCreateNode hook is expanding any given node that is of type MarkdownRemark, meaning a .md file. It checks the file location and based on that it sets the value for the slug, which aftewards it expands the said node by adding a couple of fields, one will be what's inside the slug variable after all the sanity checks are done. And also one extra field with the date formatted as a ISO string.

Regarding your question:

Also what do I need to change if I want those posts to be under the path of /blog/${slug} instead of directly /${slug}?

From the top of my head you should be ok with adding the path you're mentioning without any issues. Just in case you run into problems, make a small reproduction following these steps and i would gladly take a look at it. Sounds good.

alxiong commented 4 years ago

@jonniebigodes appreciate the clarification. I got it working now (putting content/posts/ under /post/${node.fields.slug}:

if (
      Object.prototype.hasOwnProperty.call(node, 'frontmatter') &&
      Object.prototype.hasOwnProperty.call(node.frontmatter, 'title') &&
      parsedFilePath.dir === 'pages'
    ) {
      slug = `/${kebabCase(node.frontmatter.title)}/`
    } else if (parsedFilePath.dir === 'posts') {
      slug = `/blog/${node.frontmatter.slug}`
    } // ...
//...
if (Object.prototype.hasOwnProperty.call(node, 'frontmatter')) {
      // don't overwrite slug anymore
      if (Object.prototype.hasOwnProperty.call(node.frontmatter, 'date')) {
        const date = new Date(node.frontmatter.date)

        createNodeField({
          node,
          name: 'date',
          value: date.toISOString(),
        })
      }
    }
createNodeField({ node, name: 'slug', value: slug })

cheers 🥂

jonniebigodes commented 4 years ago

@alxiong no problem, glad that you managed to get it working on your end.

Stay safe.