gatsbyjs / gatsby

The best React-based framework with performance, scalability and security built in.
https://www.gatsbyjs.com
MIT License
55.28k stars 10.31k forks source link

Graphql queries in MDX #21934

Closed jnsrikanth closed 4 years ago

jnsrikanth commented 4 years ago

Description

Graphql queries not working in gatsbyjs mdx blog files.

Steps to reproduce

Clear steps describing how to reproduce the issue. Please please please link to a demo project if possible, this makes your issue much easier to diagnose (seriously).

How to Make a Minimal Reproduction: https://www.gatsbyjs.org/contributing/how-to-make-a-reproducible-test-case/

I plan to make graphql queries work from any .mdx file within my main gatsbyjs site folder "site-folder" like /src/pages or content/blog or content/services etc.

Below is an example .mdx file which has a query in it and this MDX file (index.mdx) is under "site-folder/content/services"

Below is the index.mdx file with a graphql query:

---
# This is the frontmatter which goes at the top of the MDX file
# Hashes in the frontmatter are comments
title: Blockchain Development 
author: Srikanth Jallapuram
featuredImage: ./blockchain.png
---

import { graphql } from "gatsby";
import Image from "gatsby-image";
import styles from '../../blog/ai-chatbot/another.module.css';
import ImageHolder from '../../../src/components/header-image.js';
import '../../../src/components/about/card.css';
import './blockchain.css';

export const query = graphql`
  query {
    astronaut: file(relativePath: { regex: "/gatsby-astronaut/i" }) {
      childImageSharp {
        fluid(maxWidth: 600) {
          ...GatsbyImageSharpFluid_tracedSVG
        }
      }
    }
  }
`;

Blockchain is amongst the cutting edge technology that has the potential to revolutionize aspects of our daily lives. Blockchain has had a great impact on the financial industry and is now moving towards impacting other industries as well. Blockchain technology, or simply Blockchain,  is the documentation system that supports cryptocurrency technology. It is essentially a decentralized and distributed record-keeping system that keeps track of all electronic bitcoin transactions.

## Why Blockchain Technology?

Its use is so widespread that, as of September 2018, there have been over 28 million blockchain wallet users worldwide. Since its conception, around 41 million people are using Blockchain wallets and have tracked over 200 billion USD worth in financial transactions. There is also an increasing number of countries exploring the possibility of adopting cryptocurrency, with over 140 countries reached by blockchain technology.

## Wide-range of Application

I have the following gatsby-node.js and gatsby-config.js files and have setup my site to process MDX files from multiple locations.

In this case, I have MDX files located in multiple locations such as "site-folder/content/blog" and "site-folder/content/services" and "site-folder/src/pages"

Here is the gatsby-config.js

module.exports = {
  siteMetadata: {

    title: `Technovature Software`,
    author: `Srikanth Jallapuram`,
    description: `High Technology Digital and Software Development company in Mobile, cloud, Big Data, Internet of Things, IoT, Data Sciences, Artificial Intelligence and Software Automation`,
    siteUrl: `http://www.technovature.com`,
    social: {
      twitter: `technovature`,
      fbAppId: ' ',
    },
  },
  plugins: [
    `gatsby-transformer-json`,
    `gatsby-transformer-sharp`,
    `gatsby-plugin-sharp`,
    'gatsby-plugin-sass',
    {
      resolve: `gatsby-source-filesystem`,
      options: {
        path: `${__dirname}/content/blog`,
        name: `blog`,
      },
    },
    {
      resolve: `gatsby-source-filesystem`,
      options: {
        path: `${__dirname}/content`,
        name: `assets`,
      },
    },
    {
      resolve: `gatsby-source-filesystem`,
      options: {
        path: `${__dirname}/content/images`,
        name: `assets`,
      },
    },
    {
      resolve: `gatsby-source-filesystem`,
      options: {
        path: `${__dirname}/content/assets`,
        name: `assets`,
      },
    },
    {
    resolve: `gatsby-source-filesystem`,
    options: {
      path: `${__dirname}/src/images`,
      name: `images`,
      },
    },
    {
      resolve: `gatsby-plugin-styled-components`,
      options: {
        // Add any options here
      },
    },
    {
      resolve: `gatsby-plugin-mdx`,
      options: {
        extensions: ['.mdx', '.md'],
        defaultLayouts: require.resolve('./src/components/layout/layout.js'),
        gatsbyRemarkPlugins: [
          {
            resolve: `gatsby-remark-images`,
            options: {
              maxWidth: 1280,
            },
          },
          {
            resolve: `gatsby-remark-responsive-iframe`,
            options: {
              wrapperStyle: `margin-bottom: 1.0725rem`,
            },
          },
          {
            resolve: `gatsby-remark-copy-linked-files`,
          },

          {
            resolve: `gatsby-remark-smartypants`,
          },
          {
            resolve: `gatsby-remark-prismjs`,
          },
        ],
        plugins: [ `gatsby-remark-images`],
      },
    },
    `gatsby-plugin-emotion`,
    `gatsby-plugin-sitemap`,
    {
      resolve: `gatsby-plugin-google-analytics`,
      options: {
        trackingId: `UA-45568024-1`,
      },
    },
    {
      resolve: `gatsby-plugin-feed`,
      options: {
        query: `
          {
            site {
              siteMetadata {
                title
                description
                siteUrl
              }
            }
          }
        `,
        feeds: [
          {
            serialize: ({ query: { site, allMdx } }) => {
              return allMdx.edges.map(edge => {
                return Object.assign({}, edge.node.frontmatter, {
                  description: edge.node.excerpt,
                  data: edge.node.frontmatter.date,
                  url: site.siteMetadata.siteUrl + edge.node.fields.slug,
                  guid: site.siteMetadata.siteUrl + edge.node.fields.slug,
                  custom_elements: [{ 'content:encoded': edge.node.body }],
                })
              })
            },

            /* if you want to filter for only published posts, you can do
             * something like this:
             * filter: { frontmatter: { published: { ne: false } } }
             * just make sure to add a published frontmatter field to all posts,
             * otherwise gatsby will complain
             **/
            query: `
            {
              allMdx(
                limit: 1000,
                sort: { order: DESC, fields: [frontmatter___date] },
              ) {
                edges {
                  node {
                    fields { slug }
                    frontmatter {
                      title
                      date
                    }
                    body
                  }
                }
              }
            }
            `,
            output: '/rss.xml',
            title: 'Gatsby RSS feed',
          },
        ],
      },
    },
    {
      resolve: `gatsby-plugin-manifest`,
      options: {
        name: `Technovature Software Website & Blog`,
        short_name: `Technovature Software`,
        start_url: `/`,
        background_color: `#ffffff`,
        theme_color: `#663399`,
        display: `minimal-ui`,
        icon: `content/assets/gatsby-icon.png`,
      },
    },
    `gatsby-plugin-offline`,
    `gatsby-plugin-react-helmet`,
  ],
}

And here is the contents of the gatsby-node.js file:

const path = require(`path`)
const { createFilePath } = require(`gatsby-source-filesystem`)

exports.createPages = ({ graphql, actions }) => {
  const { createPage } = actions

  const blogPost = path.resolve(`./src/templates/blog-post.js`)
  return graphql(
    `
      {
        allMdx(
          sort: { fields: [frontmatter___date], order: DESC }
          limit: 1000
        ) {
          edges {
            node {
              id
              fields {
                slug
              }
              frontmatter {
                title
              }
              body
            }
          }
        }
      }
    `
  ).then(result => {
    if (result.errors) {
      throw result.errors
    }

    // Create blog posts pages.
    const posts = result.data.allMdx.edges

    posts.forEach((post, index) => {
      const previous = index === posts.length - 1 ? null : posts[index + 1].node
      const next = index === 0 ? null : posts[index - 1].node

      createPage({
        path: post.node.fields.slug,
        component: blogPost,
        context: {
          slug: post.node.fields.slug,
          previous,
          next,
        },
      })
    })
  })
}

exports.onCreateNode = ({ node, actions, getNode }) => {
  const { createNodeField } = actions

  if (node.internal.type === `Mdx`) {
    const value = createFilePath({ node, getNode })
    createNodeField({
      name: `slug`,
      node,
      value,
    })
  }
}

Expected result

What should happen? Expecting graphql queries to work in mdx files in any folder in my "site-folder" site folder.

Actual result

What happened. I see the below Error:

Error: It appears like Gatsby is misconfigured. Gatsby related graphql calls are supposed to only be evaluated at compile time, and then compiled away. Unfortunately, something went wrong and the query was left in the compiled code.

Unless your site has a complex or custom babel/Gatsby configuration this is likely a bug in Gatsby.

Environment

Run gatsby info --clipboard in your project directory and paste the output here.

gatsby info --clipboard

System: OS: Windows 7 6.1.7601 CPU: (4) x64 Intel(R) Core(TM) i5 CPU M 560 @ 2.67GHz Binaries: Node: 12.14.1 - D:\Program Files\nodejs\node.EXE npm: 6.13.4 - D:\Program Files\nodejs\npm.CMD Languages: Python: 2.7.17 - /c/Python27/python npmPackages: gatsby: ^2.14.4 => 2.19.18 gatsby-image: ^2.0.22 => 2.2.41 gatsby-plugin-emotion: ^4.1.12 => 4.1.22 gatsby-plugin-feed: ^2.0.8 => 2.3.27 gatsby-plugin-google-analytics: ^2.0.5 => 2.1.35 gatsby-plugin-manifest: ^2.0.5 => 2.2.41 gatsby-plugin-mdx: ^1.0.33 => 1.0.73 gatsby-plugin-offline: ^2.0.5 => 2.2.10 gatsby-plugin-react-helmet: ^3.0.0 => 3.1.22 gatsby-plugin-sass: ^2.1.29 => 2.1.29 gatsby-plugin-sharp: ^2.0.6 => 2.4.5 gatsby-plugin-sitemap: ^2.2.19 => 2.2.27 gatsby-plugin-styled-components: ^3.1.19 => 3.1.19 gatsby-remark-copy-linked-files: ^2.0.5 => 2.1.37 gatsby-remark-images: ^3.1.19 => 3.1.44 gatsby-remark-prismjs: ^3.3.20 => 3.3.31 gatsby-remark-responsive-iframe: ^2.0.5 => 2.2.32 gatsby-remark-smartypants: ^2.0.5 => 2.1.21 gatsby-source-filesystem: ^2.0.2 => 2.1.48 gatsby-transformer-json: ^2.2.26 => 2.2.26 gatsby-transformer-remark: ^2.1.6 => 2.6.52 gatsby-transformer-sharp: ^2.1.3 => 2.3.14

jnsrikanth commented 4 years ago

@ChristopherBiscardi and @jonniebigodes - any pointers here would be of great help.

jonniebigodes commented 4 years ago

@jnsrikanth can you make a small repo with with the code you posted in your issue description? So that it can be better looked at? I don't mind taking a look at it, but with a repo backing this up would go a long way. Sounds good?

jnsrikanth commented 4 years ago

@jonniebigodes sure that sounds a good idea. I will send you a minified repo with the above configuration very shortly.

jnsrikanth commented 4 years ago

@jonniebigodes - here you go. https://github.com/jnsrikanth/github-mdx-repo

Let me know.

jnsrikanth commented 4 years ago

@jonniebigodes and just so that I am clear - In addition to being able to import graphql inside any mdx file across the "site" folder (not just the 'pages' folder), as a use case I am looking for a way to query for a specific blog title from within an MDX page using a graphql query and be able to apply a gatsby-link to it so that a user could navigate to a blog item from from the current MDX file (which is not necessarily inside a blog folder). Hope I am clear on stating the above?

Let me know.

Thanks, @jnsrikanth

jnsrikanth commented 4 years ago

@jonniebigodes - had a look into it? Were you able to find something? Looking forward to it. Silence nowadays from you makes me a bit nervous... ;)

jonniebigodes commented 4 years ago

@jnsrikanth sorry for the radio silence, but i had to address some things around my house, i had to close the laptop for a couple of days and put on my plumber pants (pardon the bad pun). Indeed i looked at it and i have a response for you. Going to detail it shortly,

Before we go a bit deeper and to give you some context based on my findings and a bit of research, when mdx was brought into Gatsby it was defined to operate in two modes. As a page parser/provider when you put mdx files in the src\pages folder or as a content provider when you put in mdx files in some folder other than src\pages and link it with gatsby-source-filesystem. Your case is actually a nice edge case to tackle as it uses both modes. So with that in mind, going to detail what i did and my findings.

a astronaut pulled into a mdx page through a graphql query

export const query = graphql query { astronaut: file(relativePath: { eq: "gatsby-astronaut.png" }) { childImageSharp { fluid(maxWidth: 300) { ...GatsbyImageSharpFluid } } } } ;

Our Locations

Hyderabad

Plot No. 18, Level 2, iLabs Centre, Oval Building,
Madhapur, Hitech City, Hyderabad 500081
Telangana, India

E-mail: info@technovature.com

Phone: +91 7013175234

- Issued `gatsby develop` and i'm presented with (ignore the bad formatting) the following:
![jnsrikanth_1](https://user-images.githubusercontent.com/22988955/76234087-21478700-6221-11ea-9623-82cf592821fe.png)

As you can see even though it's a page, it's actually going for the layout you defined in `gatsby-config.js`. With that in mind.

- Created a new layout component in `src\components\layout` called `default-page-layout.js` with the following:
```js
import React from "react";
import SiteSeach from "../search/site-search";
export default ({ children }) => (
  <div>
    <h1>My Layout</h1>

    <div>{children}</div>
  </div>
);
  • Changed gatsby-config.js to the following:

    // all the remainder config is the same
    {
      resolve: `gatsby-plugin-mdx`,
      options: {
        extensions: [".mdx", ".md"],
        /* defaultLayouts: require.resolve("./src/components/layout/layout.js"), */
        defaultLayouts: {
          posts: require.resolve("./src/components/layout/layout.js"), // this layout component will be for the posts
          default: require.resolve(
            "./src/components/layout/default-page-layout.js"
          ) // this is for the pages (mdx ones)
        },
        gatsbyRemarkPlugins: [
          {
            resolve: `gatsby-remark-images`,
            options: {
              maxWidth: 1280
            }
          },
          {
            resolve: `gatsby-remark-responsive-iframe`,
            options: {
              wrapperStyle: `margin-bottom: 1.0725rem`
            }
          },
          {
            resolve: `gatsby-remark-copy-linked-files`
          },
    
          {
            resolve: `gatsby-remark-smartypants`
          },
          {
            resolve: `gatsby-remark-prismjs`
          }
        ],
        plugins: [`gatsby-remark-images`]
      }

    With this, issuing again a new development build with yarn develop, yelds the following:

jnsrikanth_2

A quick tip for you. The Gatsby image does not work in mdx as it does in a normal js component. Even if you throw the fixed/fluid the image will always have the same size, hence the extra styling in it above.

Part of the issue is solved.

Now onto the next part, trying queries in other places.

I tried a couple of things and none pan out. I wasn't able to add queries and made them resolve in for instance a mdx blog post. In another folder. I mean your logic is sound and in paper it should work, but it will not. I think that is due to how Gatsby processes the graphql queries and materializes them. With a traditional workflow in Gatsby, every component that is tied to page or is inside a component that is tied to a page the introspection will go through and run the associated queries and materialize the data. Hence the metric ton of .json files and folders with uuids you get inside your public folder when you trigger a build. In there there's the materialization of the queries.

Also the docs back up this.

To be able to achieve this alot of work had to be done and i don't know if that's in the roadmap. As my fair generalistic assessment of the flow in mdx is the following:

  • The folder that contains the mdx is picked up by gatsby-source-filesystem.
  • The plugin starts it's job, generating nodes and with the help of @mdx/react transpilling the js and markdown and then passes onto Gatsby to continue it's job. (as a side note to whoever is maintaining the plugin, it would be a good thing that the plugin to be more permissive in development mode, meaning that it does not kill the development server if for instance i save a mdx file and i accidently forgot to close a html element or import. The messages in the console help out to some degree but killing the server is a bit to extreme ).

To include graphql in for instance in a post, when the instrospection and transpilation is going through the same process for a page would have to be done and with that alot of gatsby's parts would have to be modified and like i said above i do't know if that's in the roadmap for the foreseeable future.

But with this it doesn't mean you're left without options. I tinkered a bit and created a simple search functionality to be used in your site.

Created a new component called site-search.js in src\components\search with the following:

import React, { useState } from "react";
import { graphql, useStaticQuery, Link } from "gatsby";

const SiteSearch = () => {
  const [searchItem, setSearchItem] = useState("");
  const [searchResults, setSearchResults] = useState([]);
  const results = useStaticQuery(graphql`
    {
      allMdx {
        edges {
          node {
            frontmatter {
              title
            }
            fields {
              slug
            }
          }
        }
      }
    }
  `);
  const searchData = e => {
    e.preventDefault();
    if (searchItem.trim().length === 0) {
      return;
    }
    console.log(`searchData:${JSON.stringify(results,null,2)}`)
    const {allMdx}= results
    const {edges} = allMdx
    const blogSearch= edges.filter(x=>x.node.frontmatter.title.toLowerCase().includes(searchItem))
    console.log(`after search ${JSON.stringify(blogSearch,null,2)}`)
    setSearchResults(blogSearch)
  };
  return (
    <div>
      <form onSubmit={e => e.preventDefault()}>
        <label htmlFor="searchBox">
          Search
        </label>
        <input
          id="searchBox"
          name="searchBox"
          onChange={e => setSearchItem(e.target.value)}
          placeholder="Enter something to search"
        />
        <button type="submit" onClick={searchData}>Search</button>
      </form>
      <div>
        <h3>Results</h3>
        {searchResults.length === 0 ? (
          <p>Nothing so far. Try searching for something</p>
        ) : (
          searchResults.map(edge => (
            <Link to={edge.node.fields.slug}>
              <p>{edge.node.frontmatter.title}</p>
            </Link>
          ))
        )}
      </div>
    </div>
  );
};

export default SiteSearch;
  • Changed the default-page-layout.js file i mentioned above to the following:
import React from "react";
import SiteSeach from "../search/site-search";
export default ({ children }) => (
  <div>
    <h1>My Layout</h1>

    <div>{children}</div>
    <div style={{ marginTop: "1rem" }}>
      <SiteSeach />
    </div>
  </div>
);
  • Issued yarn develop to generate a development build and i opened up the same mdx page, more specifically contact.mdx and i'm presented with: jnsrikanth_3

And searching for something i'm presented with:

jnsrikanth_4

  • Changed your posts layout, more specifically src\components\layout\layout.js to:
    
    import React from "react";
    import PropTypes from "prop-types";
    import { StaticQuery, graphql } from "gatsby";
    import Head from "../head/head.js";
    import Header from "../header/header.js";
    import favicon from "../images/favicon.ico";
    import Helmet from "react-helmet";
    import SiteSeach from "../search/site-search";
    import "../../styles/global.css";

const Layout = ({ data, children }) => ( <>

{children}
© {new Date().getFullYear()}, Apple Tech Solutions Pvt. Ltd.

</> );

Layout.propTypes = { children: PropTypes.node.isRequired, data: PropTypes.object.isRequired };

const LayoutWithQuery = props => ( <StaticQuery query={graphql query LayoutQuery { site { siteMetadata { title } } } } render={data => <Layout data={data} {...props} />} /> );

LayoutWithQuery.propTypes = { children: PropTypes.node.isRequired };

export default LayoutWithQuery;


Doing a seach yelds the same result as with a page and the links are working properly. 

I know that's not what you wanted, but i wouldn't want to leave you without options, this is a bit of a middle ground to bypass some of the shortcomings of mdx in it's present state in Gatsby. You could probably could expand this to better suit your needs. 

Sorry for not being able to provide you with a solution for your issue.

Feel free to provide feedback so that we can close this issue, or continue to work on it until we find a suitable solution.
jnsrikanth commented 4 years ago

@jonniebigodes thanks for a detail drill down into the issue and providing some improvements/tips for running graphql queries inside MDX pages and a few options. The search functionality that you developed and provided seems very useful, although in a different context such as a site search itself.

What I was looking for in a more precise fashion is my ability to go directly to the path of a blog (a slug maybe?) using a (gatsby) Link if a qraphql query is not feasible. For example in the case of Services folder (/content/services) I have a set of folder names associated with AI, BigData, Blockchain etc. I am able to create links in the Home page (/src/pages/index.js) by making direct references to the path of that Services folder and pointing a Gatsby Link towards it. I am able to successfuly navigate in this case to that services folder where an index.mdx file is served from that folder.

Can I place a similar link to a blog post using a slug or a path into the Blog folder? I maybe already alluding to a possibility of a solution here, but I wanted to know what Gatsby provides as a more reliable method to do the same?

Let me know.

Thanks, @jnsrikanth

jnsrikanth commented 4 years ago

closing this ticket.

ebuildy commented 2 years ago

Actually this is working fine:

toto.mdx

---
title: Hot storage
---

import { graphql } from 'gatsby'

export const query = graphql`query MyQuery {
  allMdx {
    nodes {
      frontmatter {
        title
        dhub_stack
      }
    }
  }
}`

# Hello

<>{props.data.allMdx.nodes[0].frontmatter.title}</>