GraphQL Error Unknown field on type `Query` on first builds but successful after a second run

[vktrwlt]

Hi, When running gatsby develop or build most of the time it will fail on the first try and then without any code changes running the same command after will build successfully.

I am writing my own source plugin so maybe I am missing a step but during the development process before I implemented page queries, the builds will always compile and I could query the schema. I would then transfer the query from graphiql to in page query and I would get build errors.

This is the error I am getting: success open and validate gatsby-configs — 0.008 s success load plugins — 0.146 s success onPreInit — 0.485 s success delete html and css files from previous builds — 0.022 s success initialize cache — 0.005 s success copy gatsby files — 0.065 s success onPreBootstrap — 0.006 s success source and transform nodes — 0.056 s success building schema — 0.235 s success createPages — 0.001 s success createPagesStatefully — 0.140 s success onPreExtractQueries — 0.008 s success update schema — 0.474 s error GraphQL Error Unknown field allLa2019 on type Query

After its fails, I run the same command again and sometime it will be successful success open and validate gatsby-configs — 0.007 s success load plugins — 0.148 s success onPreInit — 0.283 s success delete html and css files from previous builds — 0.016 s success initialize cache — 0.010 s success copy gatsby files — 0.076 s success onPreBootstrap — 0.005 s success source and transform nodes — 0.054 s success building schema — 0.611 s success createPages — 0.001 s success createPagesStatefully — 0.160 s success onPreExtractQueries — 0.005 s success update schema — 0.455 s success extract queries from components — 0.156 s success run graphql queries — 0.104 s — 8/8 78.14 queries/second success write out page data — 0.004 s success write out redirect data — 0.001 s ⢀ onPostBootstrapdone generating icons for manifest success onPostBootstrap — 0.305 s

info bootstrap finished - 3.904 s

[pieh]

Does your source plugin do async operation to get data (i.e. do network request)? Do you return promise from your sourceNodes API hook to let gatsby know when your plugin finished?

[vktrwlt]

exports.sourceNodes = async ({ actions, createNodeId, createContentDigest }, configOptions) => {
    const { createNode } = actions;

    const dataUrls = [
        {
            name: 'ldn2019',
            url: 'url'
        },
        {
            name: 'la2019',
            url: 'url'
        }
    ];

    dataUrls.forEach(async url => {
        // console.log(url);

        const data = await axios.get(url.url);

        const processDatum = data => {
            let id = data.ID || data.id;

            const nodeId = createNodeId(`${url.name}-${id}`);
            const nodeContent = JSON.stringify(data);

            const nodeData = Object.assign({}, data, {
                id: nodeId,
                parent: null,
                children: [],
                internal: {
                    type: `${url.name}`,
                    content: nodeContent,
                    contentDigest: createContentDigest(data)
                }
            });

            return nodeData;
        };

        data.data.forEach(datum => createNode(processDatum(datum)));
    });
};

This is my sourceNode code, it does return a promise

[jonniebigodes]

@victorwltsang i've picked up on the your issue, and the code you posted left me a bit intrigued and without any more knowledge of the plugin you're actually implementing, i'm going need for you to confirm my code analysis on what you posted. sounds good? Ok, taking it from the top, and by the time you've seen this your code will edited to provide better readability.

// assuming that you have already added axios to your dependencies and required it on the file.
const dataUrls = [
        {
            name: 'ldn2019',
            url: 'url'//<- you forgot a ' character here don't know if it was intentional so that the url was not seen, but i'm going with that.
        },
        {
            name: 'la2019',
            url: 'url'
        }
    ];

Onto the next part

dataUrls.forEach(async url=>{
        const data= await axios.get(url.ur)
        const processDatum = data => {
            let id = data.ID || data.id;

            const nodeId = createNodeId(`${url.name}-${id}`);
            const nodeContent = JSON.stringify(data);
            const nodeData= Object.assign({},data,{
                id:nodeId,
                parent:null,
                children:[],
                internal:{
                    type:`${url.name}`,
                    content:nodeContent,
                    contentDigest: createContentDigest(data)
                }
            })
            return nodeData;
        }
        data.data.forEach(datum => createNode(processDatum(datum)));
       // return something?
    })

From what i'm reading here, and please feel free to correct me, but this is based on what i'm reading. You are iterating the dataUrls array, each item making a http get request, which in turn based on the data const containing the current get request you assign some other const with some values and invoke some of the apis functions and then return the transformed object and then you take on the data object coming from axios with the response you got and iterate that and call again another api function and go back to the processDatum?

And when the it all finishes you let it "die" nothing is actually returned, and as the docs states you need to return a promise empty or not, or just have the return statement there. On my machine with a couple of tests, it started by throwing some errors, and then it moved on to running indefinitely. I can only assume that on your end, the code builds on the second run because gatsby is being a bit permissive as the data you got back is cached and it will only focus on going forward building incrementally.

If you don't mind my two cents on the issue, but i would recomend a different approach, and this because it will save you some time tracking some issues that could arise further down the line. With that in mind what i would recomend, would be to break it into smaller pieces. Starting with taking a look at axios.all(), create what is basically an n length array of promises returned from the axios.get() calls, then treat the results and any 404's or 500 errors that could happen, when you're pulling the data, as you have to assume that networks are not 100% reliable and some urls you hit might be down for the count or maintenance, or any other unspecified reason. After that, process the data returned and invoke the gatsby api functions that you need. And finally after all the work is done, just use the return statement or if needed, the promise with the data you want in there. I know that this could sound a bit cumbersome in the js ecosystem with all their bells and whistles. But sometimes this type of approach is actually alot safer. Hoping to hear some further feedback from your part and see if you managed to correct the issue and have the code all up and running and the plugin published. 👍

[vktrwlt]

@jonniebigodes,

Thanks for pointing me in the right direction. I thought my code was returning a promise but it wasn't the case at all. I was basing my code on the gatsby pixabay example. I am trying to decouple a wordpress site with a plugin named wpheadless which seems to work pretty for the project I am working on. This is what I end up with.

const axios = require('axios');

exports.sourceNodes = ({ actions, createNodeId, createContentDigest }, configOptions) => {
    const { createNode } = actions;
    const myPromises = [];
    const dataUrls = [
        {
            name: 'ldn2019',
            url: 'hidden'
        },
        {
            name: 'la2019',
            url: 'hidden'
        }
    ];
    for (let i = 0; i < dataUrls.length; i++) {
        myPromises.push(
            new Promise((resolve, reject) => {
                axios.get(`${dataUrls[i].url}`).then(res => {
                    // map into these results and create nodes
                    res.data.map(data => {
                        // Create your node object
                        // console.log(data);
                        let id = data.ID || data.id;
                        if (data.speaker_list) {
                            data.speaker_list.map(speaker => {
                                // console.log('d', speaker.headshot);
                                if (speaker.headshot === false) {
                                    speaker.headshot = '';
                                }
                            });
                        }
                        const nodeId = createNodeId(`${dataUrls[i].name}-${id}`);
                        const nodeContent = JSON.stringify(data);
                        const nodeData = Object.assign({}, data, {
                            id: nodeId,
                            parent: null,
                            children: [],
                            internal: {
                                type: `${dataUrls[i].name}`,
                                content: nodeContent,
                                contentDigest: createContentDigest(data)
                            }
                        });
                        createNode(nodeData);
                    });
                    resolve();
                });
            })
        );
    }

    return Promise.all(myPromises);
};