How to work with `gatsby-image` and Algolia on build?

[danoszz]

Introduction

First of all, thank you for making such a great plugin for implementing Algolia. It has been a real blessing for our development team. I am not really sure where to address this issue, but I've found out this issue occurs while using gatsby-plugin-algolia. Hence writing the issue here. Please let me know if I should bring this up somewhere else and this is unrelated to this plugin.

Problem

We have implemented search with Algolia which returns a title and an image of an art object. To optimize the loading times, we've used gatsby-image to render the GraphQL query returned by Algolia with featuredImage.childImageSharp.fluid. This works perfectly on gatsby develop.

However, when deploying or gatsby build && serve the images don't load and give a 404.

Hypothesis

• Synchronising our index with Algolia happens on gatsby-build
• Generated URL's of thumbnails bygatsby-image happens on gatsby build too

Our guess of the problem is that the URL's Algolia stores are not in sync with the URL's of the images of the built website.

Expected result

Algolia returns the same URL as requested in the rendered image

Actual Result

The URL Algolia returns is not the same as requested by the image

# Example requested URL
--- /static/1531268446cf37a32509968330a34e28/a9afa/98-theeservies.jpg

# Example stored URL at Algolia
+++ /static/1531268446cf37a32509968330a34e28/a07a5/98-theeservies.jpg

Steps to reproduce

1. Create a new Algolia Index and implement this plugin with your Gatsby website
2. Create a <customHits /> component and render an <Img fluid={yourSrc.fluid}/>
3. Query the data in gatsby-config.js something like this example

const yourQuery = { allMarkdownRemark(filter: {frontmatter: {templateKey: {eq: "your-item"}}}) { nodes { id fields { slug } frontmatter { title templateKey portrayedPersons featuredImage { childImageSharp { fluid(maxWidth: 600, quality: 80, toFormat: JPG) { aspectRatio src srcSet sizes originalImg originalName } } } } } } }

const finalQuery = [ { query: yourQuery, transformer: ({ data }) => data.allMarkdownRemark.nodes } ]

module.exports = { plugins: [ { resolve: "gatsby-plugin-algolia", options: { appId: process.env.GATSBY_ALGOLIA_APP_ID, apiKey: process.env.ALGOLIA_ADMIN_KEY, indexName: process.env.ALGOLIA_INDEX_NAME, queries: finalQuery } } / your other plugins such needed for gatsby-images, read more about that here https://www.gatsbyjs.org/docs/gatsby-image/#images-that-stretch-across-a-fluid-container / ] }

4. Run `gatsby develop` and everything should be working
5. Running `gatsby build && gatsby serve` should give a 404 on the image

# Question

How can we make sure the URL's are the same to get `gatsby-image` and Algolia working on build? Is it possible to trigger synchronisation manually?

### Environment

System: OS: macOS 10.15.2 CPU: (4) x64 Intel(R) Core(TM) i5-7287U CPU @ 3.30GHz Shell: 5.7.1 - /bin/zsh Binaries: Node: 13.11.0 - /usr/local/bin/node Yarn: 1.22.4 - /usr/local/bin/yarn npm: 6.14.4 - /usr/local/bin/npm Languages: Python: 2.7.16 - /usr/bin/python Browsers: Chrome: 80.0.3987.149 Firefox: 66.0.3 Safari: 13.0.4 npmPackages: gatsby: ^2.20.9 => 2.20.9 gatsby-cli: ^2.11.3 => 2.11.3 gatsby-image: ^2.3.1 => 2.3.1 gatsby-plugin-algolia: ^0.5.0 => 0.5.0 gatsby-plugin-netlify: ^2.2.1 => 2.2.1 gatsby-plugin-netlify-cms: ^4.2.2 => 4.2.2 gatsby-plugin-prefetch-google-fonts: ^1.4.3 => 1.4.3 gatsby-plugin-react-helmet: ^3.2.1 => 3.2.1 gatsby-plugin-sass: ^2.2.1 => 2.2.1 gatsby-plugin-sharp: ^2.5.3 => 2.5.3 gatsby-plugin-typography: ^2.4.1 => 2.4.1 gatsby-remark-copy-linked-files: ^2.2.1 => 2.2.1 gatsby-remark-images: ^3.2.1 => 3.2.1 gatsby-remark-relative-images: ^0.3.0 => 0.3.0 gatsby-source-filesystem: ^2.2.2 => 2.2.2 gatsby-transformer-remark: ^2.7.1 => 2.7.1 gatsby-transformer-sharp: ^2.4.3 => 2.4.3 npmGlobalPackages: gatsby-cli: 2.10.4

[danoszz]

Update

Since this is a breaking bug for a week now in our release to live, I'll keep logging personal finds and updates regarding the issue. Maybe the rubber-repo-duck-method helps us 💁‍♂️

After replacing the links in Algolia manually, the image works! This means that the problem is definitely in between the generated thumbnails by gatsby-image and the sync to Algolia.

[Haroenv]

hmm, can you check if the image plugin is before, or after the algolia plugin in the configuration? I think if the algolia plugin is listed first, the images aren't yet created, and thus old urls might be indexed.

If the Algolia plugin is already last, it might be related to how the urls are created (maybe async, and later in the life cycle than onPostBuild?).

Can you create a simple reproduction with just a few images? No need to put Algolia credentials, I can test with my own :)

[danoszz]

hmm, can you check if the image plugin is before, or after the algolia plugin in the configuration? I think if the algolia plugin is listed first, the images aren't yet created, and thus old urls might be indexed.

The Algolia plugin was before the image plugins. Just moved it as last and tested with gatsby clean && gatsby build && gatsby serve. Did not work, although I found something interesting:

1. If I didn't run gatsby clean but only gatsby build && gatsby serve the images show up (!). Could this be a cache problem?
2. This could be a nice workaround, but in production, it is desirable to build once and deploy since NetlifyCMS pushes changes to production automatically.

If the Algolia plugin is already last, it might be related to how the urls are created (maybe async, and later in the life cycle than onPostBuild?).

I am not really sure since I don't know a lot about this. I'll research this.

Can you create a simple reproduction with just a few images? No need to put Algolia credentials, I can test with my own :)

On it! Will post the link ASAP.

[danoszz]

@Haroenv it took me a -long- while, but this is the most minimal I could get without breaking stuff and losing concepts.

🔗 Repo with minimal reproduction

[Haroenv]

@danozz when I try building this (yarn build), I get the same id in Algolia as in my public folder. Is this not what you're getting?

[danoszz]

@danozz when I try building this (yarn build), I get the same id in Algolia as in my public folder. Is this not what you're getting?

It also compiles without errors here, but the problem occurs when I run gatsby serve or deploy to Netlify. What kind of id do you mean? The objectID?

[Haroenv]

I meant the image id seems to be the same as in the public folder. I'm surprised that the id changes again once you serve, it doesn't for me. Do you run the build on netlify or on your machine?

[danoszz]

@Haroenv The images don't show up on the local build and Netlfify build

[danoszz]

Thank you for testing it @Haroenv this means my machine is having a rough week, not gatsby-plugin-algolia. Do the images show up when you run gatsby serve too,?

I've taken some time to clean up the repo and the minimal reproduction works right now too locally and live. So, it's the production build which isn't running. Brings me a step closer.

My assumption, for now, is that I have some unwiped traces of an integrating Netlify Large Media, which, unfortunately isn't supported. I will keep this thread alive for now to post my findings and possible solution if someone ever runs into this specific problem!

[Haroenv]

That's a good sign. Maybe it's worth changing the name of all your images to make sure no old cache can be used? I don't think you can globally clean Netlify's cache

[danoszz]

So I will close this topic down. It seems that the culprit is something with sharp and Node versions. Not sure what yet, but definitely not this plugin.

Thank you again for your help. 👨‍💻