algolia / gatsby-plugin-algolia

A plugin to push to Algolia based on graphQl queries
https://yarn.pm/gatsby-plugin-algolia
Apache License 2.0
177 stars 45 forks source link

Only take chunksize? #7

Closed elvogel closed 6 years ago

elvogel commented 6 years ago

I'm using the Algolia plugin in a static blog site from the gatsby-starter-hero-blog and I'm running into an issue where my blog post is larger than the document size of 10k. Is it possible to set up the plugin so it merely take the 'first' chunksize amount without barfing an error over a larger-sized file? I'd prefer to simply get a warning that it's 'only' taking the first 10k or something, as opposed to trying to whittle down the size of the blog post.

Thanks!

Haroenv commented 6 years ago

You mean you’d rather have just the beginning of the post than splitting it up? Right now there’s no splitting yet in this plugin

elvogel commented 6 years ago

I'd love the split, but in its current state it's barfing on blog posts bigger than 10k

Haroenv commented 6 years ago

@elvogel, if I look at https://github.com/greglobinski/gatsby-starter-hero-blog/blob/master/src/utils/algolia.js (in the boilerplate you said you use), it is in fact already chunking the text. Could you give a configuration of a blog which throws that error?

elvogel commented 6 years ago
    {
      resolve: `gatsby-plugin-algolia`,
      options: {
        appId: process.env.ALGOLIA_APP_ID ? process.env.ALGOLIA_APP_ID : "",
        apiKey: process.env.ALGOLIA_ADMIN_API_KEY ? process.env.ALGOLIA_ADMIN_API_KEY : "",
        indexName: process.env.ALGOLIA_INDEX_NAME ? process.env.ALGOLIA_INDEX_NAME : "",
        queries,
        chunkSize: 10000 // default: 1000
      }
    },
Haroenv commented 6 years ago

The chunk size in the options of the plugin is to say how many items to push at once, not how big those items are. That logic should be done by yourself in the query transformer, like how it's done in the hero-blog starter I linked

Haroenv commented 6 years ago

Is this still an issue @elvogel? I'm confused about what was going on here, since they are splitting the posts.

elvogel commented 6 years ago

I'm in the process of upgrading to the newer version of the site template, which has that code. The version I was using didn't have it. Thanks

Haroenv commented 6 years ago

makes sense!