Plugin not pushing all pages to index

[stevetrask]

I am running the plugin with the terminal output looking ok 'Indexing of 1306 items' but when I look on the Algolia dashboard it looks to have only indexed 6 pages where it should be around 50

I have tried create a new index, removing any excluded files and using the default settings. I have also looked at the --verbose output and the data is there but not all of it is being pushed up, any ideas what might be causing this?

[stevetrask]

I found the issue it's nesting the data incorrectly and putting the pages inside other objects and duplicating object ids

[pixelastic]

The plugin will look at the final generated HTML pages to index content. In your case it seems like it is maybe indexing "index" or "archive" pages (that would contain many different posts in one HTML) and somehow skipping the most interesting files.

You can check if my assumptions are correct by looking at the url field of the records.

You can define your own set of excluded files by using the excluded_files option (see documentation here). Did you configure something specific in here?

[stevetrask]

Thanks for the response, I only excluded, 404, 502 and htaccess - seems to have got in a real muddle reading the data. We have decided to move to a more controlled way to do this, so Jekyll creates a json file that we can have more flexibility with, would be great if there was an off the self grunt task to push that files contents!