search

akhoury / nodebb-plugin-import

migrate your old crappy forum to NodeBB
MIT License
78 stars 31 forks source link

How do I change the batch size? #225

Open djensen47 opened 6 years ago

djensen47 commented 6 years ago

It looks like the batch size may not be configurable and it is hardcoded to 600000. But I might be wrong. Is there a way to configure a smaller batch size. I think I have a query that is running long.

akhoury commented 6 years ago

not yet, but I can add it, if you can't wait, find this file node_module/nodebb-plugin-import/server/exporter/index.js and change these 2 values, see if this helps 

var COUNT_BATCH_SIZE = 600000;
var DEFAULT_EXPORT_BATCH_SIZE = 600000;

https://github.com/akhoury/nodebb-plugin-import/blob/master/server/exporter/index.js#L8-L9

CherryDT commented 2 years ago

Note that there also seems to be a bug where it always skips 1 record after each batch. Change batch + 1 to batch in the same file to fix that (there is only one occurence).