.eachBatch() does not accept batch size argument

[leonatoieiri]

According to the documentation eachBatch() have an optional argument to customize the batch size, however this is not working at the moment.

I think this is happening because this feature is only available on Waterline 0.14, and the current version available on npm is 0.13.6

This is the error message:

UsageError: Missing or invalid iteratee function for .stream(). Details: For eachBatchFn, instead of a function, got: 100

Sample code:

await Subscriber.stream({ country: 'pt' }).eachBatch(100, async records => { console.log('Got ' + records.length + ' records.'); });

Link to the documentation

Node version: v12.10.0 Sails version (sails): sails@1.2.3 ORM hook version (sails-hook-orm): sails-hook-orm@2.1.1 Sockets hook version (sails-hook-sockets): sails-hook-sockets@2.0.0 DB adapter & version (e.g. sails-mysql@5.55.5): sails-mysql@1.0.1

---

[sailsbot]

@leonatoieiri Thanks for posting! We'll take a look as soon as possible.

In the mean time, there are a few ways you can help speed things along:

• look for a workaround. (Even if it's just temporary, sharing your solution can save someone else a lot of time and effort.)
• tell us why this issue is important to you and your team. What are you trying to accomplish? (Submissions with a little bit of human context tend to be easier to understand and faster to resolve.)
• make sure you've provided clear instructions on how to reproduce the bug from a clean install.
• double-check that you've provided all of the requested version and dependency information. (Some of this info might seem irrelevant at first, like which database adapter you're using, but we ask that you include it anyway. Oftentimes an issue is caused by a confluence of unexpected factors, and it can save everybody a ton of time to know all the details up front.)
• read the code of conduct.
• if appropriate, ask your business to sponsor your issue. (Open source is our passion, and our core maintainers volunteer many of their nights and weekends working on Sails. But you only get so many nights and weekends in life, and stuff gets done a lot faster when you can work on it during normal daylight hours.)
• let us know if you are using a 3rd party plugin; whether that's a database adapter, a non-standard view engine, or any other dependency maintained by someone other than our core team. (Besides the name of the 3rd party package, it helps to include the exact version you're using. If you're unsure, check out this list of all the core packages we maintain.)

  ---

Please remember: never post in a public forum if you believe you've found a genuine security vulnerability. Instead, disclose it responsibly.

For help with questions about Sails, click here.

[kesgri]

Same here. Docs say there is another way: .meta({batchSize: 100}). for eachRecord. Tried using it with eachBatch (hoped it would work)... I need to go through 50M records and in batches of 30 it's plain slow.

[johnabrams7]

@leonatoieiri Thanks for taking the time to explore this. To further understand the issue, can you provide us a minimal Sails repo that reproduces the error? I think you might be right about the Waterline version causing this limitation; if that's the case, we'll update the docs until that version is released.

[sereisoglu]

Hi, I have the same problem. Any updates?

[eashaw]

Hi @leonatoieiri and @sereisoglu, we discovered that sails-generate was using an old version of sails-hook-orm, this has been fixed. To fix this issue in your existing app, you'll need to update sails-hook-orm to v3.0.0, which includes Waterline 0.14.

[sereisoglu]

@eashaw It's works. Thank you.