Chunks sizes should be customisable

[sixela]

Is your feature request related to a problem? Please describe.

We're trying to use MainWP for a very large number of child sites (1k+ minimum) and are hitting "performance" issues: some operation take too long to finish despite increasing the number of simultaneous requests via the settings panel.

Example: adding a user to 100 sites takes roughly 50s

Describe the solution you'd like

I believe the issue comes from the hardcoded value of 10 for mainwp_fetch_urls_chunk_size which induces a sleep( 5 ) for each loop, here. IMHO this should probably be available for easy customization.

After a very quick glance at the rest of the code it's highly possible other similar chunked-loops could benefit from easy tweaking as well.

Describe alternatives you've considered

• Maye the chunk size could follow the max simultaneous requests from settings
• Maybe the sleep could be reduced and/or removed

Additional context

Maybe there's a valid reason for that 10-sized chunked loop but i'm having a hard time understanding it, moreover I could only go back to commit 0cbcb9d013bb21d5c88fba9088de5e94a45e10f7 without finding any sort of justification for that limit

[bogdan-mainwp]

Hi @sixela, thanks for your patience.

I just checked the like you referenced and noticed that the chunk size is not hard-coded to 10. 10 is set as default value, but you can use the mainwp_fetch_urls_chunk_size filter to change that value on your setup. Have you tried that?

thanks

[sixela]

Thanks for the feedback, totally missed that indeed :) Sure thing i'll try, I guess you can close this non-issue then !

Patience as nothing to do here, there were no hurry of any kind ;-)

However, I'm curious to understand why such a low default limit and why that sleep 5 on each loop...why not simply go all hands on the provided list instead of doing in chunks ?