I have a site with over 20k posts/pages/cpts and just as many taxonomies.
I have my max_allowed_packet set to 60MB and this naturally failed. I increased it to 512MB then 1GB and it still fails. This is something that can easily be controlled by this plugin.
Instead of making multiple massive queries, how about you break it up into multiple smaller queries? Given you have an array already, it wouldn't be hard to chunk it out and make that massive query multiple smaller queries.
I will see about adding a PR for this. It should not be expected to modify MySQL config just for an import process then to have to change it back. Having a very large max_allowed_packet (which the max is 1GB anyways) is not ideal and has numerous other concerns.
For example, I cannot get past:
// Get WPML translations.
$results = $wpdb->get_results( "SELECT * FROM {$wpdb->prefix}icl_translations" );
Since it returns over 800,000 records and even with 2GB of memory, it crashes with an out of memory error and this is just the beginning. Between memory and max_packet_size constraints, this is not ideal. Pagination would make the only requirement needed to be memory which is acceptable.
The requirements are a bit over the top for this since it looks like everything is done in a single query (why? this is not good) and there are numerous unnecessary $wpdb->prepare statements which is not efficient nor needed (being done on data fresh from the DB and some are just integers).
I have a site with over 20k posts/pages/cpts and just as many taxonomies.
I have my
max_allowed_packetset to 60MB and this naturally failed. I increased it to 512MB then 1GB and it still fails. This is something that can easily be controlled by this plugin.Instead of making multiple massive queries, how about you break it up into multiple smaller queries? Given you have an array already, it wouldn't be hard to chunk it out and make that massive query multiple smaller queries.
I will see about adding a PR for this. It should not be expected to modify MySQL config just for an import process then to have to change it back. Having a very large
max_allowed_packet(which the max is 1GB anyways) is not ideal and has numerous other concerns.For example, I cannot get past:
Since it returns over 800,000 records and even with 2GB of memory, it crashes with an out of memory error and this is just the beginning. Between memory and max_packet_size constraints, this is not ideal. Pagination would make the only requirement needed to be memory which is acceptable.
The requirements are a bit over the top for this since it looks like everything is done in a single query (why? this is not good) and there are numerous unnecessary
$wpdb->preparestatements which is not efficient nor needed (being done on data fresh from the DB and some are just integers).