search

liquidweb / woocommerce-custom-orders-table

Store WooCommerce order data in a custom table for improved performance.
GNU General Public License v3.0
476 stars 51 forks source link

Use offsets with migration queries #157

Closed stevegrunwell closed 4 years ago

stevegrunwell commented 4 years ago

When an error is encountered during migration, the default behavior is to leave that post in-place and move on.

Previously, we would filter out these skipped IDs from the beginning of each new batch of migrations, but that would result in fewer and fewer orders being migrate with each batch.

For example, imagine we're doing batch sizes of 50 orders at a time. Of those 50, 5 orders have errors preventing migration. The next time we run the query, we pull 50 orders — the five we already knew had errors, plus the next 45 in the queue. If we find 4 more errors in this batch, the next time through the loop will automatically remove NINE orders, pulling in only 41.

To get around this, this commit rewrites the SQL query to use an offset; if we're pulling batches of 50 but have nine orders we know are bad, include LIMIT 9, 50 to start retrieval from the 10th item.

Fixes #148, fixes #112.

stevegrunwell commented 4 years ago

Re-ran the one broken test, it appears that the running of WC_Tests_WC_Order_Query::test_order_query_date_queries() can be somewhat inconsistent; with no other changes, suddenly it passed.

@bswatson, this is ready for your review.