search

Automattic / woocommerce-subscriptions-core

Subscriptions core package for WooCommerce
Other
86 stars 32 forks source link

CI: Database tables don't exist when running unit tests #156

Open brucealdridge opened 2 years ago

brucealdridge commented 2 years ago

Describe the bug

When running the unit tests in github actions, the logs show that not all tables are setup correctly. All tests show as completing successfully and is just a logged error.

Log snippet

............................................<div id="error"><p class="wpdberror"><strong>WordPress database error:</strong> [Table &#039;woocommerce_test.wptests_wc_order_stats&#039; doesn&#039;t exist]<br /><code>SELECT customer_id FROM wptests_wc_order_stats WHERE order_id = 626</code></p></div><div id="error"><p class="wpdberror"><strong>WordPress database error:</strong> [Table &#039;woocommerce_test.wptests_wc_order_stats&#039; doesn&#039;t exist]<br /><code>SHOW FULL COLUMNS FROM `wptests_wc_order_stats`</code></p></div><div id="error"><p class="wpdberror"><strong>WordPress database error:</strong> [Table &#039;woocommerce_test.wptests_wc_order_coupon_lookup&#039; doesn&#039;t exist]<br /><code>SHOW FULL COLUMNS FROM `wptests_wc_order_coupon_lookup`</code></p></div><div id="error"><p class="wpdberror"><strong>WordPress database error:</strong> [Table &#039;woocommerce_test.wptests_wc_order_product_lookup&#039; doesn&#039;t exist]<br /><code>SHOW FULL COLUMNS FROM `wptests_wc_order_product_lookup`</code></p></div><div id="error"><p class="wpdberror"><strong>WordPress database error:</strong> [Table &#039;woocommerce_test.wptests_wc_order_tax_lookup&#039; doesn&#039;t exist]<br /><code>SHOW FULL COLUMNS FROM `wptests_wc_order_tax_lookup`</code></p></div><div id="error"><p class="wpdberror"><strong>WordPress database error:</strong> [Table &#039;woocommerce_test.wptests_wc_order_stats&#039; doesn&#039;t exist]<br /><code>SELECT customer_id FROM wptests_wc_order_stats WHERE order_id = 627</code></p></div><div id="error"><p class="wpdberror"><strong>WordPress database error:</strong> [Table &#039;woocommerce_test.wptests_wc_order_stats&#039; doesn&#039;t exist]<br /><code>SHOW FULL COLUMNS FROM `wptests_wc_order_stats`</code></p></div><div id="error"><p class="wpdberror"><strong>WordPress database error:</strong> [Table &#039;woocommerce_test.wptests_wc_order_coupon_lookup&#039; doesn&#039;t exist]<br /><code>SHOW FULL COLUMNS FROM `wptests_wc_order_coupon_lookup`</code></p></div><div id="error"><p class="wpdberror"><strong>WordPress database error:</strong> [Table &#039;woocommerce_test.wptests_wc_order_product_lookup&#039; doesn&#039;t exist]<br /><code>SHOW FULL COLUMNS FROM `wptests_wc_order_product_lookup`</code></p></div><div id="error"><p class="wpdberror"><strong>WordPress database error:</strong> [Table &#039;woocommerce_test.wptests_wc_order_tax_lookup&#039; doesn&#039;t exist]<br /><code>SHOW FULL COLUMNS FROM `wptests_wc_order_tax_lookup`</code></p></div>...<div id="error"><p class="wpdberror"><strong>WordPress database error:</strong> [Table &#039;woocommerce_test.wptests_wc_order_stats&#039; doesn&#039;t exist]<br /><code>SELECT customer_id FROM wptests_wc_order_stats WHERE order_id = 645</code></p></div><div id="error"><p class="wpdberror"><strong>WordPress database error:</strong> [Table &#039;woocommerce_test.wptests_wc_order_stats&#039; doesn&#039;t exist]<br /><code>SHOW FULL COLUMNS FROM `wptests_wc_order_stats`</code></p></div><div id="error"><p class="wpdberror"><strong>WordPress database error:</strong> [Table &#039;woocommerce_test.wptests_wc_order_coupon_lookup&#039; doesn&#039;t exist]<br /><code>SHOW FULL COLUMNS FROM `wptests_wc_order_coupon_lookup`</code></p></div><div id="error"><p class="wpdberror"><strong>WordPress database error:</strong> [Table &#039;woocommerce_test.wptests_wc_order_product_lookup&#039; doesn&#039;t exist]<br /><code>SHOW FULL COLUMNS FROM `wptests_wc_order_product_lookup`</code></p></div><div id="error"><p class="wpdberror"><strong>WordPress database error:</strong> [Table &#039;woocommerce_test.wptests_wc_order_tax_lookup&#039; doesn&#039;t exist]<br /><code>SHOW FULL COLUMNS FROM `wptests_wc_order_tax_lookup`</code></p></div>................ 693 / 999 ( 69%)
............................................S.S.S.S............ 756 / 999 ( 75%)
............................................................... 819 / 999 ( 81%)
............................................................... 882 / 999 ( 88%)
............................................................... 945 / 999 ( 94%)
......................................................          999 / 999 (100%)

Time: 1.04 minutes, Memory: 94.00MB

OK, but incomplete, skipped, or risky tests!
Tests: 999, Assertions: 2390, Skipped: 10.

View the full logs

brucealdridge commented 1 year ago

Found this again looking into #324 I'm not sure why this is generating so many errors and yet still reporting that the tests pass. Removing low priority label as this could be a sign of a larger problem with the tests.

A small snippet from a recent log.

PHPUnit 6.5.14 by Sebastian Bergmann and contributors.

.............................................................   61 / 1008 (  6%)
WordPress database error Table 'woocommerce_test.wptests_wc_order_stats' doesn't exist for query SELECT customer_id FROM wptests_wc_order_stats WHERE order_id = 246 made by PHPUnit\TextUI\Command::main, PHPUnit\TextUI\Command->run, PHPUnit\TextUI\TestRunner->doRun, PHPUnit\Framework\TestSuite->run, PHPUnit\Framework\TestSuite->run, PHPUnit\Framework\TestCase->run, PHPUnit\Framework\TestResult->run, PHPUnit\Framework\TestCase->runBare, PHPUnit\Framework\TestCase->runTest, ReflectionMethod->invokeArgs, WC_Subscriptions_Test->test_get_related_orders, WC_Data->delete, WC_Data_Store->delete, Abstract_WC_Order_Data_Store_CPT->delete, wp_delete_post, do_action('delete_post'), WP_Hook->do_action, WP_Hook->apply_filters, Automattic\WooCommerce\Admin\API\Reports\Orders\Stats\DataStore::delete_order, Automattic\WooCommerce\Admin\API\Reports\Customers\DataStore::get_existing_customer_id_from_order
WordPress database error Table 'woocommerce_test.wptests_wc_order_stats' doesn't exist for query SHOW FULL COLUMNS FROM `wptests_wc_order_stats` made by PHPUnit\TextUI\Command::main, PHPUnit\TextUI\Command->run, PHPUnit\TextUI\TestRunner->doRun, PHPUnit\Framework\TestSuite->run, PHPUnit\Framework\TestSuite->run, PHPUnit\Framework\TestCase->run, PHPUnit\Framework\TestResult->run, PHPUnit\Framework\TestCase->runBare, PHPUnit\Framework\TestCase->runTest, ReflectionMethod->invokeArgs, WC_Subscriptions_Test->test_get_related_orders, WC_Data->delete, WC_Data_Store->delete, Abstract_WC_Order_Data_Store_CPT->delete, wp_delete_post, do_action('delete_post'), WP_Hook->do_action, WP_Hook->apply_filters, Automattic\WooCommerce\Admin\API\Reports\Orders\Stats\DataStore::delete_order
WordPress database error Table 'woocommerce_test.wptests_wc_order_coupon_lookup' doesn't exist for query SHOW FULL COLUMNS FROM `wptests_wc_order_coupon_lookup` made by PHPUnit\TextUI\Command::main, PHPUnit\TextUI\Command->run, PHPUnit\TextUI\TestRunner->doRun, PHPUnit\Framework\TestSuite->run, PHPUnit\Framework\TestSuite->run, PHPUnit\Framework\TestCase->run, PHPUnit\Framework\TestResult->run, PHPUnit\Framework\TestCase->runBare, PHPUnit\Framework\TestCase->runTest, ReflectionMethod->invokeArgs, WC_Subscriptions_Test->test_get_related_orders, WC_Data->delete, WC_Data_Store->delete, Abstract_WC_Order_Data_Store_CPT->delete, wp_delete_post, do_action('delete_post'), WP_Hook->do_action, WP_Hook->apply_filters, Automattic\WooCommerce\Admin\API\Reports\Orders\Stats\DataStore::delete_order, do_action('woocommerce_analytics_delete_order_stats'), WP_Hook->do_action, WP_Hook->apply_filters, Automattic\WooCommerce\Admin\API\Reports\Coupons\DataStore::sync_on_order_delete
WordPress database error Table 'woocommerce_test.wptests_wc_order_product_lookup' doesn't exist for query SHOW FULL COLUMNS FROM `wptests_wc_order_product_lookup` made by PHPUnit\TextUI\Command::main, PHPUnit\TextUI\Command->run, PHPUnit\TextUI\TestRunner->doRun, PHPUnit\Framework\TestSuite->run, PHPUnit\Framework\TestSuite->run, PHPUnit\Framework\TestCase->run, PHPUnit\Framework\TestResult->run, PHPUnit\Framework\TestCase->runBare, PHPUnit\Framework\TestCase->runTest, ReflectionMethod->invokeArgs, WC_Subscriptions_Test->test_get_related_orders, WC_Data->delete, WC_Data_Store->delete, Abstract_WC_Order_Data_Store_CPT->delete, wp_delete_post, do_action('delete_post'), WP_Hook->do_action, WP_Hook->apply_filters, Automattic\WooCommerce\Admin\API\Reports\Orders\Stats\DataStore::delete_order, do_action('woocommerce_analytics_delete_order_stats'), WP_Hook->do_action, WP_Hook->apply_filters, Automattic\WooCommerce\Admin\API\Reports\Products\DataStore::sync_on_order_delete
WordPress database error Table 'woocommerce_test.wptests_wc_order_tax_lookup' doesn't exist for query SHOW FULL COLUMNS FROM `wptests_wc_order_tax_lookup` made by PHPUnit\TextUI\Command::main, PHPUnit\TextUI\Command->run, PHPUnit\TextUI\TestRunner->doRun, PHPUnit\Framework\TestSuite->run, PHPUnit\Framework\TestSuite->run, PHPUnit\Framework\TestCase->run, PHPUnit\Framework\TestResult->run, PHPUnit\Framework\TestCase->runBare, PHPUnit\Framework\TestCase->runTest, ReflectionMethod->invokeArgs, WC_Subscriptions_Test->test_get_related_orders, WC_Data->delete, WC_Data_Store->delete, Abstract_WC_Order_Data_Store_CPT->delete, wp_delete_post, do_action('delete_post'), WP_Hook->do_action, WP_Hook->apply_filters, Automattic\WooCommerce\Admin\API\Reports\Orders\Stats\DataStore::delete_order, do_action('woocommerce_analytics_delete_order_stats'), WP_Hook->do_action, WP_Hook->apply_filters, Automattic\WooCommerce\Admin\API\Reports\Taxes\DataStore::sync_on_order_delete
..........................................................<div id="error"><p class="wpdberror"><strong>WordPress database error:</strong> [Table &#039;woocommerce_test.wptests_wc_order_stats&#039; doesn&#039;t exist]<br /><code>SELECT customer_id FROM wptests_wc_order_stats WHERE order_id = 246</code></p></div><div id="error"><p class="wpdberror"><strong>WordPress database error:</strong> [Table &#039;woocommerce_test.wptests_wc_order_stats&#039; doesn&#039;t exist]<br /><code>SHOW FULL COLUMNS FROM `wptests_wc_order_stats`</code></p></div><div id="error"><p class="wpdberror"><strong>WordPress database error:</strong> [Table &#039;woocommerce_test.wptests_wc_order_coupon_lookup&#039; doesn&#039;t exist]<br /><code>SHOW FULL COLUMNS FROM `wptests_wc_order_coupon_lookup`</code></p></div><div id="error"><p class="wpdberror"><strong>WordPress database error:</strong> [Table &#039;woocommerce_test.wptests_wc_order_product_lookup&#039; doesn&#039;t exist]<br /><code>SHOW FULL COLUMNS FROM `wptests_wc_order_product_lookup`</code></p></div><div id="error"><p class="wpdberror"><strong>WordPress database error:</strong> [Table &#039;woocommerce_test.wptests_wc_order_tax_lookup&#039; doesn&#039;t exist]<br /><code>SHOW FULL COLUMNS FROM `wptests_wc_order_tax_lookup`</code></p></div>...  122 / 1008 ( 12%)
......................S......................................  183 / 1008 ( 18%)
.S.S.......................................SSS...............  244 / 1008 ( 24%)
haszari commented 1 year ago

When running the unit tests in github actions, the logs show that not all tables are setup correctly. All tests show as completing successfully and is just a logged error.

Super weird, how can the tests pass if this error matters? Would be good to tidy this up for sure.

haszari commented 1 year ago

Linking to the epic:

This seems like a major gap in our Unit Test infra, though I'm not aware of the status of our unit tests, I'm assuming we have a reasonable suite of tests (i.e. they're not a work in progress or an incomplete experiment).

Marked as high priority, though this (and other type: task issues) generally should not be prioritised over fixes impacting merchants.