humanmade / altis-cloud

Cloud Module for Altis
https://docs.altis-dxp.com/cloud/
9 stars 2 forks source link

ES index_closed_exception on some queries because index is closed and reopened every minute #644

Open tcrsavage opened 2 years ago

tcrsavage commented 2 years ago

Steps to reproduce: Related to: https://github.com/humanmade/altis-cloud/issues/379

  1. Run a large number of queries against 7.1 server with cavalcade jobs also running on hosted environment

See request errors for queries which were made when the index was closed by

https://github.com/humanmade/altis-enhanced-search/blob/0adbdb4736eb9086654e83c09a870f1b3c6464d0/inc/packages/namespace.php#L639

It looks like on_check_package_status is supposed to unschedule it's self, but it does not, so continually schedules altis.search.update_index_settings which fires the callback function above, which closes the index

https://github.com/humanmade/altis-cloud/blob/master/inc/elasticsearch_packages/namespace.php#L269

2 hours ago Warning[23-Jun-2022 11:58:02 UTC] PHP Warning: Error in ElasticSearch request: {"error":{"root_cause":[{"type":"index_closed_exception","reason":"closed","index_uuid":"Ez_2HEVKTR6y_iITvXFxjA","index":"{index_name}"}],"type":"index_closed_exception","reason":"closed","index_uuid":"Ez_2HEVKTR6y_iITvXFxjA","index":"ep-backendstagingservustvcom-post-12"},"status":400} (400) in /usr/src/app/vendor/altis/cloud/inc/namespace.php on line 387

tcrsavage commented 2 years ago

Ahh, too few arguments:

add_action( 'altis.search.check_package_status', __NAMESPACE__ . '\\on_check_package_status', 10, 4 );

roborourke commented 2 years ago

🤦 will get that patched up.