babenkoivan / elastic-migrations

Elasticsearch migrations for Laravel
MIT License
187 stars 32 forks source link

`elastic:migrate:fresh` command doesn't work due to protections on index #57

Closed stevebauman closed 3 weeks ago

stevebauman commented 2 months ago
Software Version
PHP 8.3
Elasticsearch 8.14.3
Laravel 10.48.16

Describe the bug

Running php artisan elastic:migrate:fresh fails due to wildcard drops being disabled for safety by default in Elasticsearch 8.0:

https://www.elastic.co/guide/en/elasticsearch/reference/current/index-management-settings.html#action-destructive-requires-name

This protection can be disabled via the below:

PUT /_cluster/settings
{
  "transient": {
    "action.destructive_requires_name":true
  }
}

Though I think we can just update the command to loop through each index and delete them by name.

To Reproduce

php artisan elastic:migrate:fresh
Elastic\Elasticsearch\Exception\ClientResponseException 
400 Bad Request: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Wildcard expressions or all indices are not allowed"}],"type":"illegal_argument_exception","reason":"Wildcard expressions or all indices are not allowed"},"status":400}
ElvisIsKing666 commented 2 months ago

I certainly don't want PHP dropping all of my indices - I'm glad to keep that protection. Using 'elastic:migrate:refresh' is what I want to use most of the time.

stevebauman commented 2 months ago

@ElvisIsKing666 this command is intentionally destructive and behaves exactly like the php artisan migrate:fresh command. Don't run it if you don't all your indicies dropped.

ElvisIsKing666 commented 2 months ago

Thanks @stevebauman - what I meant was that when I run artisan migrate:fresh it doesn't try to drop all the DB's in my local mysql - just the one it manages. But I am assuming that the error for using '*' wildcards is letting me know that this module is trying to drop more than just the single index that is currently managed. I have not dug into the code to see how it's running the index deletion - but wildcards sound bad.

stevebauman commented 2 months ago

An index is the equivalent to a DB table:

ElvisIsKing666 commented 2 months ago

Interesting - I hadn't seen the analogy. I was thinking of elasticsearch as the DB engine - not as a single DB. I don't know much about elasticsearch yet - I've only really just started - so then it would be wise for me to partition my other indices - like APM and Logstash so that something like migrate:fresh can't affect them. I've not looked at how things are partitioned.

babenkoivan commented 1 month ago

Hi there, I think this has been raised before :)

The main use case for php artisan elastic:migrate:fresh is to drop everything when running tests and it's okay to set action.destructive_requires_name = false on test environments. This is even mentioned in README:

Note that this command uses wildcards to delete indices. This requires setting action.destructive_requires_name to false.

If you want to be safe or don't want to tweak index settings, you can always use elastic:migrate:refresh, which rolls back all migrations and then runs them again.

github-actions[bot] commented 4 weeks ago

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 7 days