Elasticsearch 6.0.0 support?

[breisig]

Now that Elasticearch 6.0.0 has been released [ https://www.elastic.co/blog/elasticsearch-6-0-0-released ], Is there support for chewy to work with 6.0.0?

[asabourin]

Curious about it too, we search across multiple types with Chewy so I suspect https://www.elastic.co/guide/en/elasticsearch/reference/6.0/removal-of-types.html will be a tricky one to handle. We'll do some testing and report.

[pyromaniac]

Will try to find a time for it the next week. But I assume it should work. Did you try it?

[mightystrong]

It doesn't work properly with 6.0. A couple of issues I could identify. First. is that 6.0 no longer recognizes the field type "string", which is chewy's default type. You have to change them all to "text".

Elasticsearch::Transport::Transport::Errors::BadRequest: [400] {"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"No handler for type [string] declared on field [type_name]"}],"type":"mapper_parsing_exception","reason":"Failed to parse mapping [journal]: No handler for type [string] declared on field [type_name]","caused_by":{"type":"mapper_parsing_exception","reason":"No handler for type [string] declared on field [type_name]"}},"status":400}

In addition, you're no longer able to add multiple models to one index. Here's an example of the error message you get when running rake chewy:reset with multiple types defined:

Elasticsearch::Transport::Transport::Errors::BadRequest: [400] {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Rejecting mapping update to [store_1511892608929] as the final mapping would have more than 1 type: [product, review_author, vendor, review]"}],"type":"illegal_argument_exception","reason":"Rejecting mapping update to [store_1511892608929] as the final mapping would have more than 1 type: [product, review_author, vendor, review]"},"status":400}

I also had to be sure to set config.eager_load = true in the config/environments/development.rb otherwise and got an error NameError: uninitialized constant StoreIndex when trying to save records.

[mightystrong]

See https://www.elastic.co/guide/en/elasticsearch/reference/6.0/breaking-changes-6.0.html for more info.

[engeen]

The main problem, as it seems to me, is the removal of multitype support https://www.elastic.co/guide/en/elasticsearch/reference/6.0/removal-of-types.html. Chewy's multiple types logic should be rewritten to use custom type field within the single mapping to use multiple index sources for the single index.

[pyromaniac]

It is not actually removed, it is just deprecated, so the only thing we need for users that will decide to use a single-type setup is some multi-index query interface. And maybe an ability to define mapping on index directly, without defining a type

[dmitrypol]

+1

[vishalsingh123]

Elasticsearch::Transport::Transport::Errors::BadRequest: [400] {"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"No handler for type [string] declared on field [type_name]"}],"type":"mapper_parsing_exception","reason":"Failed to parse mapping [journal]: No handler for type [string] declared on field [type_name]","caused_by":{"type":"mapper_parsing_exception","reason":"No handler for type [string] declared on field [type_name]"}},"status":400}

getting this error on "rake chewy:reset" ... i have changed default type from string to text still getting this error... My Elastic version is 6.1.1 please help ...

[sanychcz]

@vishalsingh123 If you run curl 'localhost:9200/_cat/indices?v' do you have any created indices?

I have the same problem, when I run rake chewy:reset, I get same error message, but index is always created. If you take a look on message from elastic log below, it says: deleting index, but when I run curl 'localhost:9200/_cat/indices?v' I see, that actually it's created (with different name cards_123123123). Here is log from elastic:

[2018-01-10T11:05:02,206][INFO ][o.e.c.m.MetaDataCreateIndexService] [1_mxHwD] [cards_1515578702175] creating index, cause [api], templates [], shards [5]/[1], mappings [card] [2018-01-10T11:05:02,274][INFO ][o.e.c.m.MetaDataCreateIndexService] [1_mxHwD] [cards] creating index, cause [api], templates [], shards [5]/[1], mappings [card] [2018-01-10T11:05:02,826][INFO ][o.e.c.m.MetaDataDeleteIndexService] [1_mxHwD] [cards/cZ29bNxzSAyy8ablpvgOsA] deleting index

Actually the error occurs while trying to use rake:reset If you try to run direct tasks mention here https://github.com/toptal/chewy#index-manipulation . from rails console. It works as it must.

Hope it helps.

[jduff]

It looks like some work has started to support ES6 here https://github.com/toptal/chewy/pull/622

I also just added a bit of information related to rake chewy:reset rake chewy:deploy and friends failing, relating to the error you are seeing @vishalsingh123

@pyromaniac you mentioned the only issue being that multiple types is deprecated, not removed. This is partially correct, it looks like indexes created in ES5 with multiple types will continue to work in ES6, but you cannot create new multiple type indexes in ES6. This makes getting started with this gem and ES6 challenging since the stash specification and journal are created in the same index which causes the chewy:deploy task to fail every time.

[jpaas]

Any update on ES6 support? Also thanks very much for this gem, it has become a critical component of the platform we run our business on!

[pyromaniac]

Sorry, but nothing much yet, still have a lot of stuff happening besides it. But I definitely will get back here eventually and rather sooner than later.

[breisig]

Any new updates on ES6 support?

[jduff]

@breisig with some of the patches that were merged a few months ago it has been working well for me. I’m not using multi-type indexes at all though so not sure if you might run into issues if you are using those.

[baronworks]

@jduff just wondering if you are using any joins to define relations. Currently am using the parent child and trying to find a work around so we can upgrade elasticsearch and still use Chewy.

[jduff]

I don’t have any complicated joins in the data being indexed, but I do include a bunch of associated models and have data from them in the index.

[shaneog]

I also would like to know if there's an ETA on ES6 support. I have one multi-type index which is preventing me from upgrading.

[shaneog]

We're now 7 months away from the official EOL date for Elasticsearch 5.6.x

@pyromaniac Is there an ETA on this?

[pyromaniac]

Well, I'm trying hard to find a time for this atm. So it should be kinda soonish.

[shaneog]

Thank you for the response! The work you do is very well appreciated. I look forward to seeing the changes!

[mattzollinhofer]

It sounds like some people are successfully using the current version of chewy with ES6. But this thread has gotten a little confusing. Are there still known outstanding issues for ES 6?

@pyromaniac do you think you could create issue(s) with a label of “ES 6” and if there’s any I can take a look at I could try to help?

I’m not on ES 6 yet, but will be moving there soon so would like to help if I can.

[pyromaniac]

@mattzollinhofer well, yeah, it might be a good idea. I just need to stop procrastinating and get to it.

Also, multi-index queries support just landed in master as a next step of ES6 support

[btoconnor]

I pinned my Gemfile to master on Github - not sure if this is expected, but when running chewy:reset after adding a multi-model index, I'm still getting the error about "rejected mapping":

"Rejecting mapping update to [foo_search_1544896127077] as the final mapping would have more than 1 type:

[jpaas]

Not sure if its there or I can't find it, but is there a way to associate multiple types with an index yet?

[nyddogghr]

I'm still having the issue about multiple indexes ("Rejecting mapping update to [development_search] as the final mapping would have more than 1 type) when using master. I do have 2 define_types in my same Chewy::Index class.

ES 5 has now reached its EOL (https://www.elastic.co/fr/support/eol), any update on this?

[rubenfonseca]

Is chewy abandoned? It's ok that it is, we just need some clear warning on the homepage.

[marselmustafin]

@pyromaniac first of all, thanks for the work you did, I really appreciate it.

I have a question about multi-type indexes. As it stated in ES6 docs, it's still possible to use multiple types as one type, using type field explicitly. Is there way to use same approach in chewy, taking into account that we use specific AR model on type definition?

[rabotyaga]

Hey @marselmustafin 👋 ! Chewy will not provide custom type fields for multi-type mappings in any automated way. You can use this approach on the application level, or switch to the single-typed mappings and use the multi-index queries.

[rabotyaga]

@rubenfonseca We've just released chewy 5.2.0 with a couple of fixes and added support for the ruby 3.0, and working to bring in ES6 support, and then - ES7.

[aabhushan]

@rabotyaga That is great news. Do we have any timeline around ES6 support? Even if it is an estimate that would be great, thanks for the all work!

[rabotyaga]

@aabhushan Strictly speaking, nothing stops you from using chewy 5.2 with ES6. You just have to remove any multi-type mappings and make sure you don't use any of the deprecated ES5 features. Regarding the timeline for chewy 6.0, I would be glad to give some estimates, but I'm not sure I can say something more precise, than "in a month, probably".

[abartov]

I can confirm current Chewy works fine with ES 6.x. (I use 6.4.1 in production.)

[aabhushan]

That is great news, @rabotyaga @abartov, upgrading ES to 6 and wanted to make sure we won't face any issues. Thanks!!

[rabotyaga]

Chewy 6.0.0 has just been released.