81 elasticsearch

[mooreniemi]

This branch is for switching off Thinking Sphinx for AWS Elasticsearch. This should save us on budget (need to confirm exact numbers) and improve the search experience. It also gets us off a troublesome and unnecessary mysql dependency with native extensions.

Some things I need to still do:

• [x] mock ES callbacks in test
• [ ] unit tests
• [x] staging integration, prod integration (config vars first, and need to see if this will build on heroku-16)
• [ ] adjust the query

[mooreniemi]

It's on staging now, tried submitting a pin and hit this error not formatting so it word wraps...

Feb 21 15:07:44 transbucket-staging app/web.1 method=POST path=/pins format=json controller=PinsController action=create status=500 
error='Elasticsearch::Transport::Transport::Errors::BadRequest: [400] 
{"error":
{"root_cause":[{"type":"mapper_parsing_exception","reason":"failed to parse field [complications] of type [text] in document with id '28'. Preview of field's value: '{name=pain, id=3, taggings_count=1}'"}],"type":"mapper_parsing_exception","reason":"failed to parse field [complications] of type [text] in document with id '28'.
 Preview of field's value: '{name=pain, id=3, taggings_count=1}'","caused_by":{"type":"illegal_state_exception","reason":"Can't get text on a START_OBJECT at 1:703"}},"status":400}' duration=3496.78 view=0.00 db=33.63
Feb 21 15:07:44 transbucket-staging app/web.1 Elasticsearch::Transport::Transport::Errors::BadRequest ([400] 
{"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"failed to parse field [complications] of type [text] in document with id '28'. Preview of field's value: '{name=pain, id=3, taggings_count=1}'"}],"type":"mapper_parsing_exception","reason":"failed to parse field [complications] of type [text] in document with id '28'. Preview of field's value: '{name=pain, id=3, taggings_count=1}'","caused_by":{"type":"illegal_state_exception","reason":"Can't get text on a START_OBJECT at 1:703"}},"status":400}):

[mooreniemi]

I think I fixed the above, I see errors on staging but looks like just due to timeout. I am seeing success locally and it does actually work on staging, too.

[mooreniemi]

Right now the query is set OR style but we probably want to make the search more open to input. I think this is a good first step though.

[mooreniemi]

Another problem: Prod and Staging are pointing at the same ES index, made index name dynamic.

[mooreniemi]

I'm going to put this branch up on Prod and let it run a few days. If it looks ok, I'll merge it to master and remove TS.

[mooreniemi]

Ran heroku rake environment elasticsearch:import:model CLASS='Pin' INCLUDE='PinImage,Surgeon,Procedure' FORCE=true --app transbucket, succeeded after I fixed a couple Pins with null procedure_ids.

So far so good...

[mooreniemi]

I'm turning on error logs on AWS for now. I will probably remove them in a few days after things look safe.