Elasticsearch and Denormalization in Rails

[nickflux]

@davetron5000 your thoughts are welcome (as are anyone elses) The two purposes of this post are to: a) show some useful code b) briefly mention some real world problems we are working on

[davetron5000]

Great post! This is just the sort of quick and easy thing that demonstrates what problems we are solving and how we go about them.

A general comment would be to remove the discussion of the receiving queue—I think you can just start with the sample queue as your basic "problem to solve" and go from there.

I'd also try to craft the first paragraph to show what the conclusion is, like:

Complex relational databases can lead to tortuous SQL queries and slow responses from your web application. If you're trying to return a long list of objects that are built up from five, ten, or even 17 related tables your response times can be unacceptably slow. At Stitch Fix, we run into this frequently, and have found that using ElasticSearch, along with some conventions around de-normalizing a complex we of objects, has allowed us to easily create snappy views for our users. [or whatever]

(also removed the bit about Heroku purposefully—you had me at "unacceptably slow").

Another thing to think about doing is to walk the reader through the code a bit more explicitly. I'd start by showing the ActiveRecord object and talking about how to_hash is what Elasticsearch::Model is going to use to decide what gets indexed. That's a good place to show off our non-hack coding skills because you can say "instead of dumping everything into the model, we defer to a denormalizer to do the work, separation of concerns etc. etc.".

Then, I'd start with a version of the denormalizer that just does simple attributes, let that sink in, and then show the stuff about the complex boolean derived value, e.g. "of course, you don't have to just put delegated methods into the hash, you can calculate complex derived values, too"

This leads naturally to where you discuss the problem of having to deal with presenting an ActiveRecord object vs. the thing-that-comes-out-of-ElasticSearch.

I'm gonna add a few minor comments in the diff itself.

[nickflux]

Thanks @davetron5000 . I will incorporate all these great suggestions.

[nickflux]

@davetron5000 see updated version

[davetron5000]

This is great! Reads really well, and brings the reader to the conclusion very nicely. I've got a few small comments I'll drop in, but this is a great post.

[nickflux]

Thanks again @davetron5000. Changes made and date of the post set for tomorrow.