Add elasticsearch-dsl querying.

mitodl / lore

Learning Object Repository for Education

https://goo.gl/nDVx4D

GNU Affero General Public License v3.0

16 stars 6 forks source link

Add elasticsearch-dsl querying. #668

Open ShawnMilo opened 9 years ago

ShawnMilo commented 9 years ago

Based on the research done in #516, add direct use of the elasticsearch and elasticsearch_dsl libraries.

These will temporarily run in parallel to the Haystack code, until the front-end and restful API are updated, then all the Haystack code can be completed.

[x] Duplicate existing Haystack functionality with above libraries #669
[x] Add converter to transform existing requests from what Haystack expects to what Elasticsearch-DSL uses, and a converter to transform the results of an Elasticsearch-DSL query to what the existing front-end/API expects. #680
[ ] Update front-end (React) code to use the new functions
[ ] Update the restful API to use the new functions

ShawnMilo commented 9 years ago

FYI @pdpinch. #669 is the part I'm working on now (first checkbox above).

Ferdi commented 9 years ago

Update front-end (React) code to use the new functions

I thought react only uses the REST api ?

noisecapella commented 9 years ago

He's referring to the change in output from the REST API which will require slight changes to the React code that consumes it.

pdpinch commented 9 years ago

@ShawnMilo I think the 3rd and 4th checkboxes are complete on this, and we can close this issue. Can you confirm?

ShawnMilo commented 9 years ago

@pdpinch Not really. The data being pulled by elasticsearch-dsl is being converted to what Haystack had been returning. The actual learning resources aren't really different, since they're essentially being returned as a list of dictionaries.

However, the aggregate data (vocabulary counts) are being converted to the Haystack format by the convert_aggregate function in search/utils.py.

It would be nice to clean up, but I don't think it's really hurting anything to leave it as-is. @noisecapella or @giocalitri can comment on how much trouble it would be to clean up the react.js parts so we can scrap convert_aggregate.