search

akshayarora2009 / InpediaSearchService

The "most important" backend to facilitate search to clients
0 stars 0 forks source link

Relevant Search #1

Open jatinarora2409 opened 8 years ago

jatinarora2409 commented 8 years ago

SCORE CALCULATION In a search, one of the most important part is RELEVANT RESULTS. As studied and experimented, elasticsearch gives search results along with some scores which are proportional to the matching with the search query.But many times a client wants results which arent only based on the query but are also based on some relevance of the documents. For eg. As discussed, youtube when we search sees views,time of upload and user uploading the video as a basis of ordering the search results. Options to tackle the problem.

  1. We can build our own mechanism which will use the value of some fields specified by the user. PROS -
    • We can built the mechanism the way we want and can be user friendly.
    • Integration of more algorthims and using new technology will be easier. CONS -
    • It will be slow as our data will be first processed by elasticsearch parallely and then by our script on a single server, creating a bottleneck for search.
    • It will be less reliable.( If we do it ourselves then we wont be able to test it at a very high scale).
  2. Use the mechanism provided by the ELASTIC SEARCH. PROS –
    • It will be quick as most of calculation will be done on different servers and results will be managed by elasticsearch.
    • It is reliable as it has been tested and used widely in the world. CONS -
    • We will be dependent on elasticsearch for most the development in this area. I think that we should go with the later one for the search. Calculation of score again by us in any case , is going to be a slow task, until and unless we find a way to preprocess data while storing it again limiting the query domain as no new inputs could be passed in query.MECHANISM IN ELASTICSEARCH - (PLEASE REFER TO ELASTIC SEARCH BOOK FOR BETTER LEARNING) Many Mechanisms are provided in elasticsearch -
  3. Index-Time Field-Level Boosting
  4. Query-Time Boosting
  5. Boosting an Index In boosting , we can boost a field that if words matched from the given field scored will be boosted.
  6. Function_score Query a) field_value_factor b) script_score
akshayarora2009 commented 8 years ago

That sounds fair enough. We can, at the moment, go ahead with the features provided to us by Elasticsearch.

Even more features can be integrated or discussed later in the project.