Search and Annotation Refinement

[heobu]

Improvements on the search and annotation methodology.

Search terms, the words within each search term, main query, all of them might need to be searched in a way that will allow for some suffixes, different combination of words and so on for better search results.

But at the same time, it is important not to worsen the search results. For example, splitting search terms unnecessarily as in the case of "left amygdala" might be an issue since we are not just looking for any occurrence of the word "left" either.

NLTK could be used for tokenization of the article content and search terms.

System Requirement 1.3.

[heobu]

• NLTK for search keyword tokenization

• Order by:

  • page hit count
    • score by how many matches in annotations & elastic search

[heobu]

"left amygdala"

• split the search term into words (tokenization)
• search for each word in the annotations
• if the word is found, it is included in the search results for that combination, with lower priority than the full "left amygdala" match.

The expectation is that since the word "left" is not supposed to be in annotations, only some other amygdala related results will be found with this method.

[8hk]

related with #103

[8hk]

closing since #103 closed