Improve performance of searching algorithms

[rodedzats]

Check the performance of the current searching of relevant posts and improve it

[yossigil]

Please checkout minimal hashing. I am not sure it is relevant to stack traces, whic might be too small for the technique to be effective.

[yonzarecki]

After looking at the complete code, the process that goes through getting a query is:

• Calling STRetriever's getMostRelevantPosts
  • calls DBConnector to get all posts with the same exception type
  • calls getMostRelevantStackTraces with returned posts
    • getMostRelevantStackTraces sorts all posts and returns top K
    • returns output
  • returns output

As we can see there are 2 components that might take time:

• sorting of all posts
• retrieval of posts from the DB.

Tony is working on optimizing the DB, and what we can optimize is the sorting.

We think the best option is to use PriorityQueues and get top K elements. We'll continue to work on this optimization in the following days.

[yonzarecki]

We've updated the code to use heap-based top k access. And as I stated before, We can't see any more significant runtime optimization beyond faster database access at the time. So I'll be closing this issue.