DOsinga / deep_learning_cookbook

Deep Learning Cookbox
Apache License 2.0
688 stars 337 forks source link

Is there a way to predict body from title (example 06.1 - Question Matching) #36

Closed hedgy123 closed 5 years ago

hedgy123 commented 5 years ago

Hello,

In example 06.1 (Question matching), the sim_model calculates the similarity between two embeddings - those of the title and those of the body. If I'd like to predict the closest matching body given a title, is there a better way to do this other than loop over every possible combination of [title, body], extract the corresponding similarity, then output the body corresponding to the highest similarity? For large datasets such an approach would be far too slow.

I'd really appreciate your help.
DOsinga commented 5 years ago

Hi,

You should be able to get this done similarly to how the inverse image search works using Postgres or you can use Scikit-Learn's k-nearest neighbors if you want to do this in memory.

Hope this helps!