Use a pretrained model released by a paper in our lit review

[violetguos]

• TBD, we don't really know if this is gonna work since research ppl's code are NEVER reliable...
• It's a BI-LSTM model for sentence embedding, reference: https://github.com/hamidkarimi/HDSF

If we have time:

• Preprocess their look up table for the word embedding, I don't think they stemmed their words
• Then hopefully, their pytorch code can actually work and at least won't take us too long

[violetguos]

I had a look at the author's repo. The data set they use has this kind of corpus structure. Instead of

words words words...[punctuations] [symbols] other words

They built a hash table of index -> word, so all the corpora have the form

[integer index] [integer index for another word] etc etc

In order to reuse their code (even just the LSTM definition in torch), we need to completely process our data into the same form as that one. I don't think it's worth investing that much time and effort.