search

stanfordnlp / treelstm

Tree-structured Long Short-Term Memory networks (http://arxiv.org/abs/1503.00075)
GNU General Public License v2.0
875 stars 236 forks source link

Using the trained model for new relatedness score ? #2

Closed ganarajpr closed 9 years ago

ganarajpr commented 9 years ago

As per the instructions in the readme I ran the relatedness code and got it to run for 10 epocs - which produced a new trained_models folder containing the "trainedmodel". I am not really sure how I can use this trained model to use with my own two sentences to find a relatedness score between them ?

kaishengtai commented 9 years ago

Currently, there isn't a single convenient script for preprocessing and predicting on new sentence pairs (that said, I plan to add the functionality soon).

In order to predict on a sentence pair, you should call the function TreeLSTMSim:predict(ltree, rtree, lsent, rsent) (see https://github.com/stanfordnlp/treelstm/blob/master/relatedness/TreeLSTMSim.lua#L142). ltree and rtree are dependency parse trees of the 'left' and 'right' sentences respectively (see the preprocessing script https://github.com/stanfordnlp/treelstm/blob/master/scripts/preprocess-sick.py for how these parses can be produced). lsent and rsent are 1D Torch tensors consisting of indices into the vocabulary. Since your dataset will have a different vocabulary, you'll probably want to change this function to directly accept NxD tensors of word vectors (where N = sentence length and D = word vector dim).

So basically it should be pretty straightforward:

  1. Produce dependency parses of your sentences
  2. Perhaps make slight modifications to TreeLSTMSim:predict()
  3. Load a trained model and predict