vdpappu
commented
5 years ago Siamese networks need supervised samples and generating semi-supervised samples through semi-supervised approach is not feasible. An alternative approach is to calculate the similarity between noun-phrases and/or useful verbs in the sentences reduces noise by not considering the contributions from filler words.
Initial approach: https://github.com/etherlabsio/hinton/tree/experiment/sentence_relatedness/sentence_relatedness
Few issues to address:
- Currently, the maximum similarity between cross-sentence noun phrases is considered as relatedness score - magnifies the similarity between two irrelevant tokens. This can be improved through mean pooling and/or phrase ranking
- Exploring features from intermediate layers where the tokens attend to themselves instead of
/ tokens could give more reliable scores
shashankpr
Currently, we use Cosine Similarity for similarity metric. With complex architectures like BERT, it may not be effective as the objective functions used for pre-training or fine-tuning does not directly reflect sentence relatedness without labelled dataset. Siamese Network (https://www.youtube.com/watch?v=6jfw8MuKwpI) provides an effective alternative for similarity tasks. This involves training a small network whose outputs highlights the similarity/dissimilarity between two inputs.