Article Similarity

[KshitijKarthick]

Application of T-vecs model

Provide a means to compare 2 or more articles using our T-vecs model based on semantic similarity across languages.

[KshitijKarthick]

Naive Algorithm for measuring semantic similarity b/w articles

Commit 6ea8d45f39633ffd949ad1980629dee23fede816 employs an algorithm to compute article semantic similarity in the following steps

• Obtain Tf-Idf for collection of Documents from set 1
• Obtain Tf-Idf for collections of Documents from set 2
• Obtain semantic similarity between one word (from an article of set 1) to every other important word for an article from set 2.
• Average out the cosine similarity between two articles

[KshitijKarthick]

Centroid Algorithm for measuring semantic similarity b/w articles

Commit 0c562ce3244d0119b79735175fea807dc9df52a3 employs an algorithm to compute semantic similarity in the following steps

• Obtain Tf-Idf for collection of Documents from set 1
• Obtain Tf-Idf for collections of Documents from set 2
• Obtain Centroid c1 from Tf-Idf for collection of Documents from set 1 [ mean ]
• Obtain Centroid c2 from Tf-Idf for collection of Documents from set 2 [ mean ]
• Obtain semantic similarity between c1 and c2 ( cosine similarity )

Correct me if i am wrong anywhere @upman :smile:

[KshitijKarthick]

Results for centroid algorithm from commit cbf2f91f618baaa8e4410a180b4981f932fe463d :

• centroid_cosine.txt
• centroid_euclidean.txt

Closing this issue, as we obtained unsatisfactory results