Selection and summarization for Milestones

[arvindpdmn]

Given an article title and its associated references (manually collected), identify how the topic has progressed through time. Some approaches are:

• NER for date
• Pick date from the date of publication of important papers in the field
• Text summarization

Note:

• See Milestones section of this page: https://devopedia.org/site-map/author-guidelines?good-bad-examples
• Still images are allowed in this section and limited to one per milestone.
• We impose a limit of 15 milestones with a maximum of 100 words to describe each one.
• Don't use lists within milestone descriptions.

[arvindpdmn]

Does the below analogy with topic distribution make sense?

Now the dates have to be treated similar to words and we have to determine topic distribution of the corpus. Here corpus refers to the collection of references for a given devopedia article.

Individual references might be actually be having its own topic distribution but what we are interested in is the corpus distribution. Somewhat different to the topic distribution which can essentially work on the tfidf, the years_delta plays also a role here.

[arvindpdmn]

Work in progress:

• Milestone summarisation file - https://docs.google.com/document/d/1cd82YbeZRf4JVocQh5HPEzHD-AKxj1pczumXiXsRz58/edit?usp=sharing
• Milestone - reprocess https://colab.research.google.com/gist/rachanaramanna/18da56f41322fb4d8271b94ab36e7966/milestone_preproc.ipynb
• Date parser - https://colab.research.google.com/drive/1q-40NnvrjUjDugMINsFnbbAAEQebx4XF?usp=sharing
• Milestone summarizer - https://colab.research.google.com/drive/1q-40NnvrjUjDugMINsFnbbAAEQebx4XF?usp=sharing