Explore different tools/algorithms to auto-generate tags

sacmax commented 6 years ago

Explore methodologies/tools/algorithms to generate tags based on the context of the story. Please come up with a small write-up covering the process and necessary links and do push it to the repository. Please add relevant examples wherever possible. If you create a script/notebook, do push it to the repository.

deepakshankar94 commented 6 years ago

@arnabbiswas1 I can work on this

arnabbiswas1 commented 6 years ago

@deepakshankar94 Here you go.

githubssn commented 6 years ago

i would like to contribute to this as well..

arnabbiswas1 commented 6 years ago

@githubssn Done. 👍

githubssn commented 6 years ago

Hello Arnab,

I went through the data to do some data exploration and had a few suggestions:-

I think that we do need some more data to go about the analysis. Data of about 300 stories and their content would be required to start data analysis as a prototype. Currently we have content of only a few stories in the story pages data. Could we get data of story category, pages and tags for about 300 stories? Could you share the complete data of the story category, content and tags for the stories? <Currently we have data of only 4 stories in the story-pages csv file while story-tags has data of 50 stories. Essentially we dont have all the attributes of the 50 stories.>
Upon browsing the website, I find that there is a lot of languages. Perhaps we could do the initial analysis on 300 English stories before we go to other languages. Please ignore if this is already as per initial plan.
I felt that the data could be normalized better. Eg. some features like language, organization_name, story title, story english title, story status, story summary, etc could be associated with story-category/tags rather than story page so they do not repeat. Not sure if it is possible to change that at this stage.
In terms of overall approach, we can do Topic Modeling for generating tags.

Do we plan to connect this weekend? Let me know.

regards,

Sneha

From: Arnab Biswas notifications@github.com Sent: Saturday, June 16, 2018 10:51 AM To: DataKind-BLR/PrathamBooks-Sprint-2018 Cc: githubssn; Assign Subject: Re: [DataKind-BLR/PrathamBooks-Sprint-2018] Explore different tools/algorithms to auto-generate tags (#7)

Assigned #7https://github.com/DataKind-BLR/PrathamBooks-Sprint-2018/issues/7 to @githubssnhttps://github.com/githubssn.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHubhttps://github.com/DataKind-BLR/PrathamBooks-Sprint-2018/issues/7#event-1684540587, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AktDypgEpMvPxkhlSo78vIUHleFHUFtwks5t9JXhgaJpZM4Up7Xc.

arnabbiswas1 commented 6 years ago

Reopening the issue as it was closed by mistake.

arnabbiswas1 commented 6 years ago

@githubssn @deepakshankar94 Would you mind creating new issues with each possible approach which you feel is relevant?

arnabbiswas1 commented 6 years ago

The following wiki has been created to track the initial set of methodologies to be evaluated: https://github.com/DataKind-BLR/PrathamBooks-Sprint-2018/wiki/Methodologies-to-Evaluate

Separate issues will be raised as we explore individual issues. Hence closing.

DataKind-BLR / PrathamBooks-Sprint-2018

Explore different tools/algorithms to auto-generate tags #7