susiejojo
commented
3 years ago Expected pipeline for YouTube:
-
[ ] allow user to enter YouTube video link
-
[ ] Scrape the link using Selenium and geckobrowser(for Firefox, Chrome I'm not very sure)
-
[ ] Load the comments for the video and the commenter names
-
[ ] no need to load likes and replies
-
[ ] save to a .csv file with only the comment text.
Resources:
- https://towardsdatascience.com/how-to-scrape-youtube-comments-with-python-61ff197115d4
- https://github.com/dddat1017/Scraping-Youtube-Comments (tried this, not working atm, but worth a shot)
- https://medium.com/analytics-vidhya/extracting-youtube-comments-using-selenium-b29ee4f743ef
- https://python-youtube.readthedocs.io/en/latest/pyyoutube.html
We need to fetch the user's top n(TBD) posts from various social media platforms.
[x] Learn about the Twitter API (as the dataset is scraped from Twitter)
[x] be able to send the response from Twitter API to the Flask server
[x] Clearly Facebook has limitations. So we will look into YouTube and perform personality predictions for commenters using comments.