search

ibrahimceyisakar / youtube-sentiment-analysis

An application for sentiment analysis of YouTube comments, the backend api core is writen in FlaskRESTfull, and the frontend client is in ReactJS
MIT License
0 stars 0 forks source link

Add sorting and limiting to comments list (or check if we can scrape already sorted) #7

Open ibrahimceyisakar opened 1 year ago

ibrahimceyisakar commented 1 year ago

We may not have to do sentiment analysis for all the available video comments.
The most time taking part in the application is analyzing scraped comments sentiment.

So to make the whole process faster (and cheaper), we can follow this approach:

  1. Scrape all available comments of the video
  2. Order all comments by like_count and take top N comments based on like_count

If we can get/scrape comments already sorted by likes, then it's even easier (by adding an 'order_by' param to official or 3rd party API request), so first check if official or 3rd party API can give us comments already sorted by likes, and add limiting to it.

Note: YouTube actually allows sorting comments based on like count (on web browser and on mobile app) So probably we can get comments already sorted from the APIs