Instagram Comments Scrapper

tattle-made / data-experiments

EDA, small data science projects and other experiments

3 stars 4 forks source link

Instagram Comments Scrapper #5

Closed aatmanvaidya closed 1 month ago

aatmanvaidya commented 1 month ago

[ ] Reverse engineer the API
[x] Scrape using Selenium

Tentative Deadline : July 31 -> August 7th

aatmanvaidya commented 1 month ago

How is the Facebook scrapper working?

fetching comments using XPATH

aatmanvaidya commented 1 month ago

the scrapper is up and running now, I have test it out on different reels and posts, we will actually be able to stress it well, when we start scrapping, then only we can make it more stable

two things left to do are

[ ] find out when to stop scrolling (pgdown in the scroll div)
[ ] extract comments which are replies to a comment.