Clueless-Community / scrape-up

A web-scraping-based python package that enables you to scrape data from various platforms like GitHub, Twitter, Instagram, or any useful website.
https://pypi.org/project/scrape-up/
MIT License
250 stars 243 forks source link

Post count for Instagram #216

Closed karthikbhandary2 closed 1 year ago

karthikbhandary2 commented 1 year ago

Is your feature request related to a problem? Please describe. Currently, the Instagram scrap_up only has two methods to get the followers' count and the following count. I would like to add a new method to get the post count.

Describe the solution you'd like I will be using beautiful soup to do this.

BabarRasheed commented 1 year ago

Hi, I'm Babar Rasheed (Contributor GSSOC'23) Many websites don't offer API so to tackle this we can use Web Scraping to access data in easy and structured manner. Python libraries like bs4, BeautifulSoup, Scrapy, Selenium, etc. are generally used for web scraping. Here I'm willing to apply these libraries and use an effective way of Multiprocessing to speed up Web Scraping. Multiprocessing is very helpful when multiple URLs are scraped to get the data. It will perform scraping on multiple URLs thus saving our time.

ankitanain commented 1 year ago

Approach:

  1. Utilize the Instagram API by making a request to the users/{user_id} endpoint and extracting the media_count field from the response.
  2. Alternatively, use web scraping techniques with a library like BeautifulSoup in Python to parse the HTML of the account's profile page and extract the post count from the corresponding HTML element. @nikhil25803 Sir, please assign this issue to me under GSSOC'23.
nikhil25803 commented 1 year ago

Go ahead @karthikbhandary2

karthikbhandary2 commented 1 year ago

There is an error in the following() method. I am following 560 accounts on Instagram but it is showing it as 108. error

karthikbhandary2 commented 1 year ago

I've observed that the meta tag is not showing the following count. I can change the code so as to show a msg saying "Following count not available" or should I do something else? @nikhil25803 following error