search

iamatulsingh / pinterest-image-scrap

This python (3.7) program scrap data from pinterest without official API.
MIT License
62 stars 11 forks source link

not fetching more pins/less images than expected #7

Open kqee opened 1 year ago

kqee commented 1 year ago

to reproduce:

$~ python pinterest.py
Enter keyword: pink trees # we use this keyword as an example
[+] starting search ...
[+] saving results ...
[+] saving json data ...

upon finishing fetching all links and downloading the content, only 99 images are found.

what is expected

it should be able to fetch more than 99 links as well as downloading them, because when I did a google search I found not hundreds but thousands images of pink trees. Is this behaviour expected?

iamatulsingh commented 1 year ago

I tried same way and I got 113 images. It's true that there are lot's of images there. I'll check if I get sometime. If you can, please make some changes and open PR.

iamatulsingh commented 1 year ago

I looked at it just now. I forgot to tell you that Pinterest is a dynamic website, that means it loads images on scrolling only after the first few images (lazy loading). For scraping all the images, I'll need to completely change the code with Selenium to do and it will consume more RAM and resources for doing that. May be that could be the next library but not this I think.

qwertttyyy commented 3 months ago

Good afternoon! Have you been working on developing a version of the scraper on Selenium library to get more pix? Need such a version. I may have to make it myself.

iamatulsingh commented 2 months ago

Hi @qwertttyyy, sorry for late reply. I am not working on using selenium to get more images. But that would be nice to have. You can contribute with your code if it's working.