search

chuanenlin / shutterscrape

Web scrapper for Shutterstock
MIT License
146 stars 51 forks source link

Lenght of image container #5

Open aravind93raja opened 5 years ago

aravind93raja commented 5 years ago

img_container = scraper.find_all("div", {"class":"z_c_b"})

img_container value gets stored as 1 .

So not able to retrieve all images in the page.

how do I solve this ?

chuanenlin commented 5 years ago

@aravind93raja Looks like shutterstock updated their page layout again - I've updated the script. Let me know if it works now!

aravind93raja commented 5 years ago

@aravind93raja Looks like shutterstock updated their page layout again - I've updated the script. Let me know if it works now

Yes ,It works fine now .Thanks.

Out of interest , How do I know what parameters to change based on the website? ( Like If I want to write a similar code to Download from google images or someother website?)

chuanenlin commented 5 years ago

Great to hear. You can check the DOM structure of the site (in Chrome, you can use inspect element). In a nutshell, the scraper scans individual elements/element arrays in decreasing orders of hierarchy until it finds the "img" tag and obtains the "src" attribute of the tag which is the link to the image.

stanistolberg commented 3 years ago

@chuanenlin Hello! Why the scraper scrapes only 20 images per page while Shutterstock shows 150? Thanks