search

SurgicalSteel / FlickrScrappr

Simple Flickr Scrapping Tool
MIT License
2 stars 0 forks source link

Known BIG Issues #2

Open SurgicalSteel opened 6 years ago

SurgicalSteel commented 6 years ago

This FlickrScrappr only downloads about a quarter of all photostream, like, WHAT THE HELL?

Unfortunately, our methods of getting image ID on each page of photostream does not guarantee that it will get all the image IDs on a certain page. This because Flickr defer the rendering process of showing hidden images on each page. However, on each rendering of those images which are being deferred, there is a request to Flickr server to get image ID and the small size image URL.

What to do next is to replicate that deferred rendering request so that we will have a complete list of image IDs on each page of the photostream.

Again, it is open for contribution.

SurgicalSteel commented 6 years ago

Well, seems like the deferred request cannot be replicated as it might be a server side rendering (or something I don't really know). When I visited someone's photostream, I used developer tools on my browser to view the network activity in it, the response does not show any imageID. And the request is too cryptic (it was too hard to find the pattern) :disappointed:

I tried my best though