Closed danrademacher closed 6 years ago
Item one: development of scrapers is no longer possible
https://www.instagram.com/developer/
As such, further scraping of photos is no longer feasible, even if interest and funding existed.
A previous version of the site featured Flickr photos as well (before we assumed control of the code and data). I wonder if those old Flickr photos are still present in the database and might represent an alternative source of images. As far as I know, Flickr's API remains usable for things like harvesting, though the platform is much less widely used than IG, so hard to know how much new content one would get there.
Could you take a look in the dbase and see if there's a table of flickr images tied to CPAD ids?
CDN URL changes
It would appear that Instagram has either restricted usage of their CDN and/or changed the URLs of prior photos as well, as all photos are now giving an "Access denied" message.
This does not use any API, merely requesting previously-public photos via their CDN URL.
Perusing some Instagram user pages today and using my debugger on the photos, it seems that the URL formats being used today are indeed quite different.
A Specific Test Case
One test case, a photo previously linked to Golden Gate Park (Superunit ID 10900)
Photo URL with frame: http://instagram.com/p/wZJPWIuC7g/
Old URLs, not working: http://scontent-a.cdninstagram.com/hphotos-xaf1/t51.2885-15/10848105_355658264614724_1128453539_a.jpg https://1gu2c4mrsgexdinbogq000000.igsonar.com/hphotos-ak-xaf1/10848105_355658264614724_1128453539_s.jpg http://scontent-a.cdninstagram.com/hphotos-xaf1/t51.2885-15/10848105_355658264614724_1128453539_n.jpg
Looking Forward
In theory, one could go over all photos in the database and:
Challenges may include:
Feasibility of reprogramming to show Flickr photos instead of Instagram
Summary
DB table: flickr_photos
Front Page Photos
featuredParks
array; each such item has standard_resolution
for the image SRC URL and su_name
as the ALT textPhotos for a Specific Park
fetchSelectedParkPhotos()
Other Usages of instagram_photos TBD
parksNotIn query, app/server/services/queries/parks.js
mostSharedParks
output against some specified park IDsrandomPark query, app/server/services/queries/parks.js
Flickr photos looking good. Will open separate issue for FAQ update
https://techcrunch.com/2018/04/02/instagram-api-limit/ details major changes to the IG API that came unannounced earlier this week and have clobbered the ability of CaliParks to load most IG photos.
Let's post findings here about the new limitations and their implications for the site. This is not yet a "fix it" task since we're not sure it can be fixed.