search

qsniyg / maxurl

Finds larger/original versions of images and videos
https://qsniyg.github.io/maxurl/
Apache License 2.0
1.08k stars 66 forks source link

Discogs again #689

Open PatPanacea opened 3 years ago

PatPanacea commented 3 years ago

Sample: https://img.discogs.com/ndX3ORCM48WofqvPv9Qh3Ynsdp0=/fit-in/600x595/filters:strip_icc():format(jpeg):mode_rgb():quality(90)/discogs-images/R-15025166-1613956799-5152.png.jpg

qsniyg commented 3 years ago

Well, they've now removed the method, old urls don't work anymore either (all 404). Now we're back to square one... I'll do some research later, and if you find anything at all, please let me know!

(Edit: I'm adding the "impossible?" tag pre-emptively until a solution can be found. It's very possible they haven't exposed any endpoint to retrieve the original images)

PatPanacea commented 3 years ago

Thanks so much for your attention and your quick response. I tried a number of URL combinations and got nothing.

qsniyg commented 3 years ago

I've added a few new ways of getting larger images, but it still doesn't get the source, it will just get the "full" (600x) version from a thumbnail. Better than nothing I guess, but still not great.

mjb2010 commented 3 years ago

Relevant discussion on Discogs: https://www.discogs.com/forum/thread/828335#8567194

FWIW, it was mentioned several years prior that the libraries they use are called Pillow and Thumbor.

redactedscribe commented 2 years ago

I've added a few new ways of getting larger images, but it still doesn't get the source, it will just get the "full" (600x) version from a thumbnail. Better than nothing I guess, but still not great.

@qsniyg When viewing a Discogs image in a new tab, the page quickly refreshes and upgrades the URL to the "full" version, but supposedly better image URL is worse than the original. Sometimes the upgraded URL is slightly smaller, say 599 instead of 600px (but this may be expected if 599px is the original width and Discogs is stretching the image to 600?).

I think the lower quality is because the new URL uses :format(webp): in some cases instead of :format(jpeg): (which has more accurate/saturated colours, and a larger filesize). Pressing Back shows the better, jpeg version in this case.

Quality example jpeg->webp example (main cover image: reds clearer and greens sharper in jpeg) Smaller dimensions example (artist image: upgraded is 599px)

Firefox 95 IMU 0.19.5

ROpdebee commented 2 years ago

FWIW Discogs seems to have changed the URL format again, now it's e.g. https://i.discogs.com/aRe2RbRXu0g4PvRjrPgQKb_YmFWO3Y0CYc098S8Q1go/rs:fit/g:sm/q:90/h:600/w:576/czM6Ly9kaXNjb2dz/LWltYWdlcy9SLTk4/OTI5MTItMTU3OTQ1/NjcwNy0yMzIwLmpw/ZWc.jpeg (from here). The webp and jpeg URLs are now the same in their API responses. The URL doesn't seem to include the release ID any longer so maximising a direct image URL no longer works. The real file name is present in the Content-Disposition header of the HTTP response, though, so that could be used instead. Edit: Or just base64-decode the parts after the width (w:) param, excluding slashes. Playing with the size parameters just leads to a 403.

Edited to add: Judging by the new URL format, they probably have switched to using imgproxy over Thumbor. Compare the URLs in the examples below to the one at the bottom of imgproxy's documentation. In practice, this doesn't really affect the viability of grabbing full-size images through this endpoint, since both services use URL signatures (the seemingly random gibberish at the start of the path) to prevent tampering with the parameters. imgproxy uses HMAC-SHA-256 vs. Thumbor's HMAC-SHA-1 though, so there's even more entropy now. They're also using AWS S3 as a storage backend (not new information). It might be worth noting that the old Thumbor URLs currently still work and are redirecting to the imgproxy ones.

TL;DR: Discogs really doesn't want anyone to be able to grab full-size images.

Some examples:

qsniyg commented 1 year ago

Another interesting piece of information, using one of @ROpdebee's links as an example:

https://i.discogs.com/_xL4yC-gjc-awYVWmO4dDmOv-Za3oICJweuYqpnEzwk/rs:fit/g:sm/q:40/h:300/w:300/czM6Ly9kaXNjb2dz/LWltYWdlcy9SLTk4/OTI5MTItMTU3OTQ1/NjcwNy0yMzIwLmpw/ZWc.jpeg

> "czM6Ly9kaXNjb2dz/LWltYWdlcy9SLTk4/OTI5MTItMTU3OTQ1/NjcwNy0yMzIwLmpw/ZWc".split("/").map(x=>atob(x)).join("") 's3://discogs-images/R-9892912-1579456707-2320.jpeg'

Another image:

https://i.discogs.com/GxhlKZ2FzWY_oT8irrZ8AlA-jQXnnHYefaxL6KKffT4/rs:fit/g:sm/q:40/h:300/w:300/czM6Ly9kaXNjb2dz/LWRhdGFiYXNlLWlt/YWdlcy9SLTIwMTcz/NDM4LTE2NTUyNjY3/NDgtMzUzMy5qcGVn.jpeg

Returns this:

s3://discogs-database-images/R-20173438-1655266748-3533.jpeg

discogs-database-images is a valid S3 bucket (discogs-database-images.s3.amazonaws.com).

virtualcrane commented 10 months ago

is there any update on a fix for this?

qsniyg commented 10 months ago

@virtualcrane None so far, sorry.