Site support request: gumroad.com

Gelob commented 4 years ago

Checklist

[X] I'm reporting a new site support request
[X] I've verified that I'm running youtube-dl version 2020.09.20
[X] I've checked that all provided URLs are alive and playable in a browser
[X] I've checked that none of provided URLs violate any copyrights
[X] I've searched the bugtracker for similar site support requests including closed ones

Example URLs

Single video: https://gumroad.com/s/59bd0a45c6b12585318c698c03e4752d/kSlnXCva3zW8Ryu-u5hT4g==

Description

In this example it is 1 single mp4 file if I inspect in my browser. In other cases it the file loads in chunked .ts and if I inspect browser I see a network request for something like https://gumroad.com/r/59bd0a45c6b12585318c698c03e4752d/kSlnXCva3zW8Ryu-u5hT4g==/index.m3u8 (FYI this will 404 on this specific video but other real videos work this way).

If i pass the m3u8 URL to youtube-dl it downloads the video fine, but I'd like to avoid having to inspect element and pass the URL like above.

Nezteb commented 2 years ago

I would also love Gumroad to be supported!

Nezteb commented 2 years ago

So, I figured out how to do this without "official"/"direct" support.

Gumroad uses m3u8/ts file streams. You can find the specific stream in the network tab of your browser's dev tools. Grab the url ending in .m3u8 (you can find the full url in the request headers), run youtube-dl -F <url>, pick a format, download it, and it should download as a .mp4 file.

I used this successfully on several difference videos in order to archive the content for offline viewing (not distribution to others).

aghwotu commented 1 year ago

So, I figured out how to do this without "official"/"direct" support.

Gumroad uses m3u8/ts file streams. You can find the specific stream in the network tab of your browser's dev tools. Grab the url ending in .m3u8 (you can find the full url in the request headers), run youtube-dl -F <url>, pick a format, download it, and it should download as a .mp4 file.

I used this successfully on several difference videos in order to archive the content for offline viewing (not distribution to others).

Hello, there @Nezteb. How do you 'pick a format'? After running this command youtube-dl -F <url>, I got an error and then this error message:

ERROR: Unsupported URL: https://app.gumroad.com/confirm?destination=hls_playlist&id=d...

I am using youtube-dl on the terminal.

Nezteb commented 1 year ago

@aghwotu Here is a video demo of me downloading a Gumroad video: https://www.dropbox.com/s/1xi1j3gm77k9rj8/youtube-dl-gumroad.mov (I tried to upload to GitHub directly but the file size is over the 10MB limit)

The method has changed slightly, but it still works!

dirkf commented 1 year ago

Where on the site are the media items that OP wishes to download?

Gelob commented 1 year ago

Where on the site are the media items that OP wishes to download?

You can single video URL in the OP as an example. you just need to enter some email, it can be test@test.com to get access. Thats usually how gumroad works.

dirkf commented 1 year ago

OK. This procedure can be exercised in a browser with JS disabled.

For the problem URL, extract the second path component as video_id: 59bd0a45c6b12585318c698c03e4752d

GET this URL: 'https://app.gumroad.com/r/{video_id}'

The response will be a web page like https://app.gumroad.com/confirm?destination=show&id={video_id} containing a <form> element like this:

<form action="/confirm-redirect" accept-charset="UTF-8" method="post"><input name="authenticity_token" value="SCshXUCUtIjwZImBjk3LNpoUPEiGefDRCTn5nKLnDfMrSamraNSpai4g3D7LeSZHDA9P_st_r10UEhvQYWjM8A" autocomplete="off" type="hidden"><input name="id" id="id" value="59bd0a45c6b12585318c698c03e4752d" autocomplete="off" type="hidden">
<input name="stream" id="stream" autocomplete="off" type="hidden">
<input name="destination" id="destination" value="show" autocomplete="off" type="hidden">
<input name="display" id="display" autocomplete="off" type="hidden">
<input name="email" id="email" placeholder="Email address" type="text">
<button class="button button-primary primary" type="submit">Confirm email</button>
</form>

POST this form data using the appropriate email address, with the correct Referer and the Cookie sent with that page as Content-Type: application/x-www-form-urlencoded:

POST to https://app.gumroad.com/confirm-redirect:

authenticity_token=SCshXUCUtIjwZImBjk3LNpoUPEiGefDRCTn5nKLnDfMrSamraNSpai4g3D7LeSZHDA9P_st_r10UEhvQYWjM8A
id=59bd0a45c6b12585318c698c03e4752d
stream
destination=show
display
email=test@test.com

Now re-fetch the original page 'https://app.gumroad.com/r/{video_id}'. The request redirects to the file link, in this case https://files.gumroad.com/attachments/8386894450133/59bca733661f401193ba48b65f55594a/original/bbb_sunflower_1080p_30fps_normal.mp4?response-content-disposition=attachment&cache_key=051b&verify=1680949777-otsBuOfj18p8lMETQlGe4s5ux7/rzbeYVykjSmM+uWo=.

Presumably, the original request might sometimes succeed without the POST.

This logic shouldn't be hard to implement in an extractor.

ytdl-org / youtube-dl