search

bbolli / tumblr-utils

Utilities for dealing with Tumblr blogs, Tumblr backup
GNU General Public License v3.0
667 stars 124 forks source link

script no longer pulling down videos #105

Open doodie opened 6 years ago

doodie commented 6 years ago

hi,

any thoughts on why videos would stop downloading? the backup just references the tumblr url for the video. seems like like the last successful download of a video was late august. i did just update youtube-dl the other day but have had many videos since late august.

[root@superbrain posts]# youtube-dl --version 2018.09.26

I can confirm that running youtube-dl video_url does work from cli.

Thoughts or suggestions?

bbolli commented 6 years ago

No, sadly I myself don't use the video option. Do you get any errors?

doodie commented 6 years ago

i need to do more troubleshooting. but from what i see so far, no errors.

doodie commented 6 years ago

This is what I have figured out. I have been using this script for close to 2 years with no problems. I think something may have changed in August some time with Tumblr as that was when it stopped working.

So I have no problems downloading directly with youtube-dl direct. I have rolled back to a version from March of this year.

[root@superbrain tumblr_backup]# youtube-dl --verbose https://ve.media.tumblr.com/tumblr_pg0453RP9n1uje4jx.mp4
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--verbose', 'https://ve.media.tumblr.com/tumblr_pg0453RP9n1uje4jx.mp4']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2018.03.03
[debug] Python version 3.4.9 (CPython) - Linux-4.16.8-1.el7.elrepo.x86_64-x86_64-with-centos-7.5.1804-Core
[debug] exe versions: none
[debug] Proxy map: {}
[generic] tumblr_pg0bu4RP9n1uje4jx: Requesting header
[debug] Default format spec: best/bestvideo+bestaudio
[debug] Invoking downloader on 'https://ve.media.tumblr.com/tumblr_pg0453RP9n1uje4jx.mp4'
[download] Destination: tumblr_pg0453RP9n1uje4jx.mp4-tumblr_pg0453RP9n1uje4jx.mp4.mp4
[download] 100% of 17.63MiB in 00:00

Downloaded fine.

I then started to read your documentation and there was an option, -T type. I used video and it didnt back up anything.

[root@superbrain tumblr_backup]# /mnt/data02/backups/tumblr_backup/tumblr_backup.py -i --save-video -T video blog.tumblr.com blog.tumblr.com: 0 posts backed up

I created a bunch of test video posts to work through this, so even thought I am using -i, there are new posts. How is the script determining type? I suck at python so not able to understand how it is called. Any other info I can provide to help troubleshoot?

Anyone else having this issue?

bbolli commented 6 years ago

I created a bunch of test video posts to work through this, so even thought I am using -i, there are new posts. How is the script determining type? I suck at python so not able to understand how it is called. Any other info I can provide to help troubleshoot?

The JSON returned from Tumblr's API has a type field. the -T option just filters on this field.

More generally: this script and youtube-dl is very much at the mercy of the content providers (a.k.a. silos). They can change any one of a hundred small details and the download will fail. Maybe they started checking the referrer when getting the video, maybe something else, who knows?

cebtenzzre commented 4 years ago

@doodie Can you reproduce this with latest tumblr-utils and youtube-dl module? If so, is there a blog we could test with? Tumblr has been known to lose videos, but if they show fine in a browser and aren't being backed up, that's a bug.