search

ColumPaget / Movgrab

Simple downloader for videos from various websites
GNU General Public License v3.0
22 stars 5 forks source link

Unable to download from many video servers... #1

Closed antdude closed 7 years ago

antdude commented 8 years ago

Examples of failed downloads: http://cwtv.com/thecw/chroniclesofcisco http://www.gameinformer.com/b/features/archive/2016/05/13/replay-jak-ii.aspx http://cinemassacre.com/2016/05/11/godzilla-resurgence-2016/ http://www.cbs.com/shows/big_bang_theory/video/

They all had these similar results: $ movgrab http://www.gameinformer.com/b/features/archive/2016/05/13/replay-jak-ii.aspx Unrecognized url type. Falling Back to 'generic youtube frontend'. Try using the -t option to force the service type ( "movgrab -?" for more details )

Formats available for this Movie:

No suitable download format found from 'mp4,flv,webm,m4v,mov,mpg,mpeg,wmv,avi,3gp,reference,mp3,m4a,wma,m3u8,m3u8-stream'

FAILED TO GET DOCUMENT REFERENCE! Maybe you need to change the format preferences, or else the url you supplied is wrong ERROR: failed to Connect to http://www.gameinformer.com/b/features/archive/2016/05/13/replay-jak-ii.aspx

ColumPaget commented 8 years ago

unfortunately it's likely never going to be able to download from all of those. Each site requires a different process to get to the video file, and there's only some sites where I've been able to figure that process out. Browser-based downloaders are able to get around this because they leverage the whole DOM/javascript/whatever model that's built into the browser to obtain the final URL. Movgrab, not having all the rendering and scripting features of a browser, tries to do the same by applying some quick 'hacks', but these only work for a few sites. I'll take a look at the sites you ist and see if I can get any of them working, but it's unlikely to be possible.

antdude commented 8 years ago

Ah, bummer. :(

knezi commented 8 years ago

Hi, If I run movgrab -test-sites, many sites are broken. Output:

Checking youtube              Unknown youtube format [url=http%3A%2F%2Fr1---sn-jxnoxu-2gbe.googlevideo.com%2Fvideoplayback%3Fsver%3D3%26ratebypass%3Dyes%26itag%3D59%26source%3Dyoutube%26pl%3D20%26pcm2cms%3Dyes%26ipbits%3D0%26upn%3DeAp2xuhi2-U%26initcwndbps%3D9307500%26id%3Do-AI3N4zcV9P82ntWAimoBnT_mDGPIeo5edTSqiLCs9ZI2%26mm%3D31%26mn%3Dsn-jxnoxu-2gbe%26ms%3Dau%26mt%3D1463555134%26mv%3Dm%26ip%3D147.32.116.47%26signature%3D50B43641DA8D321EC55F5DEF0EC56359CAF5F255.23C02C0C8A067E2020971EF947C46077B0F1BE2A%26dur%3D284.490%26lmt%3D1354503304935109%26fexp%3D9408207%252C9416126%252C9416891%252C9417893%252C9422596%252C9427543%252C9428398%252C9431012%252C9433046%252C9433096%252C9433189%252C9433425%252C9433946%252C9434066%252C9435038%252C9435525%252C9436107%252C9436447%252C9436797%252C9436956%26expire%3D1463576894%26sparams%3Ddur%252Cid%252Cinitcwndbps%252Cip%252Cipbits%252Citag%252Clmt%252Cmime%252Cmm%252Cmn%252Cms%252Cmv%252Cpcm2cms%252Cpl%252Cratebypass%252Csource%252Cupn%252Cexpire%26mime%3Dvideo%252Fmp4%26key%3Dyt6&fallback_host=tc.v8.cache2.googlevideo.com&type=video%2Fmp4%3B+codecs%3D%22avc1.4D001F%2C+mp4a.40.2%22&itag=59&quality=large]
Unknown youtube format [url=http%3A%2F%2Fr1---sn-jxnoxu-2gbe.googlevideo.com%2Fvideoplayback%3Finitcwndbps%3D9307500%26id%3Do-AI3N4zcV9P82ntWAimoBnT_mDGPIeo5edTSqiLCs9ZI2%26mm%3D31%26mn%3Dsn-jxnoxu-2gbe%26sver%3D3%26ms%3Dau%26mt%3D1463555134%26mv%3Dm%26ip%3D147.32.116.47%26signature%3DE2B8F0CD4CB4DE9657F70742AC537C7BABBE536E.2F1C59C789FC5688EADB1C4AD0FA39F88302AE92%26itag%3D78%26source%3Dyoutube%26dur%3D284.490%26lmt%3D1354503299148254%26pl%3D20%26fexp%3D9408207%252C9416126%252C9416891%252C9417893%252C9422596%252C9427543%252C9428398%252C9431012%252C9433046%252C9433096%252C9433189%252C9433425%252C9433946%252C9434066%252C9435038%252C9435525%252C9436107%252C9436447%252C9436797%252C9436956%26pcm2cms%3Dyes%26ipbits%3D0%26upn%3DeAp2xuhi2-U%26expire%3D1463576894%26sparams%3Ddur%252Cid%252Cinitcwndbps%252Cip%252Cipbits%252Citag%252Clmt%252Cmime%252Cmm%252Cmn%252Cms%252Cmv%252Cpcm2cms%252Cpl%252Csource%252Cupn%252Cexpire%26mime%3Dvideo%252Fmp4%26key%3Dyt6&fallback_host=tc.v10.cache1.googlevideo.com&type=video%2Fmp4%3B+codecs%3D%22avc1.4D001F%2C+mp4a.40.2%22&itag=78&quality=medium]
 okay
Checking metacafe              BROKEN
Checking dailymotion           BROKEN
Checking break                 BROKEN
Checking ehow                  BROKEN
Checking vimeo                 BROKEN
Checking vbox7                 BROKEN
Checking blip.tv               BROKEN
Checking ted                   BROKEN
Checking myvideo               BROKEN
Checking mytopclip             BROKEN
Checking redbalcony            okay
Checking mobango               BROKEN
Checking yale                  BROKEN
Checking princeton             BROKEN
Checking reuters               okay
Checking liveleak              okay
Checking academicearth         BROKEN
Checking photobucket           okay
Checking aljazeera             okay
Checking mefeedia              okay
Checking iviewtube             BROKEN
Checking washingtonpost        BROKEN
Checking cbsnews               BROKEN
Checking france24              BROKEN
Checking euronews              BROKEN
Checking metatube              BROKEN
Checking motionfeeds           BROKEN
Checking escapist              BROKEN
Checking guardian              BROKEN
Checking redorbit              BROKEN
Checking scivee                BROKEN
Checking izlese                okay
Checking uctv.tv               okay
Checking royalsociety.tv       okay
Checking britishacademy        BROKEN
Checking kitp                  BROKEN
Checking dotsub                okay
Checking astronomy.com         BROKEN
Checking teachertube.com       BROKEN
Checking discovery             BROKEN
Checking bloomberg             BROKEN
Checking nationalgeographic    BROKEN
Checking videobash             okay
Checking ibtimes               okay
Checking smh                   BROKEN
Checking videojug              BROKEN
Checking animehere             BROKEN
Checking funnyordie           ERROR: Connection failed to  can't get file=/ 
 BROKEN```

I have vesrion 1.2.1.
ColumPaget commented 8 years ago

Hi,

Many of them aren't actually broken, but for some reason the test script says they are. I've not yet been able to get to the bottom of this.

Colum

On 5/18/16, Jan Knížek notifications@github.com wrote:

Hi, If I run movgrab -test-sites, many sites are broken. Output:

Checking youtube              Unknown youtube format
[url=http%3A%2F%2Fr1---sn-jxnoxu-2gbe.googlevideo.com%2Fvideoplayback%3Fsver%3D3%26ratebypass%3Dyes%26itag%3D59%26source%3Dyoutube%26pl%3D20%26pcm2cms%3Dyes%26ipbits%3D0%26upn%3DeAp2xuhi2-U%26initcwndbps%3D9307500%26id%3Do-AI3N4zcV9P82ntWAimoBnT_mDGPIeo5edTSqiLCs9ZI2%26mm%3D31%26mn%3Dsn-jxnoxu-2gbe%26ms%3Dau%26mt%3D1463555134%26mv%3Dm%26ip%3D147.32.116.47%26signature%3D50B43641DA8D321EC55F5DEF0EC56359CAF5F255.23C02C0C8A067E2020971EF947C46077B0F1BE2A%26dur%3D284.490%26lmt%3D1354503304935109%26fexp%3D9408207%252C9416126%252C9416891%252C9417893%252C9422596%252C9427543%252C9428398%252C9431012%252C9433046%252C9433096%252C9433189%252C9433425%252C9433946%252C9434066%252C9435038%252C9435525%252C9436107%252C9436447%252C9436797%252C9436956%26expire%3D1463576894%26sparams%3Ddur%252Cid%252Cinitcwndbps%252Cip%252Cipbits%252Citag%252Clmt%252Cmime%252Cmm%252Cmn%252Cms%252Cmv%252Cpcm2cms%252Cpl%252Cratebypass%252Csource%252Cupn%252Cexpire%26mime%3
Dvideo%252Fmp4%26key%3Dyt6&fallback_host=tc.v8.cache2.googlevideo.com&type=video%2Fmp4%3B+codecs%3D%22avc1.4D001F%2C+mp4a.40.2%22&itag=59&quality=large]
Unknown youtube format
[url=http%3A%2F%2Fr1---sn-jxnoxu-2gbe.googlevideo.com%2Fvideoplayback%3Finitcwndbps%3D9307500%26id%3Do-AI3N4zcV9P82ntWAimoBnT_mDGPIeo5edTSqiLCs9ZI2%26mm%3D31%26mn%3Dsn-jxnoxu-2gbe%26sver%3D3%26ms%3Dau%26mt%3D1463555134%26mv%3Dm%26ip%3D147.32.116.47%26signature%3DE2B8F0CD4CB4DE9657F70742AC537C7BABBE536E.2F1C59C789FC5688EADB1C4AD0FA39F88302AE92%26itag%3D78%26source%3Dyoutube%26dur%3D284.490%26lmt%3D1354503299148254%26pl%3D20%26fexp%3D9408207%252C9416126%252C9416891%252C9417893%252C9422596%252C9427543%252C9428398%252C9431012%252C9433046%252C9433096%252C9433189%252C9433425%252C9433946%252C9434066%252C9435038%252C9435525%252C9436107%252C9436447%252C9436797%252C9436956%26pcm2cms%3Dyes%26ipbits%3D0%26upn%3DeAp2xuhi2-U%26expire%3D1463576894%26sparams%3Ddur%252Cid%252Cinitcwndbps%252Cip%252Cipbits%252Citag%252Clmt%252Cmime%252Cmm%252Cmn%252Cms%252Cmv%252Cpcm2cms%252Cpl%252Csource%252Cupn%252Cexpire%26mime%3Dvideo%252Fmp4%26key%3Dyt6&fallback_host=tc.v10.cache1.googlevid
eo.com&type=video%2Fmp4%3B+codecs%3D%22avc1.4D001F%2C+mp4a.40.2%22&itag=78&quality=medium]
 okay
Checking metacafe              BROKEN
Checking dailymotion           BROKEN
Checking break                 BROKEN
Checking ehow                  BROKEN
Checking vimeo                 BROKEN
Checking vbox7                 BROKEN
Checking blip.tv               BROKEN
Checking ted                   BROKEN
Checking myvideo               BROKEN
Checking mytopclip             BROKEN
Checking redbalcony            okay
Checking mobango               BROKEN
Checking yale                  BROKEN
Checking princeton             BROKEN
Checking reuters               okay
Checking liveleak              okay
Checking academicearth         BROKEN
Checking photobucket           okay
Checking aljazeera             okay
Checking mefeedia              okay
Checking iviewtube             BROKEN
Checking washingtonpost        BROKEN
Checking cbsnews               BROKEN
Checking france24              BROKEN
Checking euronews              BROKEN
Checking metatube              BROKEN
Checking motionfeeds           BROKEN
Checking escapist              BROKEN
Checking guardian              BROKEN
Checking redorbit              BROKEN
Checking scivee                BROKEN
Checking izlese                okay
Checking uctv.tv               okay
Checking royalsociety.tv       okay
Checking britishacademy        BROKEN
Checking kitp                  BROKEN
Checking dotsub                okay
Checking astronomy.com         BROKEN
Checking teachertube.com       BROKEN
Checking discovery             BROKEN
Checking bloomberg             BROKEN
Checking nationalgeographic    BROKEN
Checking videobash             okay
Checking ibtimes               okay
Checking smh                   BROKEN
Checking videojug              BROKEN
Checking animehere             BROKEN
Checking funnyordie           ERROR: Connection failed to  can't get file=/

 BROKEN```

I have vesrion 1.2.1.

---
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
https://github.com/ColumPaget/Movgrab/issues/1#issuecomment-219944505
knezi commented 8 years ago

Thanks for reply, but still:

[knezi@holly ~]$ movgrab -t youtube https://www.youtube.com/watch?v=7mdIWaRi-7c

Formats available for this Movie: webm:640x360
ERROR: 403 response for https://r3---sn-jxnoxu-2gbe.googlevideo.com/videoplayback?key=yt6&itag=43&mime=video%2Fwebm&mt=1464852364&expire=1464874091&ipbits=0&gcr=cz&sparams=dur%2Cgcr%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cupn%2Cexpire&initcwndbps=9633750&sver=3&lmt=1280372872577639&mv=m&ip=147.32.116.47&ms=au&fexp=9416126%2C9416891%2C9419451%2C9422596%2C9423965%2C9426977%2C9428398%2C9431012%2C9431827%2C9432684%2C9432822%2C9433046%2C9433096%2C9433380%2C9433946%2C9435527%2C9435876%2C9436116%2C9436810%2C9436991%2C9437066%2C9437158%2C9437552%2C9437712%2C9438227%2C9438283&source=youtube&upn=a3DZebayAq0&pl=19&mn=sn-jxnoxu-2gbe&mm=31&dur=0.000&requiressl=yes&id=o-AKV2yc7J02UfueTS2wZ2AlEdwsRRnKOse1piVHmKVtJv&ratebypass=yes

Connection Refused, sleeping for 20 secs before retry

(The same without specified -t youtube)

mo-fayed73 commented 8 years ago

it works with me if:

ColumPaget commented 8 years ago

@knezi Oh! You have version 1.2.1? Not version 2.0.0? 1.2.1 is pretty old, although it was the last version before 2.0.0. My statement that 'many display as broken, but actually work' only applies to 2.0.0.

In 2.0.0 sites are claiming to be broken when they're not. I've not yet figured out why. 2.0.0 is the 'resurrection release' where I'm trying to get the project back on it's feet!

I'll try downloading that link that's not working for you, and see if I can discover what the issue is there.

@mo-fayed73 The 'replace https' issue is a surprise to me. Was your movgrab compiled with '--enable-ssl'?

knezi commented 8 years ago

@ColumPaget Sorry, I just downloaded it from AUR and did not realise it is outdated. I will let know the maintainer.

ColumPaget commented 8 years ago

@knezi no probs, to be honest I only just put the new version up a couple of weeks ago, and it's still very much a work in progress. AUR is... something to do with arch? I didn't even know movgrab was still being included in anyone's distro!

knezi commented 8 years ago

@ColumPaget, it is special kind of repository. There are not compiled binaries, but packages containing information about fetching the code, compiling, version etc. In order to install this package, script called makepkg exists which can automatically compile the package for you. In summary, it is a great tool for autocompiling - you just download the package and mostly everything is done for you. (See https://wiki.archlinux.org/index.php/Arch_User_Repository if interested). Where exactly is the new version located? I did not find it (in commits, at MovGrab official site...)