download-online-video / chrome-avgle-helper

👏 A Chrome extension for Avgle and other interesting video sites. Downloader inside
GNU General Public License v3.0
318 stars 65 forks source link

For more websites? #1

Open DoctorAnonymous opened 5 years ago

DoctorAnonymous commented 5 years ago

Well, I think the only practical way to download videos from different websites is to use Chrome extensions for its simulation of human behavior. Do you have any intension to develop one? Bash or Python scripts are definitely not the final answer.

vide0 commented 5 years ago

I will discuss this issue on this weekends.

yobailover commented 5 years ago

@vide0 I think you can borrow some idea from You-Get for bash or python script download video from multiple sites. Of course for some websites which hide their video src, chrome extension cannot be avoided.

vide0 commented 5 years ago

@DoctorAnonymous
The simulation of human behavior you mentioned is a good idea. But I think implement it is difficult sometimes. For example: Avgle will ask you verify you are a human by reCAPTCHA inside it. (snapshot below) And you can not get m3u8 playlist file if you did not pass the "I am a human" test.

And there are some shortage of Chrome extension, such as downloading multi-file, combing videos and more complex file operation.

So the best way I can think of now is implement downloading online video by using a combination of Chrome extension and Python/Bash scripts.

Do you have some other ideas or options to achieve it?

vide0 commented 5 years ago

@yobailover Thanks, you give me a good idea to achieve it.

And do you have some good idea to resolve the problem of captcha? because I found the there are two issues in You-Get about the captcha:

https://github.com/soimort/you-get/pulls?utf8=%E2%9C%93&q=+captcha+

DoctorAnonymous commented 5 years ago

My thought is to monitor the network requests and extract video requests. As you mentioned, video sites have added human-checking tests. If chrome extension can monitor all the video requests including .mp4, .m3u8 and .ts files, human test can be finished by users and then users can choose from a list of videos. Multi-threading and file combination are serious problems, however requiring scripts to act like browsers seems even harder.

vide0 commented 5 years ago

@DoctorAnonymous So just create a Chrome extension to monitoring any video request and dispay them to the user. And user can choose anyone to download/combine it by script/program?

namedHK commented 5 years ago

脚本下载视屏的时候出现403错误,是不是用多个代理ip来解决?

ilovefood2 commented 5 years ago

https://www.google.com/recaptcha/intro/v3.html

google recaptcha is using data analysis in background to detect human behavioural patterns, if we can implement something similar that matches pattern, then we can fool it. And here comes machine learning to help ( i bet google is using it too ). I don't know much about chrome extension, but I do know machine learning and i know chrome extension is pretty much using js, and there is js version of tensorflow which can definitely achieve what we want to do.

https://www.tensorflow.org/js

just an idea for the starter :)

ilovefood2 commented 5 years ago

also, it would be nice if you could have this project compatible with many other websites such as youtube or maybe even netflix lol

vide0 commented 5 years ago

@namedHK Actually,Downloading video from different IP addresses will cause the 403 sometimes too.

Btw, You can change a proxy and download again if you meet 403 problem. Or get a new download command by refresh Avgle page and delete local m3u8 file.

通過不同的IP地址下載視頻有時也會導致403問題。 順帶一提,遇到403,你也可以切換代理重新下載,或是刷新Avgle頁面重新獲取下載命令並記得刪除蹦迪的m3u8文件

vide0 commented 5 years ago

@ilovefood2 I dont have enough time to adding bypass recaptcha by using ML way. And I think it is not worth for that. And supporting more website, I will add into it in the future.

ilovefood2 commented 5 years ago

thx, and will you also be able to add some kinda button to start downloading? right now it seems have to manually copy to terminal and run those commands

vide0 commented 5 years ago

@ilovefood2 This feature is planned. Many features will be add into in this weekend or next weekend