search

Kethsar / ytarchive

Garbage Youtube livestream downloader
MIT License
1.18k stars 93 forks source link

HTTP Error for fragment 0: 404 Not Found #50

Closed user334 closed 1 year ago

user334 commented 3 years ago

Hi there! Today I've started to facing with an issue: whenever I'm trying to download any video stream it says (with the --debug option): 

2021/10/14 18:06:50 DEBUG: audio1: HTTP Error for fragment 0: 404 Not Found
2021/10/14 18:06:55 DEBUG: video1: HTTP Error for fragment 0: 404 Not Found
2021/10/14 18:06:55 DEBUG: audio1: HTTP Error for fragment 0: 404 Not Found

Checked two streams on old Python version, new Golang version and the build from the latest commit - problem still appears. Last night all worked as usual. Maybe Google did changed something?

Kethsar commented 3 years ago

Interesting. I just checked and it wasn't an issue for me. I have no idea what would cause that, but if it starts cropping up for more and more people as time goes on, it will indeed be likely youtube changed something.

That said, may I ask what format URL you used? i.e. /watch, /live, or direct googlevideo URLs.

user334 commented 3 years ago

That said, may I ask what format URL you used? i.e. /watch, /live, or direct googlevideo URLs.

I'm using /watch url format.

Have just checked again all the methods as I mentioned above with some more random streams on two different devices (Linux and Windows. Different IP on Windows but same ISP) - same result.

user334 commented 3 years ago

Hi!

I've made yet another test on a completely different machine with windows on another ip on another ISP with the today's commits -- same result :( Here's the log. I've added a couple of lines to show the url it fails to download. Sometimes it gives 403 instead of 404

ytarchive.exe --debug --no-save --no-merge --vp9 https://www.youtube.com/watch?v=R7vrbiDi0Tc best
Selected quality: 1080p (VP9)
2021/10/16 01:22:32 INFO: Starting download to R7vrbiDi0Tc__4005025691\Kitten Academy Live Stream-R7vrbiDi0Tc.f140.ts
2021/10/16 01:22:32 INFO: Starting download to R7vrbiDi0Tc__4005025691\Kitten Academy Live Stream-R7vrbiDi0Tc.f248.ts
2021/10/16 01:22:32 DEBUG: seqUrl from http.NewRequest
2021/10/16 01:22:32 DEBUG: https://r3---sn-n3toxu-axqs.googlevideo.com/videoplayback?expire=1634343751&ei=58ZpYYj7ItDl7QTZ8LvAAw&ip=[my_ip]&id=R7vrbiDi0Tc.1&itag=140&source=yt_live_broadcast&requiressl=yes&mh=hi&mm=44%2C29&mn=sn-n3toxu-axqs%2Csn-5goeen7y&ms=lva%2Crdu&mv=m&mvi=3&pl=21&initcwndbps=1230000&vprv=1&live=1&hang=1&noclen=1&mime=audio%2Fmp4&ns=wl4SkVXEy5WfYRNAaCI3WRgG&gir=yes&mt=1634321805&fvip=3&keepalive=yes&fexp=24001373%2C24007246&c=WEB&n=K7gjkj54AR5p03Anu&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cvprv%2Clive%2Chang%2Cnoclen%2Cmime%2Cns%2Cgir&sig=AOq0QJ8wRAIgLH8DmU1IPDN0nuNyrI8-iKmKYuh0dPKH59DtLRHJTFgCIBWGjpmx04fQ7LVY9GWb1qAxSlWHmkXgzzkdoMqyHvzt&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRQIgEhnn1fTn4JcTkcUXv3vCZ-3BYg0x6ZrBWSsyVLUWZCgCIQDYiswCt-psMMJ-yNRE7Uro85rRBJb6GJn9QnjwoGlQCA%3D%3D&sq=0
2021/10/16 01:22:32 DEBUG:
2021/10/16 01:22:32 DEBUG: seqUrl from http.NewRequest
2021/10/16 01:22:32 DEBUG: https://r3---sn-n3toxu-axqs.googlevideo.com/videoplayback?expire=1634343751&ei=58ZpYYj7ItDl7QTZ8LvAAw&ip=[my_ip]&id=R7vrbiDi0Tc.1&itag=248&aitags=133%2C134%2C135%2C136%2C137%2C160%2C242%2C243%2C244%2C247%2C248%2C271%2C278%2C313&source=yt_live_broadcast&requiressl=yes&mh=hi&mm=44%2C29&mn=sn-n3toxu-axqs%2Csn-5goeen7y&ms=lva%2Crdu&mv=m&mvi=3&pl=21&initcwndbps=1230000&vprv=1&live=1&hang=1&noclen=1&mime=video%2Fwebm&ns=wl4SkVXEy5WfYRNAaCI3WRgG&gir=yes&mt=1634321805&fvip=3&keepalive=yes&fexp=24001373%2C24007246&c=WEB&n=K7gjkj54AR5p03Anu&sparams=expire%2Cei%2Cip%2Cid%2Caitags%2Csource%2Crequiressl%2Cvprv%2Clive%2Chang%2Cnoclen%2Cmime%2Cns%2Cgir&sig=AOq0QJ8wRQIhALBLDv9QJaESTfZ_3DJ_f-lyJM4-IYIB5sJ1QBUGNkqMAiBtDKrRWJJD1yUdVdsO_3uYHHd4MTIjlN86WU24Pr9UcA%3D%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRQIgEhnn1fTn4JcTkcUXv3vCZ-3BYg0x6ZrBWSsyVLUWZCgCIQDYiswCt-psMMJ-yNRE7Uro85rRBJb6GJn9QnjwoGlQCA%3D%3D&sq=0
2021/10/16 01:22:32 DEBUG:
2021/10/16 01:22:32 DEBUG: video1: HTTP Error for fragment 0: 404 Not Found
2021/10/16 01:22:32 DEBUG: baseUrl form HandleFragHttpError
2021/10/16 01:22:32 DEBUG: https://r3---sn-n3toxu-axqs.googlevideo.com/videoplayback?expire=1634343751&ei=58ZpYYj7ItDl7QTZ8LvAAw&ip=[my_ip]&id=R7vrbiDi0Tc.1&itag=248&aitags=133%%2C134%%2C135%%2C136%%2C137%%2C160%%2C242%%2C243%%2C244%%2C247%%2C248%%2C271%%2C278%%2C313&source=yt_live_broadcast&requiressl=yes&mh=hi&mm=44%%2C29&mn=sn-n3toxu-axqs%%2Csn-5goeen7y&ms=lva%%2Crdu&mv=m&mvi=3&pl=21&initcwndbps=1230000&vprv=1&live=1&hang=1&noclen=1&mime=video%%2Fwebm&ns=wl4SkVXEy5WfYRNAaCI3WRgG&gir=yes&mt=1634321805&fvip=3&keepalive=yes&fexp=24001373%%2C24007246&c=WEB&n=K7gjkj54AR5p03Anu&sparams=expire%%2Cei%%2Cip%%2Cid%%2Caitags%%2Csource%%2Crequiressl%%2Cvprv%%2Clive%%2Chang%%2Cnoclen%%2Cmime%%2Cns%%2Cgir&sig=AOq0QJ8wRQIhALBLDv9QJaESTfZ_3DJ_f-lyJM4-IYIB5sJ1QBUGNkqMAiBtDKrRWJJD1yUdVdsO_3uYHHd4MTIjlN86WU24Pr9UcA%%3D%%3D&lsparams=mh%%2Cmm%%2Cmn%%2Cms%%2Cmv%%2Cmvi%%2Cpl%%2Cinitcwndbps&lsig=AG3C_xAwRQIgEhnn1fTn4JcTkcUXv3vCZ-3BYg0x6ZrBWSsyVLUWZCgCIQDYiswCt-psMMJ-yNRE7Uro85rRBJb6GJn9QnjwoGlQCA%%3D%%3D&sq=%d
2021/10/16 01:22:32 DEBUG:
2021/10/16 01:22:32 DEBUG: audio1: HTTP Error for fragment 0: 404 Not Found
2021/10/16 01:22:32 DEBUG: baseUrl form HandleFragHttpError
2021/10/16 01:22:32 DEBUG: https://r3---sn-n3toxu-axqs.googlevideo.com/videoplayback?expire=1634343751&ei=58ZpYYj7ItDl7QTZ8LvAAw&ip=[my_ip]&id=R7vrbiDi0Tc.1&itag=140&source=yt_live_broadcast&requiressl=yes&mh=hi&mm=44%%2C29&mn=sn-n3toxu-axqs%%2Csn-5goeen7y&ms=lva%%2Crdu&mv=m&mvi=3&pl=21&initcwndbps=1230000&vprv=1&live=1&hang=1&noclen=1&mime=audio%%2Fmp4&ns=wl4SkVXEy5WfYRNAaCI3WRgG&gir=yes&mt=1634321805&fvip=3&keepalive=yes&fexp=24001373%%2C24007246&c=WEB&n=K7gjkj54AR5p03Anu&sparams=expire%%2Cei%%2Cip%%2Cid%%2Citag%%2Csource%%2Crequiressl%%2Cvprv%%2Clive%%2Chang%%2Cnoclen%%2Cmime%%2Cns%%2Cgir&sig=AOq0QJ8wRAIgLH8DmU1IPDN0nuNyrI8-iKmKYuh0dPKH59DtLRHJTFgCIBWGjpmx04fQ7LVY9GWb1qAxSlWHmkXgzzkdoMqyHvzt&lsparams=mh%%2Cmm%%2Cmn%%2Cms%%2Cmv%%2Cmvi%%2Cpl%%2Cinitcwndbps&lsig=AG3C_xAwRQIgEhnn1fTn4JcTkcUXv3vCZ-3BYg0x6ZrBWSsyVLUWZCgCIQDYiswCt-psMMJ-yNRE7Uro85rRBJb6GJn9QnjwoGlQCA%%3D%%3D&sq=%d
2021/10/16 01:22:32 DEBUG:
2021/10/16 01:22:37 DEBUG: seqUrl from http.NewRequest
2021/10/16 01:22:37 DEBUG: https://r3---sn-n3toxu-axqs.googlevideo.com/videoplayback?expire=1634343751&ei=58ZpYYj7ItDl7QTZ8LvAAw&ip=[my_ip]&id=R7vrbiDi0Tc.1&itag=248&aitags=133%2C134%2C135%2C136%2C137%2C160%2C242%2C243%2C244%2C247%2C248%2C271%2C278%2C313&source=yt_live_broadcast&requiressl=yes&mh=hi&mm=44%2C29&mn=sn-n3toxu-axqs%2Csn-5goeen7y&ms=lva%2Crdu&mv=m&mvi=3&pl=21&initcwndbps=1230000&vprv=1&live=1&hang=1&noclen=1&mime=video%2Fwebm&ns=wl4SkVXEy5WfYRNAaCI3WRgG&gir=yes&mt=1634321805&fvip=3&keepalive=yes&fexp=24001373%2C24007246&c=WEB&n=K7gjkj54AR5p03Anu&sparams=expire%2Cei%2Cip%2Cid%2Caitags%2Csource%2Crequiressl%2Cvprv%2Clive%2Chang%2Cnoclen%2Cmime%2Cns%2Cgir&sig=AOq0QJ8wRQIhALBLDv9QJaESTfZ_3DJ_f-lyJM4-IYIB5sJ1QBUGNkqMAiBtDKrRWJJD1yUdVdsO_3uYHHd4MTIjlN86WU24Pr9UcA%3D%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRQIgEhnn1fTn4JcTkcUXv3vCZ-3BYg0x6ZrBWSsyVLUWZCgCIQDYiswCt-psMMJ-yNRE7Uro85rRBJb6GJn9QnjwoGlQCA%3D%3D&sq=0
2021/10/16 01:22:37 DEBUG:
2021/10/16 01:22:37 DEBUG: seqUrl from http.NewRequest
2021/10/16 01:22:37 DEBUG: https://r3---sn-n3toxu-axqs.googlevideo.com/videoplayback?expire=1634343751&ei=58ZpYYj7ItDl7QTZ8LvAAw&ip=[my_ip]&id=R7vrbiDi0Tc.1&itag=140&source=yt_live_broadcast&requiressl=yes&mh=hi&mm=44%2C29&mn=sn-n3toxu-axqs%2Csn-5goeen7y&ms=lva%2Crdu&mv=m&mvi=3&pl=21&initcwndbps=1230000&vprv=1&live=1&hang=1&noclen=1&mime=audio%2Fmp4&ns=wl4SkVXEy5WfYRNAaCI3WRgG&gir=yes&mt=1634321805&fvip=3&keepalive=yes&fexp=24001373%2C24007246&c=WEB&n=K7gjkj54AR5p03Anu&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cvprv%2Clive%2Chang%2Cnoclen%2Cmime%2Cns%2Cgir&sig=AOq0QJ8wRAIgLH8DmU1IPDN0nuNyrI8-iKmKYuh0dPKH59DtLRHJTFgCIBWGjpmx04fQ7LVY9GWb1qAxSlWHmkXgzzkdoMqyHvzt&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRQIgEhnn1fTn4JcTkcUXv3vCZ-3BYg0x6ZrBWSsyVLUWZCgCIQDYiswCt-psMMJ-yNRE7Uro85rRBJb6GJn9QnjwoGlQCA%3D%3D&sq=0
2021/10/16 01:22:37 DEBUG:
2021/10/16 01:22:38 DEBUG: video1: HTTP Error for fragment 0: 404 Not Found
2021/10/16 01:22:38 DEBUG: baseUrl form HandleFragHttpError
2021/10/16 01:22:38 DEBUG: https://r3---sn-n3toxu-axqs.googlevideo.com/videoplayback?expire=1634343751&ei=58ZpYYj7ItDl7QTZ8LvAAw&ip=[my_ip]&id=R7vrbiDi0Tc.1&itag=248&aitags=133%%2C134%%2C135%%2C136%%2C137%%2C160%%2C242%%2C243%%2C244%%2C247%%2C248%%2C271%%2C278%%2C313&source=yt_live_broadcast&requiressl=yes&mh=hi&mm=44%%2C29&mn=sn-n3toxu-axqs%%2Csn-5goeen7y&ms=lva%%2Crdu&mv=m&mvi=3&pl=21&initcwndbps=1230000&vprv=1&live=1&hang=1&noclen=1&mime=video%%2Fwebm&ns=wl4SkVXEy5WfYRNAaCI3WRgG&gir=yes&mt=1634321805&fvip=3&keepalive=yes&fexp=24001373%%2C24007246&c=WEB&n=K7gjkj54AR5p03Anu&sparams=expire%%2Cei%%2Cip%%2Cid%%2Caitags%%2Csource%%2Crequiressl%%2Cvprv%%2Clive%%2Chang%%2Cnoclen%%2Cmime%%2Cns%%2Cgir&sig=AOq0QJ8wRQIhALBLDv9QJaESTfZ_3DJ_f-lyJM4-IYIB5sJ1QBUGNkqMAiBtDKrRWJJD1yUdVdsO_3uYHHd4MTIjlN86WU24Pr9UcA%%3D%%3D&lsparams=mh%%2Cmm%%2Cmn%%2Cms%%2Cmv%%2Cmvi%%2Cpl%%2Cinitcwndbps&lsig=AG3C_xAwRQIgEhnn1fTn4JcTkcUXv3vCZ-3BYg0x6ZrBWSsyVLUWZCgCIQDYiswCt-psMMJ-yNRE7Uro85rRBJb6GJn9QnjwoGlQCA%%3D%%3D&sq=%d
2021/10/16 01:22:38 DEBUG:
2021/10/16 01:22:38 DEBUG: audio1: HTTP Error for fragment 0: 404 Not Found
2021/10/16 01:22:38 DEBUG: baseUrl form HandleFragHttpError
2021/10/16 01:22:38 DEBUG: https://r3---sn-n3toxu-axqs.googlevideo.com/videoplayback?expire=1634343751&ei=58ZpYYj7ItDl7QTZ8LvAAw&ip=[my_ip]&id=R7vrbiDi0Tc.1&itag=140&source=yt_live_broadcast&requiressl=yes&mh=hi&mm=44%%2C29&mn=sn-n3toxu-axqs%%2Csn-5goeen7y&ms=lva%%2Crdu&mv=m&mvi=3&pl=21&initcwndbps=1230000&vprv=1&live=1&hang=1&noclen=1&mime=audio%%2Fmp4&ns=wl4SkVXEy5WfYRNAaCI3WRgG&gir=yes&mt=1634321805&fvip=3&keepalive=yes&fexp=24001373%%2C24007246&c=WEB&n=K7gjkj54AR5p03Anu&sparams=expire%%2Cei%%2Cip%%2Cid%%2Citag%%2Csource%%2Crequiressl%%2Cvprv%%2Clive%%2Chang%%2Cnoclen%%2Cmime%%2Cns%%2Cgir&sig=AOq0QJ8wRAIgLH8DmU1IPDN0nuNyrI8-iKmKYuh0dPKH59DtLRHJTFgCIBWGjpmx04fQ7LVY9GWb1qAxSlWHmkXgzzkdoMqyHvzt&lsparams=mh%%2Cmm%%2Cmn%%2Cms%%2Cmv%%2Cmvi%%2Cpl%%2Cinitcwndbps&lsig=AG3C_xAwRQIgEhnn1fTn4JcTkcUXv3vCZ-3BYg0x6ZrBWSsyVLUWZCgCIQDYiswCt-psMMJ-yNRE7Uro85rRBJb6GJn9QnjwoGlQCA%%3D%%3D&sq=%d
2021/10/16 01:22:38 DEBUG:
2021/10/16 01:22:39 WARNING: User Interrupt, Stopping download...
2021/10/16 01:22:43 DEBUG: video1: exiting
2021/10/16 01:22:43 DEBUG: video-download thread closing
2021/10/16 01:22:43 DEBUG: audio1: exiting
2021/10/16 01:22:43 DEBUG: audio-download thread closing

ytarchive.exe --debug --no-save --no-merge --vp9 https://www.youtube.com/watch?v=R7vrbiDi0Tc best
Selected quality: 1080p (VP9)
2021/10/16 01:22:46 INFO: Starting download to R7vrbiDi0Tc__2694924203\Kitten Academy Live Stream-R7vrbiDi0Tc.f140.ts
2021/10/16 01:22:46 INFO: Starting download to R7vrbiDi0Tc__2694924203\Kitten Academy Live Stream-R7vrbiDi0Tc.f248.ts
2021/10/16 01:22:46 DEBUG: seqUrl from http.NewRequest
2021/10/16 01:22:46 DEBUG: https://r3---sn-n3toxu-axqs.googlevideo.com/videoplayback?expire=1634343765&ei=9cZpYc_cGZflyQWwpJuYDQ&ip=[my_ip]&id=R7vrbiDi0Tc.1&itag=140&aitags=140&source=yt_live_broadcast&requiressl=yes&mh=hi&mm=44%2C29&mn=sn-n3toxu-axqs%2Csn-5goeen7y&ms=lva%2Crdu&mv=m&mvi=3&pl=21&initcwndbps=1230000&vprv=1&live=1&hang=1&noclen=1&mime=audio%2Fmp4&ns=Nahg8jlfStFhFvS4l98uPc8G&gir=yes&mt=1634321805&fvip=3&keepalive=yes&fexp=24001373%2C24007246&c=WEB&n=E9C3zgNpf4KRtvGBA&sparams=expire%2Cei%2Cip%2Cid%2Caitags%2Csource%2Crequiressl%2Cvprv%2Clive%2Chang%2Cnoclen%2Cmime%2Cns%2Cgir&sig=AOq0QJ8wRQIhANqH_YJ92vhoTPFvr_N7_KU2fcbwglY5omoek3xu0yh9AiAfKP4zMikr-KCepi-aNtiotp9sBnJ3h_M35KgegTPrCg%3D%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRAIgNX5LGVBVThMykWuwkbukMWprE7xW35aafuA2eonSTvkCIB-ugVxlxSKm8W1p7OsstCLik7O8smDOT15v4xhMQ5hM&sq=0
2021/10/16 01:22:46 DEBUG: seqUrl from http.NewRequest
2021/10/16 01:22:46 DEBUG: https://r3---sn-n3toxu-axqs.googlevideo.com/videoplayback?expire=1634343765&ei=9cZpYc_cGZflyQWwpJuYDQ&ip=[my_ip]&id=R7vrbiDi0Tc.1&itag=248&aitags=242%2C243%2C244%2C247%2C248%2C271%2C278%2C313&source=yt_live_broadcast&requiressl=yes&mh=hi&mm=44%2C29&mn=sn-n3toxu-axqs%2Csn-5goeen7y&ms=lva%2Crdu&mv=m&mvi=3&pl=21&initcwndbps=1230000&vprv=1&live=1&hang=1&noclen=1&mime=video%2Fwebm&ns=Nahg8jlfStFhFvS4l98uPc8G&gir=yes&mt=1634321805&fvip=3&keepalive=yes&fexp=24001373%2C24007246&c=WEB&n=E9C3zgNpf4KRtvGBA&sparams=expire%2Cei%2Cip%2Cid%2Caitags%2Csource%2Crequiressl%2Cvprv%2Clive%2Chang%2Cnoclen%2Cmime%2Cns%2Cgir&sig=AOq0QJ8wRQIgenNM60ebkcyvTA95ORwBCjoc3LwJVR886_LoGrgNnzECIQDHEAdC4hMYopkxbeM4egqQ-ssCB-sXKo2X0RxCb0JFVw%3D%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRAIgNX5LGVBVThMykWuwkbukMWprE7xW35aafuA2eonSTvkCIB-ugVxlxSKm8W1p7OsstCLik7O8smDOT15v4xhMQ5hM&sq=0
2021/10/16 01:22:46 DEBUG:
2021/10/16 01:22:46 DEBUG:
2021/10/16 01:22:46 DEBUG: video1: HTTP Error for fragment 0: 403 Forbidden
2021/10/16 01:22:46 DEBUG: baseUrl form HandleFragHttpError
2021/10/16 01:22:46 DEBUG: https://r3---sn-n3toxu-axqs.googlevideo.com/videoplayback?expire=1634343765&ei=9cZpYc_cGZflyQWwpJuYDQ&ip=[my_ip]&id=R7vrbiDi0Tc.1&itag=248&aitags=242%%2C243%%2C244%%2C247%%2C248%%2C271%%2C278%%2C313&source=yt_live_broadcast&requiressl=yes&mh=hi&mm=44%%2C29&mn=sn-n3toxu-axqs%%2Csn-5goeen7y&ms=lva%%2Crdu&mv=m&mvi=3&pl=21&initcwndbps=1230000&vprv=1&live=1&hang=1&noclen=1&mime=video%%2Fwebm&ns=Nahg8jlfStFhFvS4l98uPc8G&gir=yes&mt=1634321805&fvip=3&keepalive=yes&fexp=24001373%%2C24007246&c=WEB&n=E9C3zgNpf4KRtvGBA&sparams=expire%%2Cei%%2Cip%%2Cid%%2Caitags%%2Csource%%2Crequiressl%%2Cvprv%%2Clive%%2Chang%%2Cnoclen%%2Cmime%%2Cns%%2Cgir&sig=AOq0QJ8wRQIgenNM60ebkcyvTA95ORwBCjoc3LwJVR886_LoGrgNnzECIQDHEAdC4hMYopkxbeM4egqQ-ssCB-sXKo2X0RxCb0JFVw%%3D%%3D&lsparams=mh%%2Cmm%%2Cmn%%2Cms%%2Cmv%%2Cmvi%%2Cpl%%2Cinitcwndbps&lsig=AG3C_xAwRAIgNX5LGVBVThMykWuwkbukMWprE7xW35aafuA2eonSTvkCIB-ugVxlxSKm8W1p7OsstCLik7O8smDOT15v4xhMQ5hM&sq=%d
2021/10/16 01:22:46 DEBUG:
2021/10/16 01:22:46 DEBUG: audio1: HTTP Error for fragment 0: 403 Forbidden
2021/10/16 01:22:46 DEBUG: baseUrl form HandleFragHttpError
2021/10/16 01:22:46 DEBUG: https://r3---sn-n3toxu-axqs.googlevideo.com/videoplayback?expire=1634343765&ei=9cZpYc_cGZflyQWwpJuYDQ&ip=[my_ip]&id=R7vrbiDi0Tc.1&itag=140&aitags=140&source=yt_live_broadcast&requiressl=yes&mh=hi&mm=44%%2C29&mn=sn-n3toxu-axqs%%2Csn-5goeen7y&ms=lva%%2Crdu&mv=m&mvi=3&pl=21&initcwndbps=1230000&vprv=1&live=1&hang=1&noclen=1&mime=audio%%2Fmp4&ns=Nahg8jlfStFhFvS4l98uPc8G&gir=yes&mt=1634321805&fvip=3&keepalive=yes&fexp=24001373%%2C24007246&c=WEB&n=E9C3zgNpf4KRtvGBA&sparams=expire%%2Cei%%2Cip%%2Cid%%2Caitags%%2Csource%%2Crequiressl%%2Cvprv%%2Clive%%2Chang%%2Cnoclen%%2Cmime%%2Cns%%2Cgir&sig=AOq0QJ8wRQIhANqH_YJ92vhoTPFvr_N7_KU2fcbwglY5omoek3xu0yh9AiAfKP4zMikr-KCepi-aNtiotp9sBnJ3h_M35KgegTPrCg%%3D%%3D&lsparams=mh%%2Cmm%%2Cmn%%2Cms%%2Cmv%%2Cmvi%%2Cpl%%2Cinitcwndbps&lsig=AG3C_xAwRAIgNX5LGVBVThMykWuwkbukMWprE7xW35aafuA2eonSTvkCIB-ugVxlxSKm8W1p7OsstCLik7O8smDOT15v4xhMQ5hM&sq=%d
2021/10/16 01:22:46 DEBUG: video: Attempting to retrieve a new download URL
2021/10/16 01:22:46 DEBUG:
2021/10/16 01:22:46 DEBUG: audio: Attempting to retrieve a new download URL
2021/10/16 01:22:51 DEBUG: seqUrl from http.NewRequest
2021/10/16 01:22:51 DEBUG: https://r3---sn-n3toxu-axqs.googlevideo.com/videoplayback?expire=1634343765&ei=9cZpYc_cGZflyQWwpJuYDQ&ip=[my_ip]&id=R7vrbiDi0Tc.1&itag=140&aitags=140&source=yt_live_broadcast&requiressl=yes&mh=hi&mm=44%2C29&mn=sn-n3toxu-axqs%2Csn-5goeen7y&ms=lva%2Crdu&mv=m&mvi=3&pl=21&initcwndbps=1230000&vprv=1&live=1&hang=1&noclen=1&mime=audio%2Fmp4&ns=Nahg8jlfStFhFvS4l98uPc8G&gir=yes&mt=1634321805&fvip=3&keepalive=yes&fexp=24001373%2C24007246&c=WEB&n=E9C3zgNpf4KRtvGBA&sparams=expire%2Cei%2Cip%2Cid%2Caitags%2Csource%2Crequiressl%2Cvprv%2Clive%2Chang%2Cnoclen%2Cmime%2Cns%2Cgir&sig=AOq0QJ8wRQIhANqH_YJ92vhoTPFvr_N7_KU2fcbwglY5omoek3xu0yh9AiAfKP4zMikr-KCepi-aNtiotp9sBnJ3h_M35KgegTPrCg%3D%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRAIgNX5LGVBVThMykWuwkbukMWprE7xW35aafuA2eonSTvkCIB-ugVxlxSKm8W1p7OsstCLik7O8smDOT15v4xhMQ5hM&sq=0
2021/10/16 01:22:51 DEBUG:
2021/10/16 01:22:51 DEBUG: seqUrl from http.NewRequest
2021/10/16 01:22:51 DEBUG: https://r3---sn-n3toxu-axqs.googlevideo.com/videoplayback?expire=1634343765&ei=9cZpYc_cGZflyQWwpJuYDQ&ip=[my_ip]&id=R7vrbiDi0Tc.1&itag=248&aitags=242%2C243%2C244%2C247%2C248%2C271%2C278%2C313&source=yt_live_broadcast&requiressl=yes&mh=hi&mm=44%2C29&mn=sn-n3toxu-axqs%2Csn-5goeen7y&ms=lva%2Crdu&mv=m&mvi=3&pl=21&initcwndbps=1230000&vprv=1&live=1&hang=1&noclen=1&mime=video%2Fwebm&ns=Nahg8jlfStFhFvS4l98uPc8G&gir=yes&mt=1634321805&fvip=3&keepalive=yes&fexp=24001373%2C24007246&c=WEB&n=E9C3zgNpf4KRtvGBA&sparams=expire%2Cei%2Cip%2Cid%2Caitags%2Csource%2Crequiressl%2Cvprv%2Clive%2Chang%2Cnoclen%2Cmime%2Cns%2Cgir&sig=AOq0QJ8wRQIgenNM60ebkcyvTA95ORwBCjoc3LwJVR886_LoGrgNnzECIQDHEAdC4hMYopkxbeM4egqQ-ssCB-sXKo2X0RxCb0JFVw%3D%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRAIgNX5LGVBVThMykWuwkbukMWprE7xW35aafuA2eonSTvkCIB-ugVxlxSKm8W1p7OsstCLik7O8smDOT15v4xhMQ5hM&sq=0
2021/10/16 01:22:51 DEBUG:
2021/10/16 01:22:51 DEBUG: video1: HTTP Error for fragment 0: 403 Forbidden
2021/10/16 01:22:51 DEBUG: baseUrl form HandleFragHttpError
2021/10/16 01:22:51 DEBUG: https://r3---sn-n3toxu-axqs.googlevideo.com/videoplayback?expire=1634343765&ei=9cZpYc_cGZflyQWwpJuYDQ&ip=[my_ip]&id=R7vrbiDi0Tc.1&itag=248&aitags=242%%2C243%%2C244%%2C247%%2C248%%2C271%%2C278%%2C313&source=yt_live_broadcast&requiressl=yes&mh=hi&mm=44%%2C29&mn=sn-n3toxu-axqs%%2Csn-5goeen7y&ms=lva%%2Crdu&mv=m&mvi=3&pl=21&initcwndbps=1230000&vprv=1&live=1&hang=1&noclen=1&mime=video%%2Fwebm&ns=Nahg8jlfStFhFvS4l98uPc8G&gir=yes&mt=1634321805&fvip=3&keepalive=yes&fexp=24001373%%2C24007246&c=WEB&n=E9C3zgNpf4KRtvGBA&sparams=expire%%2Cei%%2Cip%%2Cid%%2Caitags%%2Csource%%2Crequiressl%%2Cvprv%%2Clive%%2Chang%%2Cnoclen%%2Cmime%%2Cns%%2Cgir&sig=AOq0QJ8wRQIgenNM60ebkcyvTA95ORwBCjoc3LwJVR886_LoGrgNnzECIQDHEAdC4hMYopkxbeM4egqQ-ssCB-sXKo2X0RxCb0JFVw%%3D%%3D&lsparams=mh%%2Cmm%%2Cmn%%2Cms%%2Cmv%%2Cmvi%%2Cpl%%2Cinitcwndbps&lsig=AG3C_xAwRAIgNX5LGVBVThMykWuwkbukMWprE7xW35aafuA2eonSTvkCIB-ugVxlxSKm8W1p7OsstCLik7O8smDOT15v4xhMQ5hM&sq=%d
2021/10/16 01:22:51 DEBUG: audio1: HTTP Error for fragment 0: 403 Forbidden
2021/10/16 01:22:51 DEBUG: baseUrl form HandleFragHttpError
2021/10/16 01:22:51 DEBUG:
2021/10/16 01:22:51 DEBUG: video: Attempting to retrieve a new download URL
2021/10/16 01:22:51 DEBUG: https://r3---sn-n3toxu-axqs.googlevideo.com/videoplayback?expire=1634343765&ei=9cZpYc_cGZflyQWwpJuYDQ&ip=[my_ip]&id=R7vrbiDi0Tc.1&itag=140&aitags=140&source=yt_live_broadcast&requiressl=yes&mh=hi&mm=44%%2C29&mn=sn-n3toxu-axqs%%2Csn-5goeen7y&ms=lva%%2Crdu&mv=m&mvi=3&pl=21&initcwndbps=1230000&vprv=1&live=1&hang=1&noclen=1&mime=audio%%2Fmp4&ns=Nahg8jlfStFhFvS4l98uPc8G&gir=yes&mt=1634321805&fvip=3&keepalive=yes&fexp=24001373%%2C24007246&c=WEB&n=E9C3zgNpf4KRtvGBA&sparams=expire%%2Cei%%2Cip%%2Cid%%2Caitags%%2Csource%%2Crequiressl%%2Cvprv%%2Clive%%2Chang%%2Cnoclen%%2Cmime%%2Cns%%2Cgir&sig=AOq0QJ8wRQIhANqH_YJ92vhoTPFvr_N7_KU2fcbwglY5omoek3xu0yh9AiAfKP4zMikr-KCepi-aNtiotp9sBnJ3h_M35KgegTPrCg%%3D%%3D&lsparams=mh%%2Cmm%%2Cmn%%2Cms%%2Cmv%%2Cmvi%%2Cpl%%2Cinitcwndbps&lsig=AG3C_xAwRAIgNX5LGVBVThMykWuwkbukMWprE7xW35aafuA2eonSTvkCIB-ugVxlxSKm8W1p7OsstCLik7O8smDOT15v4xhMQ5hM&sq=%d
2021/10/16 01:22:51 DEBUG:
2021/10/16 01:22:51 DEBUG: audio: Attempting to retrieve a new download URL
2021/10/16 01:22:55 WARNING: User Interrupt, Stopping download...
2021/10/16 01:22:56 DEBUG: video1: exiting
2021/10/16 01:22:56 DEBUG: audio1: exiting
2021/10/16 01:22:56 DEBUG: audio-download thread closing
2021/10/16 01:22:56 DEBUG: video-download thread closing
Kethsar commented 3 years ago

Oh. Okay. ytarchive does not currently support streams that have been on for longer than 12 hours. I plan to add support for it, but beyond 12 hours (typically anyway), youtube stops supporting VOD archival for the stream and starts deleting fragments.

What this means is that even when I do add support, it will only manage to grab fragments from 12 hours before you start the download and beyond.

user334 commented 3 years ago

But... I have already downloaded streams which lasts a lot over 12 hours. I have a solid 100-hours file of the stream I'm interested in. And now I was trying to grab a fresh data from there but this fail occurs. Maybe it's possible to just skip the completely missed parts and download all of those we still can? It'd be very usefull to have some time selector in that and other cases. Something like showing user the available range of time (or "may be available" like in this case) and letting them to select the exact timings. Hope it is possible to do.

PS. Thinking of 100hrs stream I'd like to inform that I used --threads 50 with --no-fragmenting and no error occurred at all. No 401s nor a memory leak or overload.

Kethsar commented 3 years ago

From what I can tell the 401s definitely vary from user to user. I know someone else that was also able to do 50 threads fine, but I've never managed above 8 before getting 401d. I was also kinda curious about that before. Are you certain that 100 hour stream you downloaded hadn't started less than 12 hours before you started its download? If it was past 12 hours since it started, you may have have gotten lucky.

As for a time selector, it's planned, though maybe not in such a hand-holdy way as you described. See #36

user334 commented 3 years ago

I was also kinda curious about that before. Are you certain that 100 hour stream you downloaded hadn't started less than 12 hours before you started its download? If it was past 12 hours since it started, you may have have gotten lucky.

Your questions makes me uncertain :thinking: So what is a start of a stream? And how a 100-hrs stream can be both lasts over 12 hrs and be started in less then 12-hrs at the same time? Here's the facts:

  1. YouTube page of this stream says that it has started October 7-th
  2. YouTube player timeline changes: sometimes it's like couple of hours back and sometimes it's a full 12 hours
  3. When I firstly downloaded this stream on Oct. 7 I got like 10 hrs of video. I thought that we only have time the timeline shows to us so next day I've tried to get a fresh portion of data and got a whole two days of video. From that point I've started to think YouTube saves a lot more than it shows to us in it's player. So then I did a complete redownload of the stream every next day (and this is the point where we need to have a time selector) and every time it started from the same point in time just the file got bigger and bigger. The last success attempt was the night before opening this issue. I've got a solid 30 gigs of data (with a crazy 8 gigs of audio wink-wink) which starts the same Oct. 7 and lasts 111 hrs which is almost five days in a row.
  4. After that I was unable to download any portion of my stream with the error I've started this issue with.
Kethsar commented 3 years ago

And how a 100-hrs stream can be both lasts over 12 hrs and be started in less then 12-hrs at the same time?

How long a stream lasts doesn't matter, apparently. I was told that someone had ytarchive download a stream for over 2 weeks before. What matters is when the stream started and when you start the download attempt. If the stream has already been active for more than 12 hours by the time you try to start the download, that is when it no longer works.

Now then, why are you stopping and re-starting the download of a stream if you want to continue downloading it? Is ytarchive closing on its own before the stream has finished? That you were able to re-download each of those times is probably pure luck. Every stream that was beyond 12 hours that I have tried to download has failed.

user334 commented 3 years ago

What matters is when the stream started and when you start the download attempt. If the stream has already been active for more than 12 hours by the time you try to start the download, that is when it no longer works

I'm... still don't understand where is a start of a stream to count from. It's not a time YouTube page shows us as a "start of a stream" neither a length of an actual stream? So what is it? Where to count from? Now it seems so fuzzy :thinking: Is there any behind the scene switch or something?

Every stream that was beyond 12 hours that I have tried to download has failed

I've searched some random streams on YT and tried to download them. The result is I was able to download streams after Oct. 11 and unable to download streams before Oct. 8. So.. May be the limit we're talking about is not an hours but a couple of days?

That you were able to re-download each of those times is probably pure luck

I don't personally think there were some magic "restart" button on the other side of the stream I was downloading people there were clicking on every 12h. Sounds so strange.

Now then, why are you stopping and re-starting the download of a stream if you want to continue downloading it? Is ytarchive closing on its own before the stream has finished?

No. ytarchive works flawlessly. It downloads streams continuously as it should. It's just my workflow. To be more precise in that exact case I'm deciding whether I'm gonna download a stream or not after a day. That's why it'd be so great to have a time selector :)

user334 commented 3 years ago

Most of the old streams are giving 404 so them are unable to be downloaded at all :( Maybe there is a way to download an available\most recent parts? Maybe via just skipping the 404-ones?

user334 commented 3 years ago

Hi! After a week of downloading of the stream ytarchive started to give me 404's again.. So I've started to dig Google again. And I've found a program that is able to download all of the stream from the very start and has a proper (but not so convenient) time seeking. It's written on C# and is available only for Windows but I think you can cherry-pick some logic from there for this project. Here it is: https://github.com/rytsikau/ee.Yrewind

UPD: oh, okay, so I've read the description for that program now and it's not an opensource one. At least not fully. And it uses web-browser to communicate with YouTube. Seems not fun yet it works. Maybe we can ask the developer to share some code or just mitm the actual requests.

leavittx commented 2 years ago

Voting for this feature! Skipping fragments with >12h age would be absolutely fine for me too! ytarchive is great software!

Kethsar commented 1 year ago

Fixed in v0.3.2