search

faissaloo / SponSkrub

Strip advertisements from downloaded YouTube videos
GNU General Public License v3.0
178 stars 7 forks source link

Got 400 the server must be broken, try again later #30

Closed bharathyes closed 3 years ago

bharathyes commented 3 years ago

Utilising sponskrub either via yt-dlp or directly is throwing the error:

$ sponskrub -- eYq7WapuDLU 'Dhee ft. Arivu - Enjoy Enjaami (Prod. Santhosh Narayanan) [eYq7WapuDLU].mkv' 'test.mkv'
Downloading video sponsor data
Got 400 the server must be broken, try again later

I checked my DNS requests and there is a valid response given for the API.

pukkandan commented 3 years ago

Just to make sure the issue isn't caused by some mistake I made in compilation, can you test it again with this version of sponskrub (instead of mine)?. You can install it using:

sudo curl -L https://github.com/faissaloo/SponSkrub/releases/latest/download/sponskrub -o /usr/local/bin/sponskrub
sudo chmod a+rx /usr/local/bin/sponskrub
bharathyes commented 3 years ago

@pukkandan Ok. This binary works, or at least it is connecting to the API. 

$ yt-dlp -f 335+251/399+251/248+251/137+251/334+251/398+251/247+251/333+251/397+251/244+251/bestvideo+bestaudio --sponskrub-cut --sponskrub-force "OCg6BWlAXSw"
[youtube] OCg6BWlAXSw: Downloading webpage
[info] Downloading format(s) 399+251
WARNING: Requested formats are incompatible for merge and will be merged into mkv.
[download] #AlaVaikunthapurramuloo - Samajavaragamana Full Video Song (4K) _ Allu Arjun _ Trivikram _ Thaman S [OCg6BWlAXSw].mkv has already been downloaded
[SponSkrub] Trying to remove sponsor sections
WARNING: Cutting out sponsor segments will cause the subtitles to go out of sync.
WARNING: If sponskrub is run multiple times, unintended parts of the video could be cut out.
[SponSkrub] No segments in the SponsorBlock database

Checked with a couple of videos with non-music parts marked on the browser. Here it just says no segments in database. I assume even non-music segments will be skipped since I have not used any exclude flags.

pukkandan commented 3 years ago

Ok. This binary works, or at least it is connecting to the API.

must be something wrong with my build then.

I assume even non-music segments will be skipped since I have not used any exclude flags.

This version has different defaults and switches than mine. Run sponskrub -h for details

@faissaloo Sorry for sending this issue your way when it seems like it was in fact an issue from my side.

I would appreciate if you could have a look at my fork and see if I am doing anything wrong for this to happen. I don't see any relevant differences b/w this and my fork. Could it be an issue with how I am building the executable? For reference, I built the releases using https://github.com/yt-dlp/SponSkrub/blob/master/build.sh

I know next to nothing about DLang, so any help is appreciated

bharathyes commented 3 years ago

@pukkandan I am unsure of what I am doing at this point. To update, I only ran the new binary via yt-dlp using old flags which gave no ads in db. Seems it only removes sponsors and not nonmusic. I have no idea how to pass sponskrub flags via yt-dlp.

Working for sponsors:

$ yt-dlp -f 335+251/399+251/248+251/137+251/334+251/398+251/247+251/333+251/397+251/244+251/bestvideo+bestaudio --sponskrub-cut "6fsrLOgl69U"
[youtube] 6fsrLOgl69U: Downloading webpage
[info] Downloading format(s) 399+251
WARNING: Requested formats are incompatible for merge and will be merged into mkv.
[download] Destination: Fire Epoxy Table - Woodworking Projects - Resin Art [6fsrLOgl69U].f399.mp4
[download] 100% of 196.96MiB in 00:12
[download] Destination: Fire Epoxy Table - Woodworking Projects - Resin Art [6fsrLOgl69U].f251.webm
[download] 100% of 12.44MiB in 00:00
[Merger] Merging formats into "Fire Epoxy Table - Woodworking Projects - Resin Art [6fsrLOgl69U].mkv"
Deleting original file Fire Epoxy Table - Woodworking Projects - Resin Art [6fsrLOgl69U].f399.mp4 (pass -k to keep)
Deleting original file Fire Epoxy Table - Woodworking Projects - Resin Art [6fsrLOgl69U].f251.webm (pass -k to keep)
[SponSkrub] Trying to remove sponsor sections
WARNING: Cutting out sponsor segments will cause the subtitles to go out of sync.
frame=22973 fps= 24 q=-1.0 Lsize=  415790kB time=00:12:45.79 bitrate=4447.8kbits/s speed=0.816x
[SponSkrub] Sponsor sections have been removed

Non-music not detected via yt-dlp. I tried passing -include-nonmusic flag to this but only got an error yt-dlp: error: using .netrc conflicts with giving username/password

$ sponskrub OCg6BWlAXSw '#AlaVaikunthapurramuloo - Samajavaragamana Full Video Song (4K) _ Allu Arjun _ Trivikram _ Thaman S [OCg6BWlAXSw].mkv' '
result.mkv'
Downloading video sponsor data
This video has no ad information available, either it has no ads or no one has logged any on SponsorBlock yet.

Now calling sponskrub directly still doesn't seem to work on non-music parts. (-include-nonmusic was a bit ambiguous for me so I ran with and without not knowing which should remove the segments. Neither seem to work for me.)

$ sponskrub OCg6BWlAXSw '#AlaVaikunthapurramuloo - Samajavaragamana Full Video Song (4K) _ Allu Arjun _ Trivikram _ Thaman S [OCg6BWlAXSw].mkv' '
result.mkv'
Downloading video sponsor data
This video has no ad information available, either it has no ads or no one has logged any on SponsorBlock yet.
$ sponskrub -include-nonmusic OCg6BWlAXSw '#AlaVaikunthapurramuloo - Samajavaragamana Full Video Song (4K) _ Allu Arjun _ Trivikram _ Thaman S [O
Cg6BWlAXSw].mkv' 'result.mkv'
Downloading video sponsor data
Got 400 the server must be broken, try again later

Can you explain how to remove non-music segments and if there is a way to invoke the new sponskrub binaries along with its corresponding flags from yt-dlp. Sorry if I am being repetitive.

pukkandan commented 3 years ago

You can pass the arguments to sponskrub using --ppa "sponskrub:-include-nonmusic". So the whole command would be:

yt-dlp --sponskrub-cut --ppa "sponskrub:-include-nonmusic" "6fsrLOgl69U"
faissaloo commented 3 years ago

@pukkandan

I would appreciate if you could have a look at my fork and see if I am doing anything wrong for this to happen. I don't see any relevant differences b/w this and my fork. Could it be an issue with how I am building the executable? For reference, I built the releases using https://github.com/yt-dlp/SponSkrub/blob/master/build.sh

It's not immediately obvious to me that there's anything wrong with your fork, although it is out of date so I'd try pulling the latest changes from my version. It may be that I fixed a bug at some point while adding proxy support without realising

pukkandan commented 3 years ago

although it is out of date

It was not. But in any case, I have rebased my changes on top of your master. This should atleast make debugging easier

@bharathyes could you test with the new release

bharathyes commented 3 years ago

@pukkandan Still getting 400 the server must be broken error for both music videos and sponsor video ( sponsor segments in videos like 6fsrLOgl69U works with faissaloo version ).

$ sudo curl -L https://github.com/faissaloo/SponSkrub/releases/latest/download/sponskrub -o /usr/local/bin/sponskrub
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   138  100   138    0     0    266      0 --:--:-- --:--:-- --:--:--   266
100   617  100   617    0     0    761      0 --:--:-- --:--:-- --:--:--  150k
100 5750k  100 5750k    0     0  1502k      0  0:00:03  0:00:03 --:--:-- 2977k
$ sudo chmod a+rx /usr/local/bin/sponskrub
$ sponskrub -chapter 6fsrLOgl69U 'Fire Epoxy Table - Woodworking Projects - Resin Art [6fsrLOgl69U].mkv' 'result.mkv'
Downloading video sponsor data
Marking the shilling...
frame=25694 fps=0.0 q=-1.0 Lsize=  214338kB time=00:14:16.52 bitrate=2050.0kbits/s speed=1.22e+03x
Done!
$ sudo curl -L https://github.com/yt-dlp/SponSkrub/releases/latest/download/sponskrub -o /usr/local/bin/sponskrub
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   140  100   140    0     0    379      0 --:--:-- --:--:-- --:--:--   379
100   617  100   617    0     0    932      0 --:--:-- --:--:-- --:--:--   932
100 5050k  100 5050k    0     0  1381k      0  0:00:03  0:00:03 --:--:-- 2035k
$ sudo chmod a+rx /usr/local/bin/sponskrub
$ sponskrub -chapter 6fsrLOgl69U 'Fire Epoxy Table - Woodworking Projects - Resin Art [6fsrLOgl69U].mkv' 'result.mkv'
Downloading video sponsor data
Got 400 the server must be broken, try again later
faissaloo commented 3 years ago

@bharathyes I notice you're using the builds on the release pages, I wonder if you would have the same problem when building both from source? According to https://stackoverflow.com/questions/46635748/process-for-dlang-static-linking dlang statically links against libcurl on Linux, so it might be that @pukkandan 's machine has a slightly broken version of libcurl which gets linked to?

I've built a copy of @pukkandan 's fork here: https://files.catbox.moe/pmlelk Let me know if it fixes the issue.

bharathyes commented 3 years ago

@faissaloo Still getting the same error with this binary too. BTW, is there a way to get logs of the error occurring? Like a verbose mode?

faissaloo commented 3 years ago

@bharathyes Not currently, I've been thinking of adding a verbose mode at some point but I've not gotten round to it. If that binary is still broken I'm not quite sure what the issue is.

pukkandan commented 3 years ago

Since this was an issue only with my fork and the fork has now been archived, it may be best to close this issue