TeamNewPipe / NewPipeExtractor

NewPipe's core library for extracting data from streaming sites
GNU General Public License v3.0
1.39k stars 419 forks source link

[YouTube] Adjust throttling function extraction to changes #1191

Closed XiangRongLin closed 4 months ago

XiangRongLin commented 4 months ago

Using the info from https://github.com/TeamNewPipe/NewPipeExtractor/issues/1190

it should resolve that issue and https://github.com/TeamNewPipe/NewPipe/issues/11255

The YoutubeThrottlingParameterDeobfuscationTest are working again with this. The failing ones are bandcamp

newpipe-debug.zip

XiangRongLin commented 4 months ago

And boy did I not look at this in forever or even develop anything on my own PC. Getting everything running again was a challenge 😅

XiangRongLin commented 4 months ago

Seeing this here about A/B testing I'm wondering if we still need to support the old format.

FireMasterK commented 4 months ago

Seeing this here about A/B testing I'm wondering if we still need to support the old format.

Was your changes based on https://github.com/yt-dlp/yt-dlp/pull/10390? I think they have tests that work even for previous versions of the player code.

XiangRongLin commented 4 months ago

Seeing this here about A/B testing I'm wondering if we still need to support the old format.

Was your changes based on https://github.com/yt-dlp/yt-dlp/pull/10390? I think they have tests that work even for previous versions of the player code.

No, it was solely based on #1190

Theta-Dev commented 4 months ago

I don't think should throw out the old extraction function since that may break things for users still getting the old script (or in case they choose to revert these changes).

We already use a list of regexes to get the signature decryption function, we should do that for the nsig too.

Stypox commented 4 months ago

I can confirm it works. Thank you a lot!

We already use a list of regexes to get the signature decryption function, we should do that for the nsig too.

Yeah, I'll work on this rn.

Stypox commented 4 months ago

Done, and tested on this APK which works: app-debug.zip

@XiangRongLin or @Theta-Dev please look at the code, and if it looks good to you we can proceed with merging and releasing a hotfix.

Dhusha6 commented 4 months ago

Newpipe encountered networkerror Couldn't open youtube videos

NewMacPro commented 4 months ago

@XiangRongLin yes, error

org.schabi.newpipe.extractor.exceptions.ContentNotAvailableException: Got error: "Sign in to confirm that you're not a bot" at org.schabi.newpipe.extractor.services.youtube.extractors.YoutubeStreamExtractor.checkPlayabilityStatus(YoutubeStreamExtractor.java:948) at org.schabi.newpipe.extractor.services.youtube.extractors.YoutubeStreamExtractor.onFetchPage(YoutubeStreamExtractor.java:844) at org.schabi.newpipe.extractor.Extractor.fetchPage(Extractor.java:60) at org.schabi.newpipe.extractor.stream.StreamInfo.getInfo(StreamInfo.java:77) at org.schabi.newpipe.extractor.stream.StreamInfo.getInfo(StreamInfo.java:72) at org.schabi.newpipe.util.ExtractorHelper.lambda$getStreamInfo$3(ExtractorHelper.java:118) at org.schabi.newpipe.util.ExtractorHelper.$r8$lambda$YTHJjScxCJNO1LTCqs3IKy35iyY(ExtractorHelper.java:0) at org.schabi.newpipe.util.ExtractorHelper$$ExternalSyntheticLambda3.call(R8$$SyntheticClass:0) at io.reactivex.rxjava3.internal.operators.single.SingleFromCallable.subscribeActual(SingleFromCallable.java:43) at io.reactivex.rxjava3.core.Single.subscribe(Single.java:4855) at io.reactivex.rxjava3.internal.operators.single.SingleDoOnSuccess.subscribeActual(SingleDoOnSuccess.java:35) at io.reactivex.rxjava3.core.Single.subscribe(Single.java:4855) at io.reactivex.rxjava3.internal.operators.maybe.MaybeFromSingle.subscribeActual(MaybeFromSingle.java:41) at io.reactivex.rxjava3.core.Maybe.subscribe(Maybe.java:5377) at io.reactivex.rxjava3.internal.operators.maybe.MaybeConcatArray$ConcatMaybeObserver.drain(MaybeConcatArray.java:153) at io.reactivex.rxjava3.internal.operators.maybe.MaybeConcatArray$ConcatMaybeObserver.request(MaybeConcatArray.java:78) at io.reactivex.rxjava3.internal.operators.flowable.FlowableElementAtMaybe$ElementAtSubscriber.onSubscribe(FlowableElementAtMaybe.java:66) at io.reactivex.rxjava3.internal.operators.maybe.MaybeConcatArray.subscribeActual(MaybeConcatArray.java:42) at io.reactivex.rxjava3.core.Flowable.subscribe(Flowable.java:16144) at io.reactivex.rxjava3.internal.operators.flowable.FlowableElementAtMaybe.subscribeActual(FlowableElementAtMaybe.java:36) at io.reactivex.rxjava3.core.Maybe.subscribe(Maybe.java:5377) at io.reactivex.rxjava3.internal.operators.maybe.MaybeToSingle.subscribeActual(MaybeToSingle.java:46) at io.reactivex.rxjava3.core.Single.subscribe(Single.java:4855) at io.reactivex.rxjava3.internal.operators.single.SingleSubscribeOn$SubscribeOnObserver.run(SingleSubscribeOn.java:89) at io.reactivex.rxjava3.core.Scheduler$DisposeTask.run(Scheduler.java:644) at io.reactivex.rxjava3.internal.schedulers.ScheduledRunnable.run(ScheduledRunnable.java:65) at io.reactivex.rxjava3.internal.schedulers.ScheduledRunnable.call(ScheduledRunnable.java:56) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:301) at java.util.concurrent.ThreadPoolExecutor.processTask(ThreadPoolExecutor.java:1187) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641) at java.lang.Thread.run(Thread.java:784)


</details>