Support for auto-translated YouTube closed-captions

trizen commented 2 years ago

Checklist

[X] I made sure that there are no existing issues - open or closed - which I could contribute my information to.
[X] I'm aware that this is a request for NewPipe itself and that requests for adding a new service need to be made at NewPipeExtractor.
[X] I have taken the time to fill in all the required details. I understand that the feature request will be dismissed otherwise.
[X] This issue contains only one feature request.
[X] I have read and understood the contribution guidelines.

Feature description

Support for auto-translated YouTube closed-captions.

This can be achieved by adding the tlang=LANG URI parameter to the provided closed-captions URLs (including the auto-generated ones), where LANG is the language code in which we want to translate the closed-caption into.

The list of accepted language codes, is:

      af am ar az be bg bn bs ca ceb co cs cy da de el en eo es et eu fa fi fil
      fr fy ga gd gl gu ha haw hi hmn hr ht hu hy id ig is it iw ja jv ka kk km
      kn ko ku ky la lb lo lt lv mg mi mk ml mn mr ms mt my ne nl no ny or pa pl
      ps pt ro ru rw sd si sk sl sm sn so sq sr st su sv sw ta te tg th tk tr tt
      ug uk ur uz vi xh yi yo zh-Hans zh-Hant zu

Example (auto-generated closed-caption in English):

https://www.youtube.com/api/timedtext?v=HiMaSrXaFz0&asr_langs=de,en,es,fr,id,it,ja,ko,nl,pt,ru,tr,vi&caps=asr&exp=xctw&xoaf=5&hl=en&ip=0.0.0.0&ipbits=0&expire=1647400208&sparams=ip,ipbits,expire,v,asr_langs,caps,exp,xoaf&signature=1D31C0E268469B8454678AE346B6AE2B040B02EB.7924D7A5830FB2DDFB22F1CE281AB155BD82ACEA&key=yt8&kind=asr&lang=en&fmt=srv1

Closed-caption auto-translated into Romanian:

https://www.youtube.com/api/timedtext?v=HiMaSrXaFz0&asr_langs=de,en,es,fr,id,it,ja,ko,nl,pt,ru,tr,vi&caps=asr&exp=xctw&xoaf=5&hl=en&ip=0.0.0.0&ipbits=0&expire=1647400208&sparams=ip,ipbits,expire,v,asr_langs,caps,exp,xoaf&signature=1D31C0E268469B8454678AE346B6AE2B040B02EB.7924D7A5830FB2DDFB22F1CE281AB155BD82ACEA&key=yt8&kind=asr&lang=en&fmt=srv1&tlang=ro

This feature can be added as a settings option, where the user inserts the language code that he wants the closed-captions to be translated into, and the closed-caption menu would display an option for selecting the auto-translated caption(s) with the language-code(s) provided by the user.

Why do you want this feature?

This feature is useful for people that do not understand the English language. As most YouTube videos have auto-generated captions, providing auto-translated captions allows people that do not understand the language spoken into the video to be able to understand (to some extent) what is being said in the video.

Additional information

No response

SameenAhnaf commented 2 years ago

This feature can be added as a settings option, where the user inserts the language code that he wants the closed-captions to be translated into

I don't think any option is required to be kept in settings. Add a secondary menu to Auto-translated > in captions menu. Previously selected output languages should be on top. Other languages should be kept in the bottom alphabetically.

trizen commented 2 years ago

As there don't seem to be any plans for adding this feature in the near future, I present below a patch that enables support for auto-translated subtitles for a specific language-code specified at compile-time.

Maybe someone can extend this patch and allow the language-code to be changed during the runtime (in the settings).

Patch for NewPipe:

diff --git a/gradle/wrapper/gradle-wrapper.properties b/gradle/wrapper/gradle-wrapper.properties
index 4ed3bde..92f06b5 100644
--- a/gradle/wrapper/gradle-wrapper.properties
+++ b/gradle/wrapper/gradle-wrapper.properties
@@ -1,6 +1,5 @@
 distributionBase=GRADLE_USER_HOME
 distributionPath=wrapper/dists
 distributionUrl=https\://services.gradle.org/distributions/gradle-7.4.2-all.zip
-distributionSha256Sum=e6d864e3b5bc05cc62041842b306383fc1fefcec359e70cebb1d470a6094ca82
 zipStoreBase=GRADLE_USER_HOME
 zipStorePath=wrapper/dists
diff --git a/settings.gradle b/settings.gradle
index 0338fde..a0aed95 100644
--- a/settings.gradle
+++ b/settings.gradle
@@ -4,8 +4,8 @@ include ':app'
 // We assume, that NewPipe and NewPipe Extractor have the same parent directory.
 // If this is not the case, please change the path in includeBuild().

-//includeBuild('../NewPipeExtractor') {
-//    dependencySubstitution {
-//        substitute module('com.github.TeamNewPipe:NewPipeExtractor') using project(':extractor')
-//    }
-//}
+includeBuild('../NewPipeExtractor') {
+    dependencySubstitution {
+        substitute module('com.github.TeamNewPipe:NewPipeExtractor') using project(':extractor')
+    }
+}

Patch for NewPipeExtractor:

diff --git a/extractor/src/main/java/org/schabi/newpipe/extractor/services/youtube/extractors/YoutubeStreamExtractor.java b/extractor/src/main/java/org/schabi/newpipe/extractor/services/youtube/extractors/YoutubeStreamExtractor.java
index a5cb04b..bb4f21b 100644
--- a/extractor/src/main/java/org/schabi/newpipe/extractor/services/youtube/extractors/YoutubeStreamExtractor.java
+++ b/extractor/src/main/java/org/schabi/newpipe/extractor/services/youtube/extractors/YoutubeStreamExtractor.java
@@ -657,6 +657,15 @@ public class YoutubeStreamExtractor extends StreamExtractor {
                         .setLanguageCode(languageCode)
                         .setAutoGenerated(isAutoGenerated)
                         .build());
+
+                if (i == 0) {
+                    subtitlesToReturn.add(new SubtitlesStream.Builder()
+                        .setContent(cleanUrl + "&fmt=" + format.getSuffix() + "&tlang=ro", true)
+                        .setMediaFormat(format)
+                        .setLanguageCode("ro")
+                        .setAutoGenerated(isAutoGenerated)
+                        .build());
+                }
             }
         }

(replace the language-code "ro" with a language-code of your choice)

Now a debug app can be built with Android Studio. Here's the result: newpipe-with-translated-subtitles

crti commented 1 year ago

It would be wonderful if auto-translated captions (preference by setup in settings) could be selected trough GUI instead of only the language in the video.

neofeo commented 3 weeks ago

Would love to have this feature!

TeamNewPipe / NewPipe