search

brave / brave-browser

Brave browser for Android, iOS, Linux, macOS, Windows.
https://brave.com
Mozilla Public License 2.0
17.83k stars 2.33k forks source link

Toggling `Live caption` via media-playback button dynamically downloads Google SODA components #15864

Open stephendonner opened 3 years ago

stephendonner commented 3 years ago

Description

Toggling Live caption via media-playback button dynamically downloads Google SODA components; this is new in the Chromium 91 major bump - we need to figure out how we want to handle this

Steps to Reproduce

  1. new profile
  2. play https://www.youtube.com/watch?v=kwAGY1PwMkE
  3. click on the media-control menu item on the right-hand side of the toolbar
  4. toggle Live Caption to On
  5. look at your terminal/console window

Actual result:

live-caption

Looks like this downloads Google's SODA (Speech On-Device API) components, dynamically:

[6492:1116:0514/082151.546:VERBOSE1:network_delegate.cc(32)] NetworkDelegate::NotifyBeforeURLRequest: https://www.youtube.com/api/stats/watchtime?ns=yt&el=detailpage&cpn=PaQe6mmwRVE79tyy&docid=kwAGY1PwMkE&ver=2&referrer=https%3A%2F%2Fwww.youtube.com%2Fresults%3Fsearch_query%3Djonathan%2Bsampson&cmt=136.601&ei=DZWeYIvwNs2Dkwavnq3IDw&fmt=134&fs=0&rt=137&of=wbAUJoLFDaeeOSJrHXH1sg&euri&lact=3412&cl=372995990&state=playing&vm=CAEQABgEKiAwdkUxUVZoUUY5M0NfYURuSzlRbzUwX053b3Y2WFg2OToyQU9HdF9PSzQwYkVsaTNZNjNjRXNGTERzdWdnZmhwamJzTnI2ZWRkMHh6dVVOei1vN2c&volume=100&cbr=Chrome&cbrver=91.0.4472.38&c=WEB&cver=2.20210512.06.00&cplayer=UNIPLAYER&cos=Windows&cosver=10.0&cplatform=DESKTOP&hl=en_US&cr=US&len=1648.281&rtn=177&feature=search&afmt=251&idpj=-1&ldpj=-14&rti=137&st=96.601&et=136.601&muted=0
[6492:1116:0514/082151.546:VERBOSE1:tls_handshaker.cc(96)] TlsHandshaker: continuing handshake
[6492:1116:0514/082151.573:VERBOSE1:tls_handshaker.cc(96)] TlsHandshaker: continuing handshake
[6492:1116:0514/082151.573:VERBOSE1:tls_handshaker.cc(96)] TlsHandshaker: continuing handshake
[6492:1116:0514/082151.574:VERBOSE1:tls_client_handshaker.cc(470)] Client: handshake finished
[6492:1116:0514/082155.001:VERBOSE1:network_delegate.cc(32)] NetworkDelegate::NotifyBeforeURLRequest: https://r4---sn-n4v7knl6.googlevideo.com/videoplayback?expire=1621027181&ei=DZWeYIvwNs2Dkwavnq3IDw&ip=99.152.113.108&id=o-ADGDADG68oPksEnAu0IeM4FdrNUwVM6g-2QXr2VNX4ry&itag=251&source=youtube&requiressl=yes&mh=Rf&mm=31%2C26&mn=sn-n4v7knl6%2Csn-a5mlrn7s&ms=au%2Conr&mv=m&mvi=4&pl=20&initcwndbps=1120000&vprv=1&mime=audio%2Fwebm&ns=MGv1UU8Ye9SARsiTk4nIcegF&gir=yes&clen=26249892&otfp=1&dur=1648.281&lmt=1575389686247419&mt=1621004696&fvip=4&keepalive=yes&fexp=24001373%2C24007246&c=WEB&txp=2211222&n=zO1ysO_CI9s7fA&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cvprv%2Cmime%2Cns%2Cgir%2Cclen%2Cotfp%2Cdur%2Clmt&sig=AOq0QJ8wRQIhAPkD_9VAx-HIOLqSMdfrsTVcyFAgZEoKpb2QtFHd8_jKAiAeIL_O0azsOiehp2iN6FQgI6ERQBI6OnNIdYfdHUKsyA%3D%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRAIgNOmVEXzF982XNOoYiS2QSqdp_JYjAmHB5PSdLhokbs8CIFOj6HB-azpcSiC0P2JCsYCAx_61_HESy-GwJa4pHPD2&alr=yes&cpn=PaQe6mmwRVE79tyy&cver=2.20210512.06.00&range=4112753-4597340&rn=23&rbuf=119945
[6988:14308:0514/082155.254:VERBOSE1:component_installer.cc(267)] StartRegistration for SODA Library
[6988:14780:0514/082155.254:VERBOSE1:component_installer.cc(267)] StartRegistration for SODA en-US Models
[6988:9976:0514/082155.255:VERBOSE1:component_installer.cc(411)] FinishRegistration for SODA Library
[6988:9976:0514/082155.256:VERBOSE1:component_installer.cc(411)] FinishRegistration for SODA en-US Models
[6492:1116:0514/082155.257:VERBOSE1:network_delegate.cc(32)] NetworkDelegate::NotifyBeforeURLRequest: https://go-updater.brave.com/extensions
[6492:1116:0514/082155.257:VERBOSE1:network_delegate.cc(32)] NetworkDelegate::NotifyBeforeURLRequest: https://go-updater.brave.com/extensions
[6492:1116:0514/082155.294:VERBOSE1:network_delegate.cc(32)] NetworkDelegate::NotifyBeforeURLRequest: https://componentupdater.brave.com/service/update2/json
[6492:1116:0514/082155.296:VERBOSE1:network_delegate.cc(32)] NetworkDelegate::NotifyBeforeURLRequest: https://componentupdater.brave.com/service/update2/json
[6988:9976:0514/082155.346:VERBOSE1:request_sender.cc(181)] Request completed from url: https://update.googleapis.com/service/update2/json
[6492:1116:0514/082155.347:VERBOSE1:network_delegate.cc(32)] NetworkDelegate::NotifyBeforeURLRequest: https://crlsets.brave.com/edgedl/release2/chrome_component/AqV3dEfu7BD5b9UQsUmlNA_1.0.7/AN4h7FThv35v2FE8zZO1BHs
[6988:9976:0514/082155.348:VERBOSE1:request_sender.cc(181)] Request completed from url: https://update.googleapis.com/service/update2/json
[6492:1116:0514/082155.349:VERBOSE1:network_delegate.cc(32)] NetworkDelegate::NotifyBeforeURLRequest: https://crlsets.brave.com/edgedl/release2/chrome_component/KER6p0Gr9-8E8ZmiXE7F4A_1.0.0/ALoRDWbCNFaNcA17KLAsXzQ
[6988:9976:0514/082155.358:VERBOSE1:url_fetcher_downloader.cc(136)] url fetcher response started for: http://edgedl.me.gvt1.com/edgedl/release2/chrome_component/AqV3dEfu7BD5b9UQsUmlNA_1.0.7/AN4h7FThv35v2FE8zZO1BHs
[6988:9976:0514/082155.407:VERBOSE1:url_fetcher_downloader.cc(136)] url fetcher response started for: http://edgedl.me.gvt1.com/edgedl/release2/chrome_component/KER6p0Gr9-8E8ZmiXE7F4A_1.0.0/ALoRDWbCNFaNcA17KLAsXzQ
[6492:1116:0514/082156.009:VERBOSE1:network_delegate.cc(32)] NetworkDelegate::NotifyBeforeURLRequest: https://i.ytimg.com/an_webp/eGoSdLNceEk/mqdefault_6s.webp?du=3000&sqp=CLqZ-oQG&rs=AOn4CLACq8V-ALZRLmtxwi2X8e3NMSHACA
[6988:9976:0514/082156.099:VERBOSE1:url_fetcher_downloader.cc(113)] Downloaded 20266623 bytes in 752ms from http://edgedl.me.gvt1.com/edgedl/release2/chrome_component/AqV3dEfu7BD5b9UQsUmlNA_1.0.7/AN4h7FThv35v2FE8zZO1BHs to C:\Users\Stephen\AppData\Local\Temp\chrome_url_fetcher_6988_1606468182\AN4h7FThv35v2FE8zZO1BHs
[6988:16568:0514/082156.158:VERBOSE1:component_unpacker.cc(59)] Verifying component: C:\Users\Stephen\AppData\Local\Temp\chrome_url_fetcher_6988_1606468182\AN4h7FThv35v2FE8zZO1BHs
[6988:16568:0514/082156.359:VERBOSE1:component_unpacker.cc(77)] Verification successful: C:\Users\Stephen\AppData\Local\Temp\chrome_url_fetcher_6988_1606468182\AN4h7FThv35v2FE8zZO1BHs
[6988:16568:0514/082156.360:VERBOSE1:component_unpacker.cc(90)] Unpacking in: C:\Users\Stephen\AppData\Local\Temp\6988_512607737
[6988:9976:0514/082156.654:VERBOSE1:url_fetcher_downloader.cc(113)] Downloaded 41780976 bytes in 1305ms from http://edgedl.me.gvt1.com/edgedl/release2/chrome_component/KER6p0Gr9-8E8ZmiXE7F4A_1.0.0/ALoRDWbCNFaNcA17KLAsXzQ to C:\Users\Stephen\AppData\Local\Temp\chrome_url_fetcher_6988_1603615579\ALoRDWbCNFaNcA17KLAsXzQ
[6988:8408:0514/082156.688:VERBOSE1:component_unpacker.cc(103)] Unpacked successfully
[6988:8408:0514/082156.691:VERBOSE1:component_installer.cc(119)] Install: version=1.0.7 current version=0.0.0.0
[6988:8408:0514/082156.691:VERBOSE1:component_installer.cc(137)] unpack_path=C:\Users\Stephen\AppData\Local\Temp\6988_512607737 install_path=C:\Users\Stephen\AppData\Local\BraveSoftware\Brave-Browser-Nightly\User Data\SODA\1.0.7
[6988:9976:0514/082156.694:VERBOSE1:component_installer.cc(454)] Component ready, version 1.0.7 in C:\Users\Stephen\AppData\Local\BraveSoftware\Brave-Browser-Nightly\User Data\SODA\1.0.7
[6988:9976:0514/082156.694:VERBOSE1:soda_component_installer.cc(143)] Component ready, version 1.0.7 in C:\Users\Stephen\AppData\Local\BraveSoftware\Brave-Browser-Nightly\User Data\SODA\1.0.7
[6988:9976:0514/082156.695:VERBOSE1:component_updater_service.cc(387)] Update completed with error 0
[6492:1116:0514/082156.695:VERBOSE1:network_delegate.cc(32)] NetworkDelegate::NotifyBeforeURLRequest: https://go-updater.brave.com/extensions
[6492:1116:0514/082156.754:VERBOSE1:network_delegate.cc(32)] NetworkDelegate::NotifyBeforeURLRequest: https://componentupdater.brave.com/service/update2/json
[6988:16568:0514/082156.762:VERBOSE1:component_unpacker.cc(59)] Verifying component: C:\Users\Stephen\AppData\Local\Temp\chrome_url_fetcher_6988_1603615579\ALoRDWbCNFaNcA17KLAsXzQ
[6988:9976:0514/082156.787:VERBOSE1:request_sender.cc(181)] Request completed from url: https://update.googleapis.com/service/update2/json
[6988:16568:0514/082157.136:VERBOSE1:component_unpacker.cc(77)] Verification successful: C:\Users\Stephen\AppData\Local\Temp\chrome_url_fetcher_6988_1603615579\ALoRDWbCNFaNcA17KLAsXzQ
[6988:16568:0514/082157.137:VERBOSE1:component_unpacker.cc(90)] Unpacking in: C:\Users\Stephen\AppData\Local\Temp\6988_2124775342
[6988:16568:0514/082157.629:VERBOSE1:component_unpacker.cc(103)] Unpacked successfully
[6988:16568:0514/082157.635:VERBOSE1:component_installer.cc(119)] Install: version=1.0.0 current version=0.0.0.0
[6988:16568:0514/082157.635:VERBOSE1:component_installer.cc(137)] unpack_path=C:\Users\Stephen\AppData\Local\Temp\6988_2124775342 install_path=C:\Users\Stephen\AppData\Local\BraveSoftware\Brave-Browser-Nightly\User Data\SODALanguagePacks\en-US\1.0.0
[6988:9976:0514/082157.645:VERBOSE1:component_installer.cc(454)] Component ready, version 1.0.0 in C:\Users\Stephen\AppData\Local\BraveSoftware\Brave-Browser-Nightly\User Data\SODALanguagePacks\en-US\1.0.0
[6988:9976:0514/082157.645:VERBOSE1:soda_language_pack_component_installer.cc(126)] Component ready, version 1.0.0 in C:\Users\Stephen\AppData\Local\BraveSoftware\Brave-Browser-Nightly\User Data\SODALanguagePacks\en-US\1.0.0
[6492:1116:0514/082157.652:VERBOSE1:network_delegate.cc(32)] NetworkDelegate::NotifyBeforeURLRequest: https://go-updater.brave.com/extensions
[6988:9976:0514/082157.652:VERBOSE1:component_updater_service.cc(387)] Update completed with error 0
[6492:1116:0514/082157.690:VERBOSE1:network_delegate.cc(32)] NetworkDelegate::NotifyBeforeURLRequest: https://componentupdater.brave.com/service/update2/json
[6988:9976:0514/082157.724:VERBOSE1:request_sender.cc(181)] Request completed from url: https://update.googleapis.com/service/update2/json

These show up in brave://components:

soda-components

Expected result:

Unsure; but we should figure out if we want this enabled at all, and/or wrapped via our proxy calls, as we do with other component downloads.

Reproduces how often:

100%

Brave version (brave://version info)

Brave 1.26.19 Chromium: 91.0.4472.38 (Official Build) nightly (64-bit)
Revision 8155e7afab5d695cf0e028f4d77203287523cda9-refs/branch-heads/4472_35@{#6}
OS Windows 10 OS Version 2009 (Build 21376.1)

Version/Channel Information:

@rebron @bsclifton @karenkliu @brave/legacy_qa

stephendonner commented 3 years ago

Should also mention: 1) the UI options for these are missing in Brave, for which I filed https://github.com/brave/brave-browser/issues/15844 2) there are two feature flags in brave://flags: Live Caption SODA for Live Caption

Both are set to default even with the component downloaded and presumably active.

soda-feature-flags

stephendonner commented 3 years ago

I'll test and report back on whether disabling either/both of those flags disables the UI and network requests (as I'd expect them to).

bsclifton commented 3 years ago

We can proxy this; besides proxying, we'll want to create a security/privacy review before re-enabling

diracdeltas commented 3 years ago

cc @pes10k @jumde

jumde commented 3 years ago

This will be disabled for now since it requires an API key. Following up with the chromium team to check if this will be available for chromium derivatives.

genesiscz commented 1 year ago

Any update?

Crefix commented 1 year ago

Any update?