Figure out if we can cache contents using MediaSource API

[sangwoo108]

@fallaciousreasoning found that BLOB: url with simple buffer can be cached. Fetch API with BLOB URL goes through the blob strorage. So we can use it. (https://github.com/brave/brave-core/pull/17246 )

But in case MediaSource, BLOB url doesn't go through BLOB storage at all. So we might need to directly access the buffer which attached to MediaSource object.

[sangwoo108]

content::DownloadManager provides blob downloading.

// For downloads of blob URLs, the caller can pass a URLLoaderFactory to
  // use to load the Blob URL. If none is specified and the blob URL cannot be
  // mapped to a blob by the time the download request starts, then the download
  // will fail.
  virtual void [DownloadUrl](https://source.chromium.org/chromium/chromium/src/+/main:content/public/browser/download_manager.h;bpv=1;bpt=1;l=127?gsn=DownloadUrl&gs=kythe%3A%2F%2Fchromium.googlesource.com%2Fchromium%2Fsrc%3Flang%3Dc%252B%252B%3Fpath%3Dcontent%2Fpublic%2Fbrowser%2Fdownload_manager.h%239G0tWtqurbtrdvTxMLpXmWmXHqRzdENznr8_ATxC-2A)(
      std::unique_ptr<download::[DownloadUrlParameters](https://source.chromium.org/chromium/chromium/src/+/main:components/download/public/common/download_url_parameters.h;drc=5d07288021882430d97db7b56e895122b6b81afd;bpv=0;bpt=1;l=47)> [parameters](https://source.chromium.org/chromium/chromium/src/+/main:content/public/browser/download_manager.h;bpv=1;bpt=1;l=128?gsn=parameters&gs=kythe%3A%2F%2Fchromium.googlesource.com%2Fchromium%2Fsrc%3Flang%3Dc%252B%252B%3Fpath%3Dcontent%2Fpublic%2Fbrowser%2Fdownload_manager.h%23OCY7ZADtWGvbmtHXrAsFBzARTKHayS4j_tUUeXzxINs),
      scoped_refptr<network::[SharedURLLoaderFactory](https://source.chromium.org/chromium/chromium/src/+/main:services/network/public/cpp/shared_url_loader_factory.h;drc=5d07288021882430d97db7b56e895122b6b81afd;bpv=0;bpt=1;l=26)>
          [blob_url_loader_factory](https://source.chromium.org/chromium/chromium/src/+/main:content/public/browser/download_manager.h;bpv=1;bpt=1;l=130?gsn=blob_url_loader_factory&gs=kythe%3A%2F%2Fchromium.googlesource.com%2Fchromium%2Fsrc%3Flang%3Dc%252B%252B%3Fpath%3Dcontent%2Fpublic%2Fbrowser%2Fdownload_manager.h%2368JVGhCf_-SAneI01Rqj-TX7K4ED1-Fi8OaTmT1HdEo)) = 0;

[sangwoo108]

It looks like we need a url loader factory specialized for blob and blob_url_token.

https://source.chromium.org/chromium/chromium/src/+/main:content/browser/renderer_host/render_frame_host_impl.cc;l=5622;drc=8a5f4673100fc681bd6dd8663e40a36ffc6783c0;bpv=1;bpt=1

  scoped_refptr<network::SharedURLLoaderFactory> blob_url_loader_factory;
  if (blink_parameters->blob_url_token) {
    blob_url_loader_factory =
        ChromeBlobStorageContext::URLLoaderFactoryForToken(
            GetStoragePartition(), std::move(blink_parameters->blob_url_token));
  }

But, even when we use the blob storage and content::DownloadManager from the given BrowserContext, download is interrupted with network failure.

30229:259:0120/213530.710541:ERROR:playlist_media_file_downloader.cc(223)] OnDownloadUpdated: Download interrupted - reason: NETWORK_FAILED
[30229:259:0120/213530.715985:ERROR:playlist_media_file_downloader.cc(226)] 0   libbase.dylib                       0x0000000104f39b48 base::debug::CollectStackTrace(void**, unsigned long) + 28
1   libbase.dylib                       0x0000000104e318fc base::debug::StackTrace::StackTrace(unsigned long) + 32
2   libchrome_dll.dylib                 0x0000000110e91318 playlist::PlaylistMediaFileDownloader::OnDownloadUpdated(download::DownloadItem*) + 288
3   libpublic.dylib                     0x000000010cb5fa70 download::DownloadItemImpl::UpdateObservers() + 596
4   libpublic.dylib                     0x000000010cb683b8 download::DownloadItemImpl::OnTargetResolved() + 548
5   libpublic.dylib                     0x000000010cb67d7c download::DownloadItemImpl::OnDownloadTargetDetermined(base::FilePath const&, download::DownloadItem::TargetDisposition, download::DownloadDangerType, download::DownloadItem::MixedContentStatus, base::FilePath const&, base::FilePath const&, std::Cr::basic_string<char, std::Cr::char_traits<char>, std::Cr::allocator<char>> const&, download::DownloadInterruptReason) + 632

31471:259:0120/215508.183967:ERROR:download_item_impl.cc(2093)] InterruptWithPartialState() reason:NETWORK_FAILED bytes_so_far:0 hash_state:Invalid this={ id = 20 state = TARGET_RESOLVED total = 0 received = 0 reason = NETWORK_FAILED paused = F resume_mode = INVALID auto_resume_count = 0 danger = 0 all_data_saved = F last_modified = '' etag = '' has_download_file = false url_chain = 
        "blob:https://abcdef"
         current_path = ""
         target_path = "/mypath.mp4"
         rereoute_info = 'MessageLite at 0x15846f838'" referrer = "" serialized_embedder_download_data = "

" }

The network failure was an erro converted from net::ERROR_FILE_NOT_FOUND (-6)

[32959:40195:0120/223143.445663:ERROR:download_response_handler.cc(243)] OnComplete-6

And this was set when loader starts, beacuse there's no |blob_|

void BlobURLLoaderFactory::CreateLoaderAndStart(
    mojo::PendingReceiver<network::mojom::URLLoader> loader,
    int32_t request_id,
    uint32_t options,
    const network::ResourceRequest& request,
    mojo::PendingRemote<network::mojom::URLLoaderClient> client,
    const net::MutableNetworkTrafficAnnotationTag& traffic_annotation) {
  if (url_.is_valid() && request.url != url_) {
    receivers_.ReportBadMessage("Invalid URL when attempting to fetch Blob");
    mojo::Remote<network::mojom::URLLoaderClient>(std::move(client))
        ->OnComplete(network::URLLoaderCompletionStatus(net::ERR_INVALID_URL));
    return;
  }
  if (!blob_) {
    mojo::Remote<network::mojom::URLLoaderClient>(std::move(client))
        ->OnComplete(
            network::URLLoaderCompletionStatus(net::ERR_FILE_NOT_FOUND));
    return;
  }

I was trying to creat the loader without BlobUrlToken and other blob related types but this seems not working. Possibly I'm missing something or we can't help but using the token.

[sangwoo108]

Logs when using MediaSource API

[29128:259:0505/111918.047252:ERROR:chrome_browser_cloud_management_controller.cc(162)] Cloud management controller initialization aborted as CBCM is not enabled.
[29150:259:0505/111921.053996:ERROR:html_media_element.cc(1329)] LoadResource
[29150:259:0505/111921.054065:ERROR:html_media_element.cc(1403)]  Is this the case?
[29150:259:0505/111921.071882:ERROR:demuxer_manager.cc(154)] DemuxerManager
[29150:259:0505/111921.073543:ERROR:demuxer_manager.cc(546)] CreateChunkDemuxer
[29150:259:0505/111921.414515:ERROR:demuxer_manager.cc(666)] OnProgress
[29150:259:0505/111921.414801:ERROR:source_buffer_stream.cc(267)] Append VIDEO: buffers dts=[0us;735000us(last frame dur=34000us)], pts interval=[0us,769000us)
[29150:259:0505/111921.417296:ERROR:demuxer_manager.cc(666)] OnProgress
[29150:259:0505/111921.417439:ERROR:source_buffer_stream.cc(267)] Append VIDEO: buffers dts=[769000us;3593000us(last frame dur=169000us)], pts interval=[769000us,3762000us)
[29150:259:0505/111921.418816:ERROR:demuxer_manager.cc(666)] OnProgress
[29150:259:0505/111921.418954:ERROR:source_buffer_stream.cc(267)] Append VIDEO: buffers dts=[3762000us;4867000us(last frame dur=34000us)], pts interval=[3762000us,4901000us)
[29150:259:0505/111921.419418:ERROR:demuxer_manager.cc(666)] OnProgress
[29150:259:0505/111921.419499:ERROR:source_buffer_stream.cc(267)] Append VIDEO: buffers dts=[4901000us;4934000us(last frame dur=34000us)], pts interval=[4901000us,4968000us)
[29150:259:0505/111921.419575:ERROR:demuxer_manager.cc(666)] OnProgress
[29150:259:0505/111921.419675:ERROR:demuxer_manager.cc(666)] OnProgress
[29150:259:0505/111921.419778:ERROR:demuxer_manager.cc(666)] OnProgress
[29150:259:0505/111921.419940:ERROR:source_buffer_stream.cc(267)] Append VIDEO: buffers dts=[4968000us;4968000us(last frame dur=33000us)], pts interval=[4968000us,5001000us)
[29150:259:0505/111921.420009:ERROR:demuxer_manager.cc(666)] OnProgress
[29150:259:0505/111921.420113:ERROR:demuxer_manager.cc(666)] OnProgress
[29150:259:0505/111921.420207:ERROR:demuxer_manager.cc(666)] OnProgress
[29150:259:0505/111921.420325:ERROR:source_buffer_stream.cc(267)] Append VIDEO: buffers dts=[5001000us;5451000us(last frame dur=34000us)], pts interval=[5001000us,5485000us)
[29150:259:0505/111921.420397:ERROR:demuxer_manager.cc(666)] OnProgress
[29150:259:0505/111921.420474:ERROR:source_buffer_stream.cc(267)] Append VIDEO: buffers dts=[5485000us;6685000us(last frame dur=33000us)], pts interval=[5485000us,6718000us)
[29150:259:0505/111921.420538:ERROR:demuxer_manager.cc(666)] OnProgress
[29150:259:0505/111921.420657:ERROR:source_buffer_stream.cc(267)] Append VIDEO: buffers dts=[6718000us;7403000us(last frame dur=34000us)], pts interval=[6718000us,7437000us)
[29150:259:0505/111921.420734:ERROR:demuxer_manager.cc(666)] OnProgress
[29150:259:0505/111921.420813:ERROR:source_buffer_stream.cc(267)] Append VIDEO: buffers dts=[7437000us;9005000us(last frame dur=34000us)], pts interval=[7437000us,9039000us)
[29150:259:0505/111921.420880:ERROR:demuxer_manager.cc(666)] OnProgress
[29150:259:0505/111921.420936:ERROR:source_buffer_stream.cc(267)] Append VIDEO: buffers dts=[9039000us;11118000us(last frame dur=1001000us)], pts interval=[9039000us,12119000us)
[29150:259:0505/111921.421015:ERROR:demuxer_manager.cc(666)] OnProgress
[29150:259:0505/111921.421095:ERROR:source_buffer_stream.cc(267)] Append VIDEO: buffers dts=[12119000us;12119000us(last frame dur=394000us)], pts interval=[12119000us,12513000us)
[29150:259:0505/111921.421175:ERROR:demuxer_manager.cc(666)] OnProgress
[29150:259:0505/111921.421226:ERROR:source_buffer_stream.cc(267)] Append VIDEO: buffers dts=[12513000us;12513000us(last frame dur=34000us)], pts interval=[12513000us,12547000us)
[29150:259:0505/111921.421296:ERROR:demuxer_manager.cc(666)] OnProgress
[29150:259:0505/111921.421384:ERROR:source_buffer_stream.cc(267)] Append VIDEO: buffers dts=[12547000us;12581000us(last frame dur=33000us)], pts interval=[12547000us,12614000us)
[29150:259:0505/111921.421447:ERROR:demuxer_manager.cc(666)] OnProgress
[29150:259:0505/111921.421512:ERROR:source_buffer_stream.cc(267)] Append VIDEO: buffers dts=[12614000us;12648000us(last frame dur=34000us)], pts interval=[12614000us,12682000us)
[29150:259:0505/111921.421577:ERROR:demuxer_manager.cc(666)] OnProgress
[29150:259:0505/111921.421659:ERROR:source_buffer_stream.cc(267)] Append VIDEO: buffers dts=[12682000us;15868000us(last frame dur=151000us)], pts interval=[12682000us,16019000us)
[29150:259:0505/111921.421721:ERROR:demuxer_manager.cc(666)] OnProgress
[29150:259:0505/111921.421970:ERROR:source_buffer_stream.cc(267)] Append VIDEO: buffers dts=[16019000us;17820000us(last frame dur=33000us)], pts interval=[16019000us,17853000us)
[29150:259:0505/111921.422274:ERROR:demuxer_manager.cc(666)] OnProgress
[29150:259:0505/111921.422452:ERROR:source_buffer_stream.cc(267)] Append VIDEO: buffers dts=[17853000us;19301000us(last frame dur=34000us)], pts interval=[17853000us,19335000us)
[29150:259:0505/111921.422580:ERROR:demuxer_manager.cc(666)] OnProgress
[29150:259:0505/111921.422739:ERROR:source_buffer_stream.cc(267)] Append VIDEO: buffers dts=[19335000us;22101000us(last frame dur=481000us)], pts interval=[19335000us,22582000us)
[29150:259:0505/111921.422928:ERROR:demuxer_manager.cc(666)] OnProgress
[29150:259:0505/111921.423060:ERROR:demuxer_manager.cc(666)] OnProgress
[29150:259:0505/111921.423151:ERROR:source_buffer_stream.cc(267)] Append VIDEO: buffers dts=[22582000us;22582000us(last frame dur=38000us)], pts interval=[22582000us,22620000us)
[29150:259:0505/111921.423232:ERROR:demuxer_manager.cc(666)] OnProgress
[29150:259:0505/111921.423307:ERROR:source_buffer_stream.cc(267)] Append VIDEO: buffers dts=[22620000us;22654000us(last frame dur=33000us)], pts interval=[22620000us,22687000us)
[29150:259:0505/111921.423367:ERROR:demuxer_manager.cc(666)] OnProgress
[29150:259:0505/111921.423445:ERROR:source_buffer_stream.cc(267)] Append VIDEO: buffers dts=[22687000us;26373000us(last frame dur=35000us)], pts interval=[22687000us,26408000us)
[29150:259:0505/111921.423519:ERROR:demuxer_manager.cc(666)] OnProgress
[29150:259:0505/111921.423618:ERROR:source_buffer_stream.cc(267)] Append VIDEO: buffers dts=[26408000us;28739000us(last frame dur=2079000us)], pts interval=[26408000us,28739001us)
[29150:259:0505/111921.423982:ERROR:demuxer_manager.cc(666)] OnProgress
[29150:259:0505/111921.424314:ERROR:media_source.cc(1076)] endOfStream
[29150:259:0505/111921.424327:ERROR:media_source.cc(1078)] has source buffer: length: 1
[29150:259:0505/111921.424350:ERROR:chunk_demuxer.cc(289)] MarkEndOfStream
[29150:259:0505/111921.424357:ERROR:source_buffer_stream.cc(1759)] MarkEndOfStream SourceBufferRange: 1
[29150:259:0505/111921.424364:ERROR:source_buffer_stream.cc(1761)]  EOT of SourceBufferRange: 28.739 s

[sangwoo108]

New finds:

• Blink doesn't seem to be granted to write file. We should wrap the buffer with BLOB and ask Browser process for writing. * Or take a file handle from browser process.

• I was trying not to clone the buffer, but the buffer seems to be consumed and gone after being sent to demuxer - Demuxer seems to spit out demuxed source streams right away and not to keep the compressed/encoded data, not 100% sure. So now, this would be really similar machainsm with what @petemill is investigating with JS.

  • The downside of this doubled buffer is memory usage, for sure. We would cache everything on the screen, even though we don't know whether it'd be used or not.

• MediaSource's special usage

  • Could attach buffer from worker thread -> in blink CrossThreadXXX seems to be related to this.
  • MediaSource could be fed with WebCodec-encoded buffer -> AppendXXXChunk() seems to be on this path, looks like named after EncodedVideoChunk from WebCodec spec.

• If this goes well, we might be able to support downloading video via render context menu on <video> tag. Currently, when video tag is backed by MediaSource, it's marked as 'undownloadable'.

Concerns

• Youtube is known to feed buffer on requestAnimationFrame and it caused some bugs on Android. Not sure if this would work well on Android, even if we succeed in this.

• Security - This feels like this could be an attacking point to easily write a file to users' local disk, with JavaScript. If we do this only if the appended bytes are successfully decoded by media framework, then can we assure it's a real media file?