search

FL33TW00D / whisper-turbo

Cross-Platform, GPU Accelerated Whisper 🏎️
https://whisper-turbo.com
Apache License 2.0
1.72k stars 75 forks source link

Chunked loader to reduce memory ceiling #43

Open FL33TW00D opened 1 year ago

FL33TW00D commented 1 year ago

Your peak memory usage when using WASM is your memory usage for the rest of time, see https://github.com/WebAssembly/design/issues/1397

This means we need to load from IndexedDB in chunks, to reduce peak memory consumption.

jozefchutka commented 11 months ago

Not sure what exactly are the memory issues you face, but when working with large files in browser, Blob-s are very helpful.

To my experience, even when using dozen of GBs of data referenced via Blob, there are no memory peaks observed.

The downside of using Blobs are that:

  1. access to the bytes is async
  2. the data are readonly, unless you decide to slice the source Blob to create another Blob object (which is a synchronous operation).

I wonder if Blobs are suitable for your case.

FL33TW00D commented 10 months ago

Not sure what exactly are the memory issues you face, but when working with large files in browser, Blob-s are very helpful.

To my experience, even when using dozen of GBs of data referenced via Blob, there are no memory peaks observed.

The downside of using Blobs are that:

  1. access to the bytes is async
  2. the data are readonly, unless you decide to slice the source Blob to create another Blob object (which is a synchronous operation).

I wonder if Blobs are suitable for your case.

I need to look into this! I'll take a look at Blobs thanks for the reminder.