search

TimeLineAnnotator / desktop

A GUI for graphical analysis and annotation of video and audio files.
https://tilia-app.com
Creative Commons Attribution Share Alike 4.0 International
12 stars 3 forks source link

Waveform makes the app prohibitively slow, freezes the app #200

Closed johentsch closed 4 days ago

johentsch commented 2 weeks ago

Edit: As it turns out, the app becomes close to unusable once a timeline with a waveform is involved.

Example file (rename to example.tla): example.json (created by removing the media_path and turning the absolute file_path into a relative one.

The example corresponds to a WAV file that is 12:38 long. When opened in TiLiA desktop, scrolling and zooming is extremely slow.

These operations take an unreasonable amount of time and make the app freeze:

Judging from the 5.9 MB example file, the waveform is stored as a large number of samples, each an individual JSON object with beginning and end. My guess is that it is the huge number of AmplitudeBar objects that make the program unusable. The first solution that comes to mind would be to simply store a numpy array of amplitudinal values combined with a sample rate.

FelipeDefensor commented 2 weeks ago

Even for smaller audio files the performnce of undoing/redoing and zooming is very bad for version v0.4.3. This has not only to do with the underlying data structure, which is exactly like you described, substituting "JSON object" for "Python object", but also with the brute-force approch taken towards restoring a previous app state and redrawing a timeline.

Fortunately, that performance issue has been mostly solved, save for fixing some outstanding bugs, and those improvements will be released with v0.5. If you checkout the "optimize-zooming" branch you will see that zooming takes a fraction of a second (~0.05 seconds, if I remember correclty) and the "optimize-restore-state" branch makes undoing/redoing even more performant.

I have not touched the data structure underlying the AudioWaveTimeline yet, as that hasn't been necessary, but I agree that a simple numpy array would be the way to go. Actually, if performance starts being an issue again, I think we should consider using numpy arrays as the underlying data structures of for all the timelines.