Long files crash Gentle DMG

[strob]

Reported on Twitter by @jarm:

struggling with Gentle using large files (window dies), should I chop them up?

same again with just audio (80MB; 2hrs47m). I tried with smaller clip (8.7MB;4m) and that works fine.

seems to hang when transcription finishes and page layout changes

I wasn't watching moments crashes happened, but both seem to be when transcription ends. this is w/ offline version btw.

[strob]

I would like to know:

• how much RAM does your computer have, and is it substantially used by the upload or transcription?
•  If you run Gentle from the Terminal like so:

./Applications/gentle.app/Contents/MacOS/gentle

do you see any errors?

Thanks!

[natelawrence]

A few notes from a primarily Windows user: My belief is that the filesize is the critical factor, rather than any specific duration of the media (as your RAM questions suggest).

When the hosted version at gentle-demo.lowerquality.com is up:

• my 1:25:45 uncompressed .wav (965,841 KB) results in success

When I run the DMG local version on a family member's Mac (4GB iMac 20" Early 2009 running MacOS 10.11.5):

• my 1:25:45 uncompressed .wav (965,841 KB) results in success (but only one at a time, otherwise the DMG crashes just before the time that the further ahead concurrent instance completes matching).

When I use the Docker container on Windows (4GB laptop):

• my 1:25:45 uncompressed .wav (965,841 KB) results in a lengthy wait with the status bar displaying "Waiting for 192.168.99.100...", after which the browser is dumped onto a blank page (with not even an html element to be found when viewing the source) at the address of http://192.168.99.100/transcriptions (with no transcription ID)
• my 1:25:45 audio from the source video exported as a .wav by VirtualDub (199,365 KB) begins upload normally, but then spits out "ERROR: Encoding failed. Make sure that you've uploaded a valid media file." at a normal transcription URL after attempting transcoding. (That .wav file will play in various media players, but due to the size I suspect that VirtualDub is doing something invalid like just putting the original compressed audio into a .wav container.)
• my 1:25:45 audio from the source video exported as an .mp3 by Audacity (201,220 KB) results in success.
• EDIT submitting the 1:25:45 source video (an 1,686,024 KB .mp4) directly to Gentle results in a similar scenario as the first of these four Windows Docker scenarios, except that at the transcription-ID-less URL the default browser message of: _This site can’t be reached 192.168.99.100 refused to connect. Search Google for transcriptions ERR_CONNECTIONREFUSED is displayed.

I will report back when I get chance to observe what RAM usage is like during upload, but I do not expect it to be very high.

The process of transcription, however (by which I assume we mean the step with the progress bar directly following ffmpeg's encoding) runs pitifully slowly on both of my local machines. On Windows Docker's VirtualBox instance consumes only about 25% of my CPU but as much RAM as it can obtain, which gives me the impression that Gentle needs a fair bit of the audio loaded into RAM at a time to do any work.

[strob]

Thanks for this detailed report!

I'm not confident in the upload behaving sanely. The server is written in Twisted, which has known problems with large uploads. That is, uploads may use 2-3x the RAM of the file. Would be great to try this.

As for the transcription, that's a "fixed cost" with RAM, but it's large. Running Gentle from the commandline exposes a --ntranscriptionthreads argument, which defaults to two; each transcription thread could use 2-3gb RAM.

The audio file is not loaded into RAM during transcription.

Using Gentle for full-transcription on a machine with 4gb RAM is unlikely to ever be a great experience, but I would like for the uploading to work better than it currently does. That may mean using twisted-largerrequest, as suggested above, or it might mean implementing a chunked uploader.