Imp imports everything into memory

Infogroep / imp

Ruben en Ayrton's Rare Audio Speler

MIT License

1 stars 0 forks source link

Imp imports everything into memory #12

Closed Ichabond closed 10 years ago

Ichabond commented 10 years ago

When uploading files to imp, they get loaded into memory entirely first, before being written to disk. This breaks machines when trying to upload a movie file that's bigger than the available memory.

angryzor commented 10 years ago

I assume this is on imp-client? Or is it server side?

Ichabond commented 10 years ago

Nope, imp server, at least that's where I observed the memory use, possibly in the client as well, but since I have 16GB RAM, this didn't cause noticeable issues, while myth only has 4GB (and 4GB swap). The movie file was right over 8GB. It died a horrible and fiery death.

angryzor commented 10 years ago

Never mind, got it. Server side. Yeah i suspected this already but didn.t test for it yet. Will fix asap, ben toch net die twisted code aant reviewen.

angryzor commented 10 years ago

Note to self: Error is on line 84 of imp-server. Entire stream is read from request into content before passing to stdIn. Should use a better setup now that I actually know how twisted works. (the existing code was basically a modification of twisted's CGI implementation)