Fix /tmp Filling Up Bottlenecks (server + client)

deeplow commented 1 year ago

The temporary (/tmp) space limitations of Qubes revealed some bottlenecks in the current architecture of Dangerzone, which is manifest themselves in large files (150+ pages). The containers version isn't as affected sync the /tmp file system has more space, but addressing this issue will improve the situation overall.

The problems

As far as I can tell, there are two main issues:

[ ] https://github.com/freedomofpress/dangerzone/issues/526 Lack of space in client: just a consequence of us waiting on all the RGB files to finally start reconstructing the file
[ ] https://github.com/freedomofpress/dangerzone/issues/574 Lack of space in server: even though the RGB files are deleted as soon as they are sent, the reality is that the RGB files are produced much faster than the client can consume them

I tried to illustrate some of this issues in a diagram:

dangerzone-bottlenecks drawio(4)

harrislapiroff commented 1 year ago

Code to resolve this is in progress on 616-main-pymupdf (will supercede #619)

apyrgio commented 2 weeks ago

We've successfully tackled this issue (see https://github.com/freedomofpress/dangerzone/issues/526#issuecomment-2444632713).

deeplow commented 2 weeks ago

Hurray! Glad to see this one closed :slightly_smiling_face:

freedomofpress / dangerzone

Fix /tmp Filling Up Bottlenecks (server + client) #616

The problems