Decoder should not crash if it runs out of memory

What steps will reproduce the problem?
(On a Linux system, or anywhere that ulimit is available:)
1. Build and/or install the vcdiff executable for open-vcdiff v0.2.
2. Download the attached delta file "allocates_4gb.vcdiff".
3. Limit memory usage to 32MB per process, and create a dummy dictionary
whose contents will be ignored:
ulimit -d 32768
ulimit -m 32768
echo foo > dummy.dict

4. Try to decode the delta file:
vcdiff decode -dictionary dummy.dict -delta allocates_4gb.vcdiff

What is the expected output? What do you see instead?
The delta file tries to allocate 4GB of memory but the ulimit only allows
32MB memory usage.  The expected behavior is for the vcdiff executable to
output an error message and return a non-zero (unsuccessful) result.  What
happens instead is that the decoder throws an unhandled C++ exception:

terminate called after throwing an instance of 'std::length_error'
  what():  basic_string::_S_create
Aborted

A relatively small malicious delta file can be tailored to produce an
arbitrarily large target file and so cause an out-of-memory crash, or at
least a huge slowdown as virtual memory starts swapping to disk.  The
decoder does have the restriction that no target window can be larger than
64MB in size, but a malicious encoding can still use multiple target
windows to reach an arbitrary size.  The attached file contains 64 windows
of 64MB each, for a total of 4GB.

If open-vcdiff is embedded within a larger application, then blocking that
application or throwing it an unhandled exception may have disastrous
consequences.

Adding try/catch statements around each of the external interfaces will
keep the unhandled exception from crashing the application that calls the
open-vcdiff decoder library, but it will not prevent the slowdown
associated with swapping virtual memory pages before running out of memory
entirely.

To solve this problem, the decoder should have configurable limits on some
of the following values:
1. The maximum size of a single COPY or RUN instruction.
2. The maximum size of a delta window (this is currently 64MB.)
3. The maximum size of a delta file.

Restricting (2.) will also implicitly restrict (1.), and restricting (3.)
will also implicitly restrict (1.) and (2.)

Original issue reported on code.google.com by openvcd...@gmail.com on 11 Sep 2008 at 11:02

Attachments:

allocates_4gb.vcdiff

Version 0.3 contains two new interfaces that limit the amount of target data 
that the
decoder is allowed to produce, and therefore limit the amount of memory the 
decoder
may consume.  These interfaces correspond to limits (2.) and (3.) mentioned in 
the
original description of this issue.

Please see the comments for the following methods in src/google/vcdecoder.h:
VCDiffStreamingDecoder::SetMaximumTargetWindowSize()
VCDiffStreamingDecoder::SetMaximumTargetFileSize()

The default limits for these values are both 64 MB.

As mentioned in Issue 16, open-vcdiff currently cannot handle input or output 
files
larger than 2GB, even on 64-bit systems.

Original comment by openvcd...@gmail.com on 11 Oct 2008 at 12:15

Changed state: Fixed

ivankosenko / open-vcdiff

Decoder should not crash if it runs out of memory #8