justinmeza / lci

A LOLCODE interpreter written in C.
http://lolcode.org
GNU General Public License v3.0
784 stars 105 forks source link

Failed realloc during program buffering causes segfault #55

Open sjoqvist opened 5 years ago

sjoqvist commented 5 years ago

Large program files and/or tight memory constraints can cause segfaults during program buffering. Expected: An exit with an error message.

A sparse file containing a LOLCODE program and trailing NULs will usually run correctly. For example, the following set of commands creates a sparse 1MB file (using something like 4kB on disk) and runs it.

echo "HAI 1.3\nVISIBLE \"hello world\"\nKTHXBYE" > program.lol
dd if=/dev/zero of=program.lol bs=1 count=0 seek=1M
lci program.lol

However, making the file 1TB causes lci to segfault.

echo "HAI 1.3\nVISIBLE \"hello world\"\nKTHXBYE" > program.lol
dd if=/dev/zero of=program.lol bs=1 count=0 seek=1T
lci program.lol

Here's the variable that keeps track of the buffer size. https://github.com/justinmeza/lci/blob/6762b724361a4fb471345961b4750657783aeb3b/main.c#L144 And here's the loop that reads the input file. https://github.com/justinmeza/lci/blob/6762b724361a4fb471345961b4750657783aeb3b/main.c#L193-L200

From what I can tell, two issues contribute to this behavior.

  1. The variable size is declared as unsigned int instead of as size_t. This makes it wrap around at 232 on most systems, even when more than 4GB of RAM is available. At that point, realloc is asked to resize the buffer to 0 bytes.
  2. The return value of realloc isn't checked. If the reallocation is unsuccessful, buffer is replaced with a null pointer.

If you're testing this on your own machine, you may want to limit the memory usage to avoid thrashing. I ran ulimit -v 6291456 to enforce a hard 6GB limit.