devinacker / exhal

Compression and decompression tools for NES, SNES, and Game Boy games by HAL Laboratory
MIT License
64 stars 8 forks source link

Partially fixed out-of-bounds memory access #6

Closed DarkKirb closed 6 years ago

DarkKirb commented 6 years ago

inhal can produce files that aren't recognized by exhal. They contain reverse backwards references with a size greater than their offset, apparently reading unrelated data in front of the buffer. This does not occur when -fast is passed to inhal, as code responsible for finding a reverse backref is skipped. This might be because inhal uses an uninitialized value in line 276, as valgrind reports.

Steps to reproduce:

./exhal kdl1.gb 0x8952 dec.bin
./inhal -n dec.bin new.bin

when compiled with -DDEBUG_OUT, one might be able to read

write_backref: writing backref to    7, size 16 (method 2)

Interestingly this specific line of code has existed forever, but i didn't have any issues with exhal until the latest changes. I'd suggest adding a check that reads

if(pos - start < size)
    size -= size - (pos - start);

before the backref_candidate() call.

devinacker commented 6 years ago

Should be fixed by 976cff41f86b6c6725a3e17e325dd880843a98ad, let me know if there are still problems somewhere.

DarkKirb commented 6 years ago

It works. thank you