search

agordon / fastx_toolkit

FASTA/FASTQ pre-processing programs
Other
167 stars 60 forks source link

Fixed buffer overflow bug, caused by erroneous array boundaries #3

Closed KonstantinShemyak closed 10 years ago

KonstantinShemyak commented 10 years ago

fastq_quality_filter segfaults on the following FASTQ file:

@ AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA + DDDfmnnei__pplWWWstXXX|hhaa~FFSDDDCDnaaRRYR`fkkdd{ss~wzzvooha^ax}}

It may result in another memory error than segmentation fault on your machine, but the fact is that src/fastq_quality_filter/fastq_quality_filter.c:get_percentile_quality attempts to write to an address out of the allocated space in line 120.

The reason is that the array quality_values[] has size MAX_QUALITY_VALUE - MIN_QUALITY_VALUE, but the index fastx->quality[i] - MIN_QUALITY_VALUE can be bigger than this. When fastx->quality[i] is being set, fastx.c checks that the value is <93 (hardcoded). 93 is bigger than MAX_QUALITY_VALUE, so the index is out of bounds.

The patch redefines [MIN|MAX]_QUALITY_VALUE from seemingly random -50/+50 to the values which are now checked for in fastx.c (-15 and 93), and replaces these hardcoded numbers with these constants.

agordon commented 10 years ago

Thank you. I have merged the two changes into one and committed.