libnclist: find_suboverlap_start doesn't check isub against nlists, can segfault

[GoogleCodeExporter]

Using libnclist from pygr 0.8.2, the find_suboverlap_start function used by 
find_intervals can cause a segfault. This can happen at least when there are no 
SublistHeaders, maybe otherwise as well; I'm not sure. The following simple C 
code demonstrates the issue:

=====
#define BUILD_C_LIBRARY
#include "intervaldb.h"
int main() {
    const int n = 4;
    IntervalMap im[n];
    unsigned i;

    for (i = 0; i < 4; ++i) {
        im[i].start = i;
        im[i].end = 2*(i+1);
        im[i].target_id = 0;
    }

    int ntop, nlists;
    SublistHeader *subheader = build_nested_list_inplace(im, 4, &ntop, &nlists);

    IntervalIterator *ito = interval_iterator_alloc();

    IntervalMap buf[1];
    for (IntervalIterator *it = ito; it;) {
        int found;
        find_intervals(it, 0, 8, im, n, subheader, nlists, buf, 1, &found, &it);
        if (found == 1)
            printf("Found overlap: %d %d\n", buf[0].start, buf[0].end);
    }

    free_interval_iterator(ito);
}
=====

When run, this code segfaults, since nlists is 0 and thus subheader is an empty 
array and find_overlap_start accesses the invalid pointer subheader[0].

To fix, add a check that "isub < nlists" in the first if statement in 
find_overlap_start.

Original issue reported on code.google.com by Deewi...@gmail.com on 8 Jun 2011 at 1:27

[GoogleCodeExporter]

Thanks for the catch.  I fixed this by making the change you suggested in 
find_suboverlap_start(), and added your test as tests/issue133.c.  I noticed 
that the SEGV only seems to happen with a buffer size of 1 (2 also works, but 
the size we typically use, 1024, seems to prevent the SEGV from occurring).

Original comment by cjlee...@gmail.com on 9 Jun 2011 at 5:51

• Changed state: FixedNeedsReview