search

jlabeit / parallel-range-lite

Parallel-Range-Light
3 stars 2 forks source link

Segmentation fault #1

Open dvalenzu opened 6 years ago

dvalenzu commented 6 years ago

Hi,

Thanks for having this nice library public! I think I found a bug, or perhaps I stumbled into an undocumented limitation; I was building the suffix array of a ~1.8GB sequence, as 32 bit integers and got a segmentation fault.

The steps to reproduce it are as follows:

Then the following happens:

./main.o genome_problematic.txt Segmentation fault (core dumped)

I don't think it is because of lack of memory, as I am runing this in a server with 1.5TB of ram. and when I montitored the memory used I got the following:

/usr/bin/time --verbose ./main.o /home/scratch-ssd/dvalenzu/Data/genome_problematic.txt (...) Maximum resident set size (kbytes): 30819104

Best, Daniel

ThomasThelen commented 6 years ago

Are you running this on a 32 bit machine?

dvalenzu commented 6 years ago

This was run in a 64 bits machine, with 1.5 TB of ram.

ThomasThelen commented 6 years ago

I think this line provides a little insight to what's going on, Maximum resident set size (kbytes): 30819104

The kernel is telling you that the process has an allocation of about 30 gigs. The segmentation fault is probably happening when you hit the upper limit and the kernel kills the process.

I'm not sure how to fix this, but here are some resources that may help you out.

Resident Set Size? https://en.wikipedia.org/wiki/Resident_set_size https://unix.stackexchange.com/questions/30940/getrusage-system-call-what-is-maximum-resident-set-size

Set the Limit Set the Set Size Limit According to the second comment on the first answer it's not possible, but I don't buy that.

setlimit This command might come in handy

conf file?

dvalenzu commented 6 years ago

Hi, In the same machine I'm able to run another process that uses more than 300 gigs (resident size), so I don't think the cause of the problem is related to system limits.