safepoint.cpp error

[GoogleCodeExporter]

Hi micha,

Thank you very much!
Its running fine up to RT step.
I am getting the following error now:

# An unexpected error has been detected by Java Runtime Environment:
#
#  Internal Error (safepoint.cpp:261), pid=13352, tid=1102268736
#  Error: guarantee(PageArmed == 0,"invariant")
#
# Java VM: Java HotSpot(TM) 64-Bit Server VM (10.0-b19 mixed mode
linux-amd64)
# An error report file with more information is saved as:
# /home/khan/readSim/FluxSimulator/bin/hs_err_pid13352.log
#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
#
sim.sh: line 3: 13352 Aborted

                 java -Xmx1G -XX:-UseBiasedLocking
-DwrapperDir="$scriptdir" -jar "$scriptdir/../lib/FluxSimulator.jar" $@ 

I have the following java installed:
java version "1.6.0_05"
Java(TM) SE Runtime Environment (build 1.6.0_05-b13)
Java HotSpot(TM) 64-Bit Server VM (build 10.0-b19, mixed mode).

Thanks again for your help.

-Sohail

Original issue reported on code.google.com by gmicha@gmail.com on 22 Jun 2010 at 2:49

[GoogleCodeExporter]

Hi Sohail,

I downloaded java jre1.6 update 5 (the very same the issue was reported for) 
and on an (Intel) 64bit the test experiment ran through the library 
construction without any issue. I would appreciate very much if you can tell me 
whether you are able to reproduce the issue. If so, could you please also 
provide me an input file and /proc/cpuinfo respectively OS of your machine?

It seems that adding/removing VM flags -XX:+UseBiasedLocking and 
-XX:-UseSafeCompiler solved similar problems. However, I would be curious to 
see why/when this issue occurs as it is commonly related to failing JNI 
calls--and the FluxSimulator does not adopt any native code, except the one of 
the Java VM.

Best,

micha

Original comment by gmicha@gmail.com on 22 Jun 2010 at 3:01

[GoogleCodeExporter]

PS: I forgot to add my experiences:

$./jre1.6.0_05/bin/java -version
java version "1.6.0_05"
Java(TM) SE Runtime Environment (build 1.6.0_05-b13)
Java HotSpot(TM) 64-Bit Server VM (build 10.0-b19, mixed mode)

--------------------------------------------------------------------------------
----

$./jre1.6.0_05/bin/java -Xmx1G -XX:-UseBiasedLocking -DwrapperDir="$scriptdir" 
-jar "$scriptdir/../lib/FluxSimulator.jar" $@

[HELLO] I am the Flux Simulator (build 20100611), nice to meet you!

[INIT] I am collecting information on the run.
        initializing profiler  *********
[CAUTION] I overwrite the expression values in file dummy.pro, please confirm:
        (Yes,No,Don't know) y

[WARNING] I will overwrite the library file dummy.lib, please confirm:
        (Yes,No,Don't know) y

[PROFILING] I am assigning the expression profile
        checking ********* OK (00:00:00)
        Reading reference annotation ******     found 4 transcripts

        NB_CELLS        60
        NB_MOLECULES    6000000
        EXPRESSION_K    -0.6
        EXPRESSION_X0   5.0E7
        EXPRESSION_X1   9500.0
        PRO_FILE_NAME   /parallel/disk3/micha/simtest/dummy.pro
        profiling       OK *********
        Updating .pro file  * OK
        molecules       6000000.0

[LIBRARY] creating the cDNA libary
        FRAG_LAMBDA     900.0
        FRAG_SIGMA      0.05
        FRAG_THRESHOLD  0.1
        FRAG_B4_RT      true
        RT_MIN  100
        RT_MAX  5500
        RT_PRIMER       RANDOM
        FILTERING       true
        FILT_MIN        175
        FILT_MAX        225
        PRO_FILE_NAME   /parallel/disk3/micha/simtest/dummy.pro
        TMP_DIR /tmp

        Initializing Fragmentation File ********* (00:00:34)
        6000000 mol inited
        Fragmentation-1 ********** (00:01:28)
        Reverse Transcription ********** (00:01:49)
        Segregating cDNA ********** (00:01:33)
        Copying results         moving **********

        Updating .pro file  * OK

--------------------------------------------------------------------------------
----

$cat /proc/cpuinfo
[...]
processor       : 7
vendor_id       : GenuineIntel
cpu family      : 6
model           : 23
model name      : Intel(R) Xeon(R) CPU           X5450  @ 3.00GHz
stepping        : 6
cpu MHz         : 1998.000
cache size      : 6144 KB
physical id     : 1
siblings        : 4
core id         : 3
cpu cores       : 4
fpu             : yes
fpu_exception   : yes
cpuid level     : 10
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov 
pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm 
constant_tsc arch_perfmon pebs bts rep_good pni monitor ds_cpl vmx est tm2 
ssse3 cx16 xtpr dca lahf_lm
bogomips        : 6000.29
clflush size    : 64
cache_alignment : 64
address sizes   : 38 bits physical, 48 bits virtual
power management:

"lm", a 64bit machine

Original comment by gmicha@gmail.com on 22 Jun 2010 at 3:16

[GoogleCodeExporter]

Original comment by gmicha@gmail.com on 22 Jun 2010 at 3:16

• Changed state: Unconfirmed

[GoogleCodeExporter]

Hi, micha!
I'm running simulator jobs by qsub.
but, I am getting the same errors like Sohail.

------------------------------------------------------------------------------

[HELLO] I am the Flux Simulator (build 20100611), nice to meet you!

[INIT] I am collecting information on the run.

[PROFILING] I am assigning the expression profile
        checking ********** OK (00:00:03)
        Reading reference annotation ********** found 34983 transcripts

        NB_CELLS        100
        NB_MOLECULES    10000000
        EXPRESSION_K    -0.6
        EXPRESSION_X0   5.0E7
        EXPRESSION_X1   9500.0
        PRO_FILE_NAME   /home/array/chseo/sge/FluxSimulator/hg19_RefSeq/pro/hg19_RefSeq_NB_10000000_READ_4000000_36.pro
        profiling       OK 
        Updating .pro file  ********** OK
        molecules       9999367.0

[LIBRARY] creating the cDNA libary
        FRAG_LAMBDA     900.0
        FRAG_SIGMA      NaN
        FRAG_THRESHOLD  NaN
        FRAG_B4_RT      true
        RT_MIN  30
        RT_MAX  1000
        RT_PRIMER       RANDOM
        FILTERING       true
        FILT_MIN        175
        FILT_MAX        225
        PRO_FILE_NAME   /home/array/chseo/sge/FluxSimulator/hg19_RefSeq/pro/hg19_RefSeq_NB_10000000_READ_4000000_36.pro
        TMP_DIR /home/array/chseo/sge/FluxSimulator/hg19_RefSeq/tmp

        Initializing Fragmentation File ********** (00:02:53)
        9999367 mol inited
        Chemical Cleavage-1 */home/array/Simulator/FluxSimulator/bin/sim.sh: line 3: 20110 Aborted                 java -Xmx1G -XX:-UseBiasedLocking -DwrapperDir="$scriptdir" -jar "$scriptdir/../lib/FluxSimulator.jar" $@

--------------------------------------------------------------------------------
--

$java -version
java version "1.6.0_12"
Java(TM) SE Runtime Environment (build 1.6.0_12-b04)
Java HotSpot(TM) 64-Bit Server VM (build 11.2-b01, mixed mode)

--------------------------------------------------------------------------------
-

Original comment by chaes...@gmail.com on 25 Jun 2010 at 12:03

[GoogleCodeExporter]

Dear Chaehwa,

the output you posted shows that the program run was terminated during the 
process "Chemical Cleavage-1" processing. It confuses me a bit that you got an 
"Aborted" and not a "Killed" in the last line. If the system shut down your 
process, it is because it ran out of resources-most commonly it is memory, 
however, I would exclude that here as all memory necessary for the program run 
is reserved in the beginning. It would help to see the /var/log messages to 
find out, why the system stopped the process if it did so.

I still was not able to re-produce the JVM crash, but after digging a bit in 
the subject I performed some modifications in the parallelization. Please find 
these changes included in the package 

http://code.google.com/p/fluxcapacitor/downloads/detail?name=FluxSimulator-20100
702.tar.gz

and confirm whether or not they eliminate the problem described in this thread.

Best wishes, micha

Original comment by gmicha@gmail.com on 2 Jul 2010 at 11:15

[GoogleCodeExporter]

Hi,

I got the same JVM crash problem during the sequencing step. Please see the 
following log and the attached configuration file for more details. 
Interestingly, this problem occurs when PAIR_END=YES, and when the read number 
is large (>10M).

If you want to reproduce the error, you may download the UCSC mouse mm9 
knowngene annotation (abount 90 Mb).

Thanks,

Wei

-------------------------------------------------------
java version "1.6.0_20"
Java(TM) SE Runtime Environment (build 1.6.0_20-b02)
Java HotSpot(TM) 64-Bit Server VM (build 16.3-b01, mixed mode)
[HELLO] I am the Flux Simulator (build 20100705), nice to meet you!

[INIT] I am collecting information on the run.
    initializing profiler  **********

    initializing library  **********

[NOEXPR] you did not ask for expression, I skip it.

[NOLIB] you did not want me to construct the library, I skip it.

[SEQUENCING] getting the reads
    PRO_FILE_NAME   /data1/weili/project/flux/data4/mm9knowngene.pro
    SEQ_FILE_NAME   /data1/weili/project/flux/data4/mm9knowngene.bed
    READ_LENGTH 75
    READ_NUMBER 10000000
    initing **********
    10299721 lines submitted
    zipping **********
    10299721 lines zipped
    sequencing #
# A fatal error has been detected by the Java Runtime Environment:
#
#  Internal Error (safepoint.cpp:244), pid=21203, tid=1081280848
#  Error: guarantee(PageArmed == 0,"invariant")
#
# JRE version: 6.0_20-b02
# Java VM: Java HotSpot(TM) 64-Bit Server VM (16.3-b01 mixed mode linux-amd64 )
# An error report file with more information is saved as:
# /data1/weili/project/flux/run4/hs_err_pid21203.log
#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
#
/data1/weili/prog/flux/current/bin/sim.sh: line 3: 21203 Aborted                
 java -Xmx1G -XX:-UseBiasedLocking -DwrapperDir="$scriptdir" -jar 
"$scriptdir/../lib/FluxSimulator.jar" $@

Original comment by li.david...@gmail.com on 13 Jul 2010 at 8:00

Attachments:

• bugreport.zip

[GoogleCodeExporter]

Also, if I set FASTQ=NO, it works without any problem. So the problem may still 
be related to the reference sequence file:)

Original comment by li.david...@gmail.com on 13 Jul 2010 at 8:26

[GoogleCodeExporter]

I also have this issue with Flux Simulator 20100705 on the following machine:

OS:Ubuntu 10.04 (lucid)
uname:Linux 2.6.36-020636rc4-generic #201009130905 SMP Mon Sep 13 09:08:38 UTC 
2010 x86_64
libc:glibc 2.11.1 NPTL 2.11.1 
rlimit: STACK 8192k, CORE 0k, NPROC infinity, NOFILE 1024, AS infinity
load average:5.06 4.14 3.48

CPU:total 4 (8 cores per cpu, 2 threads per core) family 6 model 37 stepping 5, 
cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, ht

Memory: 4k page, physical 8118232k(162016k free), swap 8787964k(8545252k free)

vm_info: OpenJDK 64-Bit Server VM (16.0-b13) for linux-amd64 JRE 
(1.6.0_18-b18), built on Jul 26 2010 23:34:55 by "buildd" with gcc 4.4.3
-------

I don't have it all the time, using the same parameters, the same genome, the 
sequencing may succeed or not. It happened on different genomes (drosophila, 
human) and with a large number or reads of length 75 (> 12M). I don't remember 
I had it when I simulated a smaller number of reads.

FASTQ is set to YES and PAIR_END is set to NO

Original comment by mikael.s...@gmail.com on 22 Sep 2010 at 4:45

[GoogleCodeExporter]

Thank you for reporting your observations, my current feeling here is that we 
talk about a threading issue under heavy duty during sequencing, that impacts 
according to system-dependent circumstances which are probably complex to 
estimate. The reported paired-end switch effect could fit under this 
hypothesis, as it doubles the load and disk activivity of each thread. I 
currently do not think that it is a general effect of the input annotation, 
some argument against that is that I ran the simulator with equal sized or 
larger and equally/more complex annotations (e.g., human GENCODE). 
Nevertheless, I will include UCSC mm9 in my tests, looping over runs with large 
read number and length (maybe 50M, 100nt).

As a complementary test, could affected people check/confirm whether 
fasta/fastq output yes/no makes a difference for them? If you would be ready to 
test a bundle before putting it online, please contact me by email 
(micha1sammeth2net, replacing 1 and 2 by corresponding non-alphanumeric 
characters).

Original comment by gmicha@gmail.com on 2 Dec 2010 at 4:09

• Changed state: Started

[GoogleCodeExporter]

I revised some reports on such system- and runtime dependent crashes in the 
native code during safepoint synchronization when thread load is high, 
especially some openJDK releases seem to show problems. The current 
recommendation is:

- get for your system a runtime environment 1.5 or higher from sun/oracle 
http://www.java.com/en/download/manual.jsp

- make sure you are using <a 
href="http://code.google.com/p/fluxcapacitor/downloads/detail?name=FluxSimulator
-20101209.tar.gz&can=2&q=">build20101209</a> or afterwards, and add to the 
parameter file the line
MAXTHREAD 1

- and if you want to support development, send me the crash report. You can 
find it at the location specified in the stderr output after
# An error report file with more information is saved as:

Fixed/Avoided in build 20101209 and later.

Original comment by gmicha@gmail.com on 9 Dec 2010 at 6:59

• Changed state: Fixed