search

pixie16 / paass

Pixie Acquisition and Analysis Software Suite
https://pixie16.github.io/paassdoc/
GNU General Public License v3.0
10 stars 29 forks source link

XiaListModeDecoder causes crash when headerLength == STATS_BLOCK #262

Closed rin-yokoyama closed 7 years ago

rin-yokoyama commented 7 years ago

Description

I was investigating the crash of utkscan on some Steven's data at ANL. While scanning some of his data, utkscan suddenly start consuming memory significantly and was killed eventually. I found that it was crashing at XiaListModeDecoder::DecodeBuffer(). Some lines were commented out in one of the switch cases, case STATS_BLOCK (like this). One of the lines, "buf+=eventLength", should not be commented out because the buf pointer stays at the same place and it loops forever creating new XiaData instances otherwise.

The ANL1471 data seem to contain some blocks with headerLength == 1 (STATS_BLOCK). I am not yet sure if those blocks in the data are actually the stats blocks or some kind of error blocks, but anyway the code should handle such unusual blocks without crashing.

Work to be done

Modify XiaListModeDecoder to handle or skip a block with headerLength == 1 without causing a crash.

Acceptance Criteria

  1. All of software compiles
  2. Should not loop forever in the while(true) statement of the DecodeBuffer() method.
  3. Be able to analyze the rest of data behind the block.
ksmith0 commented 7 years ago

Was this data written with poll2 or pacman? You may want to consider scanning the data with a tool such as evt2root or hexdump to ensure that the data file is properly constructed.

sztaylor89 commented 7 years ago

It was written with pacman. This was before poll2 existed yet. @ksmith0 how would I go about checking the data with hexdump or evt2root? are these built somewhere as part of paass? What would I be looking for?

ksmith0 commented 7 years ago

The hexdump program is something that @cthornsb wrote and I'm not sure where it lives.

I wrote evt2root, you should use version 3 (branch v3). The program is not capable of unpacking the XIA data, but will unpack the surrounding HRIBF binary format. You will want to use the evtDump program and pipe it to less then search for the word 'ERROR'.

evtDump -f hribf run0.ldf | less
ksmith0 commented 7 years ago

After looking at the code I believe @rin-yokoyama may be correct. Since this data was acquired with poll1 it is most likely that these are indeed the stats blocks (something that still needs to be added to poll2). They should at least be added to the decoder so that the unpacking scheme is there.

sztaylor89 commented 7 years ago

I get this when running this: ./evtDump -f hribf /scrat2/anl2015/FEB2015/135SB/a135feb_12-10 > out.txt

Searching for "error" comes up with no results

out.txt

sztaylor89 commented 7 years ago

Here are the files I ran the other day, and whether or not they completed, and how far they got. screenshot_20170501-102344

ksmith0 commented 7 years ago

Your evtDump output indicates that the HIRBF format is not a problem so that can be eliminated. Adding the length in time of each of your run files to your nice table will help make it clear if the stats buffers are being created. I still suspect that @rin-yokoyama is correct about the issue. Poll2 did not have stats block output and thus the decoder wasn't built to handle it. It should be added both to the polling and decoding.

sztaylor89 commented 7 years ago

length of time to scan them, or length of actual ldf run?

ksmith0 commented 7 years ago

Length of run time.

Also, I've opened issue #263 about restoring the stats block to poll2. This will help keep the discussion separate.

sztaylor89 commented 7 years ago

Every file other than # 15 ran for right about 66 mins, while # 15 ran for about 33 mins. I got these numbers based off of file sizes and the start and stop times listed in the logbook.

ksmith0 commented 7 years ago

The statsInterval (time between stats blocks) is a command line argument passed to the poll program. It could be the case that stats were enabled for some runs, while they were not for the others.

sztaylor89 commented 7 years ago

But all of these ldfs were simply rollover ones automatically created when the 2.0GB file size limit was reached. Wouldn't they all have the same command line arguments?

ksmith0 commented 7 years ago

But all of these ldfs were simply rollover ones automatically created when the 2.0GB file size limit was reached.

Ahh, I didn't realize that. Should have the same arguments then.

sztaylor89 commented 7 years ago

So after rescaning with Rin's fix, file numbers 10 and 14 scanned to completeness, while file number 15 gave this at 90%: (Sorry they are screen captures, I couldn't figure out how to copy the text from my tmux session on quack) screenshot from 2017-05-02 13-21-03 screenshot from 2017-05-02 13-21-31

ksmith0 commented 7 years ago

(Sorry they are screen captures, I couldn't figure out how to copy the text from my tmux session on quack)

Hold shift while highlighting.

ksmith0 commented 7 years ago

If you compile with CMAKE_BUILD_TYPE=Debug, the backtrace will be more useful.

sztaylor89 commented 7 years ago

I'll remake and run this particular file again. And thanks for the tmux tip. I was trying all kinds of weird things about entering copy mode in emacs in tmux...., nothing worked.

sztaylor89 commented 7 years ago

Here's the output for that same file( # 15) crash using the debug mode:

[READ] 60317 words (90%), GOOD = 55620, LOST = 2564WaveformAnalyzer::Analyze - TraceFunctions::ComputeBaseline - The range specified is smaller than the minimum necessary range.
WaveformAnalyzer::Analyze - TraceFunctions::ComputeBaseline - The range specified is smaller than the minimum necessary range.
WaveformAnalyzer::Analyze - TraceFunctions::ComputeBaseline - The range specified is smaller than the minimum necessary range.
WaveformAnalyzer::Analyze - TraceFunctions::ComputeBaseline - The range specified is smaller than the minimum necessary range.
WaveformAnalyzer::Analyze - TraceFunctions::ComputeBaseline - The range specified is smaller than the minimum necessary range.
WaveformAnalyzer::Analyze - TraceFunctions::ComputeBaseline - The range specified is smaller than the minimum necessary range.
WaveformAnalyzer::Analyze - TraceFunctions::ComputeBaseline - The range specified is smaller than the minimum necessary range.
*** Error in `./utkscan': double free or corruption (out): 0x0000000003c9df30 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x7c503)[0x7fd52b8ac503]
./utkscan(_ZN9__gnu_cxx13new_allocatorIPP7XiaDataE10deallocateEPS3_m+0x26)[0x58ee96]
./utkscan(_ZNSt11_Deque_baseIP7XiaDataSaIS1_EE17_M_deallocate_mapEPPS1_m+0x45)[0x58dba3]
./utkscan(_ZNSt5dequeIP7XiaDataSaIS1_EE17_M_reallocate_mapEmb+0x1d3)[0x58ebfb]
./utkscan(_ZNSt5dequeIP7XiaDataSaIS1_EE22_M_reserve_map_at_backEm+0x65)[0x58d891]
./utkscan(_ZNSt5dequeIP7XiaDataSaIS1_EE16_M_push_back_auxIJRKS1_EEEvDpOT_+0x28)[0x58cd56]
./utkscan(_ZNSt5dequeIP7XiaDataSaIS1_EE9push_backERKS1_+0x73)[0x58c583]
./utkscan(_ZN8Unpacker8AddEventEP7XiaData+0xee)[0x58b296]
./utkscan(_ZN8Unpacker10ReadBufferEPj+0xe9)[0x58b617]
./utkscan(_ZN8Unpacker9ReadSpillEPjjb+0x294)[0x58bb56]
./utkscan(_ZN13ScanInterface10RunControlEv+0xf8c)[0x582b88]
./utkscan(_Z17start_run_controlP13ScanInterface+0x1e)[0x57d8aa]
./utkscan(_ZN13ScanInterface7ExecuteEv+0x134)[0x5859b8]
./utkscan(main+0x14c)[0x4d97c9]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x7fd52b851b35]
./utkscan[0x4d9579]
======= Memory map: ========
00400000-00637000 r-xp 00000000 fd:02 59113598                           /home/sztaylor/paass/install/bin/utkscan
00836000-0083a000 r--p 00236000 fd:02 59113598                           /home/sztaylor/paass/install/bin/utkscan
0083a000-0083b000 rw-p 0023a000 fd:02 59113598                           /home/sztaylor/paass/install/bin/utkscan
0083b000-0083c000 rw-p 00000000 00:00 0 
0280b000-07047000 rw-p 00000000 00:00 0                                  [heap]
7fd508000000-7fd508021000 rw-p 00000000 00:00 0 
7fd508021000-7fd50c000000 ---p 00000000 00:00 0 
7fd50de9d000-7fd51696c000 rw-p 00000000 00:00 0 
7fd51696c000-7fd525f86000 r--p 00000000 fd:00 1202103                    /opt/root/6.08.02/etc/allDict.cxx.pch
7fd525fb7000-7fd525fbe000 r-xp 00000000 fd:00 788758                     /usr/lib64/librt-2.17.so
7fd525fbe000-7fd5261bd000 ---p 00007000 fd:00 788758                     /usr/lib64/librt-2.17.so
7fd5261bd000-7fd5261be000 r--p 00006000 fd:00 788758                     /usr/lib64/librt-2.17.so
7fd5261be000-7fd5261bf000 rw-p 00007000 fd:00 788758                     /usr/lib64/librt-2.17.so
7fd5261e6000-7fd528560000 r-xp 00000000 fd:00 1341526                    /opt/root/6.08.02/lib/libCling.so.6.08.02
7fd528560000-7fd52875f000 ---p 0237a000 fd:00 1341526                    /opt/root/6.08.02/lib/libCling.so.6.08.02
7fd52875f000-7fd528939000 r--p 02379000 fd:00 1341526                    /opt/root/6.08.02/lib/libCling.so.6.08.02
7fd528939000-7fd528940000 rw-p 02553000 fd:00 1341526                    /opt/root/6.08.02/lib/libCling.so.6.08.02
7fd528940000-7fd52895f000 rw-p 00000000 00:00 0 
7fd52895f000-7fd528964000 r-xp 00000000 fd:00 788744                     /usr/lib64/libnss_dns-2.17.so
7fd528964000-7fd528b63000 ---p 00005000 fd:00 788744                     /usr/lib64/libnss_dns-2.17.so
7fd528b63000-7fd528b64000 r--p 00004000 fd:00 788744                     /usr/lib64/libnss_dns-2.17.so
7fd528b64000-7fd528b65000 rw-p 00005000 fd:00 788744                     /usr/lib64/libnss_dns-2.17.so
7fd528b65000-7fd528b71000 r-xp 00000000 fd:00 788746                     /usr/lib64/libnss_files-2.17.so
7fd528b71000-7fd528d70000 ---p 0000c000 fd:00 788746                     /usr/lib64/libnss_files-2.17.so
7fd528d70000-7fd528d71000 r--p 0000b000 fd:00 788746                     /usr/lib64/libnss_files-2.17.so
7fd528d71000-7fd528d72000 rw-p 0000c000 fd:00 788746                     /usr/lib64/libnss_files-2.17.so
7fd528d72000-7fd528d78000 rw-p 00000000 00:00 0 
7fd528d78000-7fd528d9c000 r-xp 00000000 fd:00 789165                     /usr/lib64/libselinux.so.1
7fd528d9c000-7fd528f9b000 ---p 00024000 fd:00 789165                     /usr/lib64/libselinux.so.1
7fd528f9b000-7fd528f9c000 r--p 00023000 fd:00 789165                     /usr/lib64/libselinux.so.1
7fd528f9c000-7fd528f9d000 rw-p 00024000 fd:00 789165                     /usr/lib64/libselinux.so.1
7fd528f9d000-7fd528f9f000 rw-p 00000000 00:00 0 
7fd528f9f000-7fd528fb5000 r-xp 00000000 fd:00 788756                     /usr/lib64/libresolv-2.17.so
7fd528fb5000-7fd5291b5000 ---p 00016000 fd:00 788756                     /usr/lib64/libresolv-2.17.so
7fd5291b5000-7fd5291b6000 r--p 00016000 fd:00 788756                     /usr/lib64/libresolv-2.17.so
7fd5291b6000-7fd5291b7000 rw-p 00017000 fd:00 788756                     /usr/lib64/libresolv-2.17.so
7fd5291b7000-7fd5291b9000 rw-p 00000000 00:00 0 
7fd5291b9000-7fd5291bc000 r-xp 00000000 fd:00 789458                     /usr/lib64/libkeyutils.so.1.5
7fd5291bc000-7fd5293bb000 ---p 00003000 fd:00 789458                     /usr/lib64/libkeyutils.so.1.5
7fd5293bb000-7fd5293bc000 r--p 00002000 fd:00 789458                     /usr/lib64/libkeyutils.so.1.5
7fd5293bc000-7fd5293bd000 rw-p 00003000 fd:00 789458                     /usr/lib64/libkeyutils.so.1.5
7fd5293bd000-7fd5293ca000 r-xp 00000000 fd:00 794551                     /usr/lib64/libkrb5support.so.0.1
7fd5293ca000-7fd5295ca000 ---p 0000d000 fd:00 794551                     /usr/lib64/libkrb5support.so.0.1
7fd5295ca000-7fd5295cb000 r--p 0000d000 fd:00 794551                     /usr/lib64/libkrb5support.so.0.1
7fd5295cb000-7fd5295cc000 rw-p 0000e000 fd:00 794551                     /usr/lib64/libkrb5support.so.0.1
7fd5295cc000-7fd5295ce000 r-xp 00000000 fd:00 788689                     /usr/lib64/libfreebl3.so
7fd5295ce000-7fd5297cd000 ---p 00002000 fd:00 788689                     /usr/lib64/libfreebl3.so
7fd5297cd000-7fd5297ce000 r--p 00001000 fd:00 788689                     /usr/lib64/libfreebl3.so
7fd5297ce000-7fd5297cf000 rw-p 00002000 fd:00 788689                     /usr/lib64/libfreebl3.so
7fd5297cf000-7fd5297fe000 r-xp 00000000 fd:00 794543                     /usr/lib64/libk5crypto.so.3.1
7fd5297fe000-7fd5299fd000 ---p 0002f000 fd:00 794543                     /usr/lib64/libk5crypto.so.3.1
7fd5299fd000-7fd5299ff000 r--p 0002e000 fd:00 794543                     /usr/lib64/libk5crypto.so.3.1
7fd5299ff000-7fd529a00000 rw-p 00030000 fd:00 794543                     /usr/lib64/libk5crypto.so.3.1
7fd529a00000-7fd529a01000 rw-p 00000000 00:00 0 
7fd529a01000-7fd529a04000 r-xp 00000000 fd:00 789184                     /usr/lib64/libcom_err.so.2.1
7fd529a04000-7fd529c03000 ---p 00003000 fd:00 789184                     /usr/lib64/libcom_err.so.2.1
7fd529c03000-7fd529c04000 r--p 00002000 fd:00 789184                     /usr/lib64/libcom_err.so.2.1
7fd529c04000-7fd529c05000 rw-p 00003000 fd:00 789184                     /usr/lib64/libcom_err.so.2.1
7fd529c05000-7fd529cdb000 r-xp 00000000 fd:00 794549                     /usr/lib64/libkrb5.so.3.3
7fd529cdb000-7fd529edb000 ---p 000d6000 fd:00 794549                     /usr/lib64/libkrb5.so.3.3
7fd529edb000-7fd529ee9000 r--p 000d6000 fd:00 794549                     /usr/lib64/libkrb5.so.3.3
7fd529ee9000-7fd529eec000 rw-p 000e4000 fd:00 794549                     /usr/lib64/libkrb5.so.3.3
7fd529eec000-7fd529f37000 r-xp 00000000 fd:00 789176                     /usr/lib64/libgssapi_krb5.so.2.2
7fd529f37000-7fd52a137000 ---p 0004b000 fd:00 789176                     /usr/lib64/libgssapi_krb5.so.2.2
7fd52a137000-7fd52a138000 r--p 0004b000 fd:00 789176                     /usr/lib64/libgssapi_krb5.so.2.2
7fd52a138000-7fd52a13a000 rw-p 0004c000 fd:00 789176                     /usr/lib64/libgssapi_krb5.so.2.2
7fd52a13a000-7fd52a248000 r-xp 00000000 fd:00 1342167                    /opt/root/6.08.02/lib/libTreePlayer.so.6.08.02
7fd52a248000-7fd52a448000 ---p 0010e000 fd:00 1342167                    /opt/root/6.08.02/lib/libTreePlayer.so.6.08.02
7fd52a448000-7fd52a452000 r--p 0010e000 fd:00 1342167                    /opt/root/6.08.02/lib/libTreePlayer.so.6.08.02
7fd52a452000-7fd52a454000 rw-p 00118000 fd:00 1342167                    /opt/root/6.08.02/lib/libTreePlayer.so.6.08.02
7fd52a454000-7fd52a45b000 rw-p 00000000 00:00 0 
7fd52a45b000-7fd52a4fb000 r-xp 00000000 fd:00 789172                     /usr/lib64/libfreetype.so.6.10.0
7fd52a4fb000-7fd52a6fa000 ---p 000a0000 fd:00 789172                     /usr/lib64/libfreetype.so.6.10.0
7fd52a6fa000-7fd52a700000 r--p 0009f000 fd:00 789172                     /usr/lib64/libfreetype.so.6.10.0
7fd52a700000-7fd52a701000 rw-p 000a5000 fd:00 789172                     /usr/lib64/libfreetype.so.6.10.0
7fd52a701000-7fd52a709000 r-xp 00000000 fd:00 788732                     /usr/lib64/libcrypt-2.17.so
7fd52a709000-7fd52a908000 ---p 00008000 fd:00 788732                     /usr/lib64/libcrypt-2.17.so
7fd52a908000-7fd52a909000 r--p 00007000 fd:00 788732                     /usr/lib64/libcrypt-2.17.so
7fd52a909000-7fd52a90a000 rw-p 00008000 fd:00 788732                     /usr/lib64/libcrypt-2.17.so
7fd52a90a000-7fd52a938000 rw-p 00000000 00:00 0 
7fd52a938000-7fd52aaf8000 r-xp 00000000 fd:00 794562                     /usr/lib64/libcrypto.so.1.0.1e
7fd52aaf8000-7fd52acf8000 ---p 001c0000 fd:00 794562                     /usr/lib64/libcrypto.so.1.0.1e
7fd52acf8000-7fd52ad12000 r--p 001c0000 fd:00 794562                     /usr/lib64/libcrypto.so.1.0.1e
7fd52ad12000-7fd52ad1e000 rw-p 001da000 fd:00 794562                     /usr/lib64/libcrypto.so.1.0.1e
7fd52ad1e000-7fd52ad22000 rw-p 00000000 00:00 0 
7fd52ad22000-7fd52ad86000 r-xp 00000000 fd:00 794564                     /usr/lib64/libssl.so.1.0.1e
7fd52ad86000-7fd52af85000 ---p 00064000 fd:00 794564                     /usr/lib64/libssl.so.1.0.1e
7fd52af85000-7fd52af89000 r--p 00063000 fd:00 794564                     /usr/lib64/libssl.so.1.0.1e
7fd52af89000-7fd52af90000 rw-p 00067000 fd:00 794564                     /usr/lib64/libssl.so.1.0.1e
7fd52af90000-7fd52afa5000 r-xp 00000000 fd:00 789168                     /usr/lib64/libz.so.1.2.7
7fd52afa5000-7fd52b1a4000 ---p 00015000 fd:00 789168                     /usr/lib64/libz.so.1.2.7
7fd52b1a4000-7fd52b1a5000 r--p 00014000 fd:00 789168                     /usr/lib64/libz.so.1.2.7
7fd52b1a5000-7fd52b1a6000 rw-p 00015000 fd:00 789168                     /usr/lib64/libz.so.1.2.7
7fd52b1a6000-7fd52b1cb000 r-xp 00000000 fd:00 789238                     /usr/lib64/liblzma.so.5.2.2
7fd52b1cb000-7fd52b3ca000 ---p 00025000 fd:00 789238                     /usr/lib64/liblzma.so.5.2.2
7fd52b3ca000-7fd52b3cb000 r--p 00024000 fd:00 789238                     /usr/lib64/liblzma.so.5.2.2
7fd52b3cb000-7fd52b3cc000 rw-p 00025000 fd:00 789238                     /usr/lib64/liblzma.so.5.2.2
7fd52b3cc000-7fd52b3ce000 r-xp 00000000 fd:00 789163                     /usr/lib64/libpcreposix.so.0.0.1
7fd52b3ce000-7fd52b5cd000 ---p 00002000 fd:00 789163                     /usr/lib64/libpcreposix.so.0.0.1
7fd52b5cd000-7fd52b5ce000 r--p 00001000 fd:00 789163                     /usr/lib64/libpcreposix.so.0.0.1
7fd52b5ce000-7fd52b5cf000 rw-p 00002000 fd:00 789163                     /usr/lib64/libpcreposix.so.0.0.1
7fd52b5cf000-7fd52b62f000 r-xp 00000000 fd:00 789155                     /usr/lib64/libpcre.so.1.2.0
7fd52b62f000-7fd52b82e000 ---p 00060000 fd:00 789155                     /usr/lib64/libpcre.so.1.2.0
7fd52b82e000-7fd52b82f000 r--p 0005f000 fd:00 789155                     /usr/lib64/libpcre.so.1.2.0
7fd52b82f000-7fd52b830000 rw-p 00060000 fd:00 789155                     /usr/lib64/libpcre.so.1.2.0
7fd52b830000-7fd52b9e6000 r-xp 00000000 fd:00 788728                     /usr/lib64/libc-2.17.so
7fd52b9e6000-7fd52bbe6000 ---p 001b6000 fd:00 788728                     /usr/lib64/libc-2.17.so
7fd52bbe6000-7fd52bbea000 r--p 001b6000 fd:00 788728                     /usr/lib64/libc-2.17.so
7fd52bbea000-7fd52bbec000 rw-p 001ba000 fd:00 788728                     /usr/lib64/libc-2.17.so
7fd52bbec000-7fd52bbf1000 rw-p 00000000 00:00 0 
7fd52bbf1000-7fd52bc06000 r-xp 00000000 fd:00 786451                     /usr/lib64/libgcc_s-4.8.5-20150702.so.1
7fd52bc06000-7fd52be05000 ---p 00015000 fd:00 786451                     /usr/lib64/libgcc_s-4.8.5-20150702.so.1
7fd52be05000-7fd52be06000 r--p 00014000 fd:00 786451                     /usr/lib64/libgcc_s-4.8.5-20150702.so.1
7fd52be06000-7fd52be07000 rw-p 00015000 fd:00 786451                     /usr/lib64/libgcc_s-4.8.5-20150702.so.1
7fd52be07000-7fd52bef0000 r-xp 00000000 fd:00 789034                     /usr/lib64/libstdc++.so.6.0.19
7fd52bef0000-7fd52c0f0000 ---p 000e9000 fd:00 789034                     /usr/lib64/libstdc++.so.6.0.19
7fd52c0f0000-7fd52c0f9000 r--p 000e9000 fd:00 789034                     /usr/lib64/libstdc++.so.6.0.19
7fd52c0f9000-7fd52c0fb000 rw-p 000f2000 fd:00 789034                     /usr/lib64/libstdc++.so.6.0.19
7fd52c0fb000-7fd52c110000 rw-p 00000000 00:00 0 
7fd52c110000-7fd52c11e000 r-xp 00000000 fd:00 789043                     /usr/lib64/libform.so.5.9
7fd52c11e000-7fd52c31e000 ---p 0000e000 fd:00 789043                     /usr/lib64/libform.so.5.9
7fd52c31e000-7fd52c31f000 r--p 0000e000 fd:00 789043                     /usr/lib64/libform.so.5.9
7fd52c31f000-7fd52c320000 rw-p 0000f000 fd:00 789043                     /usr/lib64/libform.so.5.9
7fd52c320000-7fd52c345000 r-xp 00000000 fd:00 789065                     /usr/lib64/libtinfo.so.5.9
7fd52c345000-7fd52c545000 ---p 00025000 fd:00 789065                     /usr/lib64/libtinfo.so.5.9
7fd52c545000-7fd52c549000 r--p 00025000 fd:00 789065                     /usr/lib64/libtinfo.so.5.9
7fd52c549000-7fd52c54a000 rw-p 00029000 fd:00 789065                     /usr/lib64/libtinfo.so.5.9
7fd52c54a000-7fd52c570000 r-xp 00000000 fd:00 789055                     /usr/lib64/libncurses.so.5.9
7fd52c570000-7fd52c76f000 ---p 00026000 fd:00 789055                     /usr/lib64/libncurses.so.5.9
7fd52c76f000-7fd52c770000 r--p 00025000 fd:00 789055                     /usr/lib64/libncurses.so.5.9
7fd52c770000-7fd52c771000 rw-p 00026000 fd:00 789055                     /usr/lib64/libncurses.so.5.9
7fd52c771000-7fd52c788000 r-xp 00000000 fd:00 788754                     /usr/lib64/libpthread-2.17.so
7fd52c788000-7fd52c987000 ---p 00017000 fd:00 788754                     /usr/lib64/libpthread-2.17.so
7fd52c987000-7fd52c988000 r--p 00016000 fd:00 788754                     /usr/lib64/libpthread-2.17.so
7fd52c988000-7fd52c989000 rw-p 00017000 fd:00 788754                     /usr/lib64/libpthread-2.17.so
7fd52c989000-7fd52c98d000 rw-p 00000000 00:00 0 
7fd52c98d000-7fd52c98f000 r-xp 00000000 fd:00 788734                     /usr/lib64/libdl-2.17.so
7fd52c98f000-7fd52cb8f000 ---p 00002000 fd:00 788734                     /usr/lib64/libdl-2.17.so
[READ] 254486 words (90%), GOOD = 55683, LOST = 2564./run_scan.sh: line 3:  1762 Aborted                 (core dumped) ./utkscan -c Configs/Config_135_121616.xml -o aaa15-3 -f R30981 --frequency 250 -i Data/a135feb_12-15.ldf -b
cthornsb commented 7 years ago

As a side note, I am concerned about the number of lost spill chunks (lost at least 2546 of 55683 chunks for this file). That is a significant portion (almost 5%) which represents incomplete data spills. Was the data rate really high for these runs? This could be a result of ethernet data packet loss when using pacman.

ksmith0 commented 7 years ago

I concur with Cory's evaluation. I don't think that the losses are that significant. As it's a decay measurement the rate of loss will not be very important as long as it's uniform.

cthornsb commented 7 years ago

That's true. It could be a problem, however, if a similar setup is used for a reaction experiment and these losses aren't taken into account. But it's probably not an issue in this case.

spaulaus commented 7 years ago

This version of poll had started to be extremely temperamental. The we were plagued with rate related issues and corrupted spills. I suspect that's the cause behind the lost spill chunks.

spaulaus commented 7 years ago

It was written with pacman. This was before poll2 existed yet.

To clarify this response, the data was read out and packaged with poll, and then given to the HRIBF DAQ for writing to disk.

spaulaus commented 7 years ago

So after rescaning with Rin's fix,

Is that fix documented anywhere?

rin-yokoyama commented 7 years ago

I think he tried adding one of the commented out lines, "buf+=eventLength;", back.

spaulaus commented 7 years ago

This is not a viable solution for other datasets. These advances were commented b/c they were causing instability in data reading.

BTW : This duplicates #228

ksmith0 commented 7 years ago

Closing this as it is a duplicate.