Multiple scattering simulation fails with 1e7 particles on Windows

[Anders-Markvardsen]

Platform: Windows McStas version 2.4.1 (Jun. 26, 2017)

When trying to run

Vanadium.zip

to generate a Nexus output that can be read into Mantid this works when the particle count is 1e6 (or lower), in which case the terminal output is:

If however I increase the particle count to 1e7 the terminal is:

In both cases a nexus output is returned, but in the latter case only parts of the data are saved. This can be seen from this HDFview:

[willend]

@Anders-Markvardsen, I can confirm the behaviour.

@farhi could you help have a short look at this? I never saw the error message before on other systems: The (re-)allocation of the event list buffer at https://github.com/McStasMcXtrace/McCode/blob/af917bf4a5df157893fdd0a3a07a3082212aef8a/mcstas-comps/share/monitor_nd-lib.c#L801 fails. Are there any options= keywords to avoid the realloc (Flag_List=3 mode ?) for a case of storing "all" events?

[ebknudsen]

@Anders-Markvardsen @willend : Do you get the same problem when not using NeXus?

[willend]

Hi @ebknudsen: If the file is compiled with NeXus and run without NeXus, everything works OK here... So NeXus format during runtime is part of the equation.

[Anders-Markvardsen]

If I run templateSANS_Mantid in attached zip with 1e7 particles:

templateSANS_Mantid.zip

I get

[willend]

Hi @Anders-Markvardsen,

I've made a compile-time define that can be used as a workaround for the issue. To try this out, use this installer: http://tmp.mcstas.org/McStas-Metapackage-2.5beta02-win32.exe (corrected version/link)

and define

-DMONND_BUFSIZ=100000

in your mcgui CFLAGS area.

The image below shows the "usual" simulation of 1e7 with NeXus completing OK:

[Anders-Markvardsen]

I am trying to install your new windows build above. When I install I see the following:

I am not sure if I need to be concerned with this? After installing and within the bin folder I have mcgui.bat and mcgui-pl.bat. The latter starts up, but not if I should use it? The former complains it can find module 'PyQt5'. Do I need to install PyQt separately?

[willend]

@Anders-Markvardsen sorry about that, had forgotten this needed fixing after rolling the beta. Will build a new one ASAP.

[willend]

@Anders-Markvardsen could you try the newly uploaded http://tmp.mcstas.org/McStas-Metapackage-2.5beta02-win32.exe - may prompt you still for an OK/[y] to install the Python stuff, but at least everything will be downloaded / installed automatically

[Anders-Markvardsen]

The above installer installed well, however I find it produces the same results as before.

The results from the mcgui log window running on the instrument file attached in the zip above:

McStas version 2.5beta02 (Feb. 20, 2018) Copyright (C) DTU Physics and Risoe National Laboratory, 1997-2018 Additions (C) Institut Laue Langevin, 2003-2018 All rights reserved

• Component library: McStas-2.5beta02, built Feb. 20, 2018

Instrument opened: Vanadium.instr Setting environment: MCSTAS_OVERRIDE= C:\mcstas-2.5beta02\lib\tools\Python\mcgui..\mccodelib......

MCSTAS_CFLAGS_OVERRIDE= -g -O2 -lm -DUSE_NEXUS -llibNeXus-0 -I "C:/Program Files (x86)/NeXus Data Format/include/nexus" -L "C:/Program Files (x86)/NeXus Data Format/lib/nexus" -DMONND_BUFSIZ=100000

MCSTAS_CC_OVERRIDE= gcc

MCSTAS_MPICC_OVERRIDE= mpicc.bat

mcrun --format=NeXus -c Vanadium.instr -n0 loading user configuration from C:\Users\ajm64\AppData\mcstas\2.5beta02\mccode_config.json INFO: No output directory specified (--dir) INFO: Using directory: "Vanadium_20180220_211517" INFO: Recompiling: Vanadium.exe INFO: Regenerating c-file: Vanadium.c CFLAGS= INFO: === compile thread done

INFO: No output directory specified (--dir) INFO: Using directory: "Vanadium_20180220_211530" INFO: Using existing binary: Vanadium.exe INFO: ===

start mcdisplay-pl --format=Mantid Vanadium.instr --no-output-files -n 1 INFO: No output directory specified (--dir) INFO: Using directory: "Vanadium_20180220_211545" INFO: Using existing binary: Vanadium.exe INFO: ===

start mcrun --format=NeXus Vanadium.instr -d 1e7new -n 1e7

And the cmd windows show

And my mccode_config.json is:

{"configuration": {"MCCODE_VERSION": "2.5beta02", "MCCODE_LIB_DIR": "C:\mcstas-2.5beta02\lib\tools\Python\mcgui\..\mccodelib\..\..\..", "MCCODE": "mcstas", "MCRUN": "mcrun --format=NeXus", "MCPLOT": "mcplot-pyqtgraph", "MCDISPLAY": "mcdisplay-pl --format=Mantid", "TOOL_NAME": "mcgui", "PARTICLE": "neutron", "BROWSER": "start"}, "compilation": {"CFLAGS": "-g -O2 -lm -DUSE_NEXUS -llibNeXus-0 -I \"C:/Program Files (x86)/NeXus Data Format/include/nexus\" -L \"C:/Program Files (x86)/NeXus Data Format/lib/nexus\" -DMONND_BUFSIZ=100000", "NEXUSFLAGS": "-DUSE_NEXUS -lNeXus", "MPIFLAGS": "-DUSE_MPI -lmpi", "CC": "gcc", "MPICC": "mpicc.bat", "MPIRUN": "mpiexec.exe", "MPINODES": "4"}, "platform": {"EXESUFFIX": "exe"}}

[willend]

Everything works OK here. Are you sure you recompiled before running the simulation? (Either through a trivial edit of the instr file or through Simulation->Compile.)

[willend]

Generated NeXus file attached in this zip: mccode.h5.zip

[willend]

And BTW I ran with

-DMONND_BUFSIZ=1000000

(1e6) - looks like you have 1e5 in the above log (original default in the mccode-r.c code is 1e4)

[Anders-Markvardsen]

Good news and well I beleive less good news. With -DMONND_BUFSIZ=1000000 it works when I am using 1e7 particles, but it fails when I am using 1e8 with the error:

[willend]

Yes, I knew this would more or less be the case, sorry. :-) - This is also why I labeled this a "workaround"... As long as your memory allows you may increase MONND_BUFSIZ - I guess a factor of 10 will do it here (total will still be less than GB's I think)

[Anders-Markvardsen]

When I increase by a factor of 10 I get:

Can you explain again how you suggest setting MONND_BUFSIZ for different particle numbers please and why?

[willend]

As said it is a workaround, not a proper patch or solution.

It works by having a buffer which is "large enough" to hold all event data pr. monitor without writing these to disk/hdf file.

And I must admit I didn't look into the maths of ncount vs. MONND_BUFSIZ enough that I can explain the need for a 100x bigger buffer to hold a 10x larger ncount "yield".

Will go to a workshop / codecamp in Dresden in march with @farhi who is the most well-versed of us into the details of the Monitor_nD logic, hope to have a closer look at the issue there.

[willend]

It looks like this may be solved by a recent commit (https://github.com/McStasMcXtrace/McCode/issues/689). Currently running a test with the old 1e4 BUFSIZ to check

[willend]

Yup, current master build confirmed to work!

Both the above-mentioned patch and our 64bit build are likely elements of this coming in place.

[Anders-Markvardsen]

Cheers