Closed fouriaux closed 7 years ago
I cannot recreate the issue on my VM with gnu compiler, with either the master HEAD or 49b720f, with either single process or 4 process runs. How did you run the tests?
using srun -n 4
on each executable I got the following output files:
du -h *.dat 64K 10_write_max_buffer_1M.dat 1.0M 10_write_max_buffer_4M.dat 1.0M 1_write_max_buffer_1M.dat 1.0M 1_write_max_buffer_4M.dat
[fouriaux@bbpviz1 build]$ $ADIOS_ROOT/bin/bpls 10_write_max_buffer_1M.dat integer global_size scalar integer batch_size scalar integer offset scalar byte data {40960} [fouriaux@bbpviz1 build]$ $ADIOS_ROOT/bin/bpls 10_write_max_buffer_4M.dat integer global_size scalar integer batch_size scalar integer offset scalar byte data {40960}
Can you please attach the 'bpdump' output for each file. I found no difference between the two in my runs.
On Tue, Jul 4, 2017 at 3:23 PM, Jeremy FOURIAUX notifications@github.com wrote:
using srun -n 4 on each executable I got the following output files:
du -h *.dat 64K 10_write_max_buffer_1M.dat 1.0M 10_write_max_buffer_4M.dat 1.0M 1_write_max_buffer_1M.dat 1.0M 1_write_max_buffer_4M.dat
[fouriaux@bbpviz1 build]$ $ADIOS_ROOT/bin/bpls 10_write_max_buffer_1M.dat integer global_size scalar integer batch_size scalar integer offset scalar byte data {40960} [fouriaux@bbpviz1 build]$ $ADIOS_ROOT/bin/bpls 10_write_max_buffer_4M.dat integer global_size scalar integer batch_size scalar integer offset scalar byte data {40960}
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ornladios/ADIOS/issues/136#issuecomment-312937894, or mute the thread https://github.com/notifications/unsubscribe-auth/ADGMLXP6ZZT7hdPSiJd-YcOO4yHi3yW4ks5sKpFPgaJpZM4ONe_I .
actually I have recompiled everything, and I cannot reproduce this problem anymore. I will check what difference could have happened between those runs at our system side.
Thanks !
Using no-XML API, setting adios_set_max_buffer_size with different values leads to different sizes of final file. This behavior appears when using adios_write_byid on a variable declared several times to access several offsets by same rank. When adios_write_byid is called only one time per variable name declared, we have identical final file size whatever adios_set_max_buffer_size we set.
This was tested on x86 machine with Intel Compiler, using Master branch from June 5, 49b720f76ab9097392fdadfeda0f13067dc9e800
max_buffer_size.zip