Check not pass

[tieugene]

tkrzw-0.9.3.logs.tar.gz Fedora 33 x64. Log files:

• autoreconf.err.log - stderr of autoreconf -vif
• configure.diff - diff from original configure and autoreconf-made (not affects on result)
• configure.out.log - stderr of ./configure
• make.out.log - stdout of make make_check.err.log - stderr of make check make_check.out.log - stdout of make check make_check.log - stderr&stdout of make check

[tieugene]

Seems rebuild operation and --rebuild option cause errors. As a --lock_mem_buckets option.

[estraier]

Hi, The error seems to come from the user limit of memory locking. Please check it by this command. $ ulimit -l

You can loosen the limit by this, then please run "make check". $ ulimit -S -l 1024

On Tue, Jan 19, 2021 at 5:05 PM Eugene notifications@github.com wrote:

Seems rebuild operation and --rebuild option cause errors. As a --lock_mem_buckets option.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/estraier/tkrzw/issues/5#issuecomment-762672155, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQGJVRCXGFYZ7BM3ZVVLVJTS2U4MFANCNFSM4WHYLSCQ .

[tieugene]

As for my home host it is 64. But anyway I cannot change this value in Fedora project’s packaging «build fabric». And in other’s users hosts.

19 янв. 2021 г., в 19:28, Mikio Hirabayashi notifications@github.com написал(а):

Hi, The error seems to come from the user limit of memory locking. Please check it by this command. $ ulimit -l

You can loosen the limit by this, then please run "make check". $ ulimit -S -l 1024

On Tue, Jan 19, 2021 at 5:05 PM Eugene notifications@github.com wrote:

Seems rebuild operation and --rebuild option cause errors. As a --lock_mem_buckets option.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/estraier/tkrzw/issues/5#issuecomment-762672155, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQGJVRCXGFYZ7BM3ZVVLVJTS2U4MFANCNFSM4WHYLSCQ .

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/estraier/tkrzw/issues/5#issuecomment-762958765, or unsubscribe https://github.com/notifications/unsubscribe-auth/AALTYSYSBAABXWTGAH75QKTS2WXMBANCNFSM4WHYLSCQ.

[estraier]

Is the hard limit 64? The hard limit cannot be changed but the soft limit can. Isn't it possible to write the test script like this?: $ ulimit -S -l 1024 ; make check

[tieugene]

ulimit -S -l 1024; make check … LD_LIBRARY_PATH=.:/usr/local/lib: ./tkrzw_dbm_perf sequence --dbm hash --file mmap-para --path casket.tkh \ --iter 20000 --threads 5 --size 8 --buckets 100000 --lock_mem_buckets OpenAdvanced failed: SYSTEM_ERROR: mlock: no enough memory

19 янв. 2021 г., в 19:41, Mikio Hirabayashi notifications@github.com написал(а):

Is the hard limit 64? The hard limit cannot be changed but the soft limit can. Isn't it possible to write the test script like this?: $ ulimit -S -l 1024 ; make check

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/estraier/tkrzw/issues/5#issuecomment-762966950, or unsubscribe https://github.com/notifications/unsubscribe-auth/AALTYS7UGIJRZ3OE4KCUXNTS2WYZ3ANCNFSM4WHYLSCQ.

[estraier]

What is printed when you run this? $ ulimit -S -l 1024 ; ulimit -l

[tieugene]

bash-5.0$ LC_ALL=C ulimit -S -l 1024 bash: ulimit: max locked memory: cannot modify limit: Invalid argument

PS. BTW - this will be interesting for you: https://bugzilla.redhat.com/show_bug.cgi?id=1914292 https://bugzilla.redhat.com/show_bug.cgi?id=1914292 Process is continuing, we make everything to bump tkrzw into RHEL, but as you can see there some stones on this way.

19 янв. 2021 г., в 20:34, Mikio Hirabayashi notifications@github.com написал(а):

What is printed when you run this? $ ulimit -S -l 1024 ; ulimit -l

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/estraier/tkrzw/issues/5#issuecomment-763001473, or unsubscribe https://github.com/notifications/unsubscribe-auth/AALTYS6Z7JORAC3LCLOXKMTS2W7DHANCNFSM4WHYLSCQ.

[estraier]

Then, there's no way to test the memory locking feature on your environment. Skipping the whole make check or patching Makefile.in to omit tests related to memory lock seems a practical work around.

[tieugene]

patching Makefile.in to omit tests related to memory lock seems a practical work around.

May be good idea to switch those tests off by default and return back with special configure option? Like --enable-havyload

[estraier]

I committed the CL to suppress the test flags for mlock if there's not enough allowed capacity.

[tieugene]

Now tkrzw is packaging for RHEL8/Fedora 32..35 without any patch, with all available tests and skipping unavailable (like this). Thank you.

[tieugene]

0.9.14 - problems again. Last lines:

Setting done: elapsed_time=0.593525 num_records=100000 qps=168485 mem=22812000
  file_size=15224832 eff_data_size=1600000 efficiency=10.51%
Getting: impl=tree num_iterations=20000 value_size=8 num_threads=5
.................................................. (00001000)
...
.................................................. (00020000)
Getting done: elapsed_time=0.114678 num_records=100000 qps=872008 mem=23272000
  file_size=15224832 eff_data_size=1600000 efficiency=10.51%
Removing: impl=tree num_iterations=20000 value_size=8 num_threads=5
make[1]: Leaving directory '/builddir/build/BUILD/tkrzw-0.9.14'
RPM build errors:
make[1]: *** [Makefile:279: check-treedbm-perf] Aborted (core dumped)

[estraier]

That seems a different problem from the --lock_mem_buckets option. Could you tell me the command which crushed and the environment?

[tieugene]

Could you tell me the command which crushed and the environment?

I don't build manually - just 'autoreconf && configure && make && make check'. It's ok in localhost but problems are at virtual hosts that build packages for Fedora/CentOS. And error appears randomly from build to build without any changes in build process. Two sequential builds from one source in one OS (Fedora 34): https://koji.fedoraproject.org/koji/taskinfo?taskID=67707362 https://koji.fedoraproject.org/koji/taskinfo?taskID=67750933

[estraier]

One reason is that the direct I/O feature is not supported on every platform. Thus, I enable the test cases for direct I/O only if --enable-devel is set. I committed the change.

Another reason seems related to TreeDBM. According to one of your log files, the following command crashed somehow. ./tkrzw_dbm_perf sequence --dbm tree --file mmap-para --path casket.tkt --iter 20000 --threads 5 --size 8 --max_page_size 200 --max_branches 8

It's not reproduced on my environment so I need to investigate more.