judy not found

[mpcmpc]

Hello.

This is not really an issue - in the sense that the program works.

But on different machines at compilation DynamO does not find Judy, and I would like to speed up my simulations with a stepped potential (by the way, is there a sensible speedup?)

This is the message I see DynamO: Judy array library (libJudy) (optional, gives faster stepped interactions) : no

while Judy appears to be properly installed. I have tried both to install it via apt as well manually (always on ubuntu)

Any suggestion?

[toastedcrumpets]

Hi, This definitely counts as an issue. The tests are just small cpp files which are compiled and the test is evaluated depending on if they compile. You can find the full log of the build of these tests under build-dir/config.log can you post the contents of that file here?

You should just have to install libjudy-dev and the test will pass on Ubuntu. As for the speed, I've had improvements of +25% using libJudy so its definitely worth the installation. Marcus

[mpcmpc]

Hi, Thanks. So these are the lines of the log file referring to judy. There are some complains about conversions. As I said, judy appears to be properly installed (apt-get returns "libjudy-dev is already the newest version"). Any help is greatly appreciated :-)

Thanks, Massimo

gcc.compile.c++ /home/massimo/Simulazioni/DynamO/DynamO/build-dir/gcc-4.7/release/coil-integration-yes/magnet-OpenCL-no/threading-multi/tests/judy_test.o In file included from src/magnet/tests/judy_test.cpp:5:0: src/magnet/magnet/containers/judy.hpp: In instantiation of ‘magnet::containers::JudySet<KeyType, endIndex>::const_iterator magnet::containers::JudySet<KeyType, endIndex>::findNth(size_t) [with KeyType = unsigned int; long unsigned int endIndex = 4294967295ul; magnet::containers::JudySet<KeyType, endIndex>::const_iterator = magnet::containers::detail::ConstJudyIterator<magnet::containers::JudySet, unsigned int>; size_t = unsigned int]’: src/magnet/tests/judy_test.cpp:62:33: required from here src/magnet/magnet/containers/judy.hpp:127:2: error: invalid conversion from ‘magnet::containers::JudySet::keytype* {aka unsigned int}’ to ‘Wordt {aka long unsigned int_}’ [-fpermissive] In file included from src/magnet/magnet/containers/judy.hpp:23:0, from src/magnet/tests/judy_test.cpp:5: /usr/local/include/Judy.h:231:17: error: initializing argument 3 of ‘int Judy1ByCount(Pcvoid_t, Word_t, Wordt, PJError_t)’ [-fpermissive] In file included from src/magnet/tests/judy_test.cpp:5:0: src/magnet/magnet/containers/judy.hpp: In instantiation of ‘magnet::containers::JudyMap<KeyType, MappedType, endIndex>::const_iterator magnet::containers::JudyMap<KeyType, MappedType, endIndex>::findNth(size_t) const [with KeyType = unsigned int; MappedType = unsigned int; long unsigned int endIndex = 4294967295ul; magnet::containers::JudyMap<KeyType, MappedType, endIndex>::const_iterator = magnet::containers::detail::ConstJudyIterator<magnet::containers::JudyMap<unsigned int, unsigned int>, std::pair<unsigned int, unsigned int> >; size_t = unsigned int]’: src/magnet/tests/judy_test.cpp:117:39: required from here src/magnet/magnet/containers/judy.hpp:234:76: error: invalid conversion from ‘magnet::containers::JudyMap<unsigned int, unsigned int>::keytype* {aka unsigned int}’ to ‘Wordt {aka long unsigned int_}’ [-fpermissive] In file included from src/magnet/magnet/containers/judy.hpp:23:0, from src/magnet/tests/judytest.cpp:5: /usr/local/include/Judy.h:257:17: error: initializing argument 3 of ‘void* JudyLByCount(Pcvoid_t, Word_t, Wordt, PJError_t)’ [-fpermissive] In file included from src/magnet/tests/judy_test.cpp:5:0: src/magnet/magnet/containers/judy.hpp: In instantiation of ‘magnet::containers::JudySet<KeyType, endIndex>::const_iterator magnet::containers::JudySet<KeyType, endIndex>::lower_bound(magnet::containers::JudySet<KeyType, endIndex>::key_type) const [with KeyType = unsigned int; long unsigned int endIndex = 4294967295ul; magnet::containers::JudySet<KeyType, endIndex>::const_iterator = magnet::containers::detail::ConstJudyIterator<magnet::containers::JudySet, unsigned int>; magnet::containers::JudySet<KeyType, endIndex>::key_type = unsigned int]’: src/magnet/magnet/containers/judy.hpp:133:58: required from ‘magnet::containers::JudySet<KeyType, endIndex>::const_iterator magnet::containers::JudySet<KeyType, endIndex>::begin() const [with KeyType = unsigned int; long unsigned int endIndex = 4294967295ul; magnet::containers::JudySet<KeyType, endIndex>::const_iterator = magnet::containers::detail::ConstJudyIterator<magnet::containers::JudySet, unsigned int>]’ src/magnet/tests/judy_test.cpp:28:3: required from here src/magnet/magnet/containers/judy.hpp:93:2: error: invalid conversion from ‘magnet::containers::JudySet::keytype {aka unsigned int_}’ to ‘Wordt {aka long unsigned int_}’ [-fpermissive] In file included from src/magnet/magnet/containers/judy.hpp:23:0, from src/magnet/tests/judy_test.cpp:5: /usr/local/include/Judy.h:236:17: error: initializing argument 2 of ‘int Judy1First(Pcvoid_t, Wordt, PJError_t)’ [-fpermissive] In file included from src/magnet/tests/judy_test.cpp:5:0: src/magnet/magnet/containers/judy.hpp: In instantiation of ‘magnet::containers::JudySet<KeyType, endIndex>::const_iterator magnet::containers::JudySet<KeyType, endIndex>::next(const const_iterator&) const [with KeyType = unsigned int; long unsigned int endIndex = 4294967295ul; magnet::containers::JudySet<KeyType, endIndex>::const_iterator = magnet::containers::detail::ConstJudyIterator<magnet::containers::JudySet, unsigned int>]’: src/magnet/magnet/containers/judy.hpp:38:72: required from ‘magnet::containers::detail::ConstJudyIterator<Container, value_type>& magnet::containers::detail::ConstJudyIterator<Container, value_type>::operator++() [with Container = magnet::containers::JudySet; value_type = unsigned int; magnet::containers::detail::ConstJudyIterator<Container, value_type> = magnet::containers::detail::ConstJudyIterator<magnet::containers::JudySet, unsigned int>]’ /usr/include/c++/4.7/bits/stl_tree.h:1473:2: required from ‘void std::_Rb_tree<_Key, _Val, _KeyOfValue, _Compare, _Alloc>::_M_insert_unique(_II, _II) [with _InputIterator = magnet::containers::detail::ConstJudyIterator<magnet::containers::JudySet, unsigned int>; _Key = unsigned int; _Val = unsigned int; _KeyOfValue = std::_Identity; _Compare = std::less; _Alloc = std::allocator]’ /usr/include/c++/4.7/bits/stl_set.h:166:4: required from ‘std::set<_Key, _Compare, _Alloc>::set(_InputIterator, _InputIterator) [with _InputIterator = magnet::containers::detail::ConstJudyIterator<magnet::containers::JudySet, unsigned int>; _Key = unsigned int; _Compare = std::less; _Alloc = std::allocator]’ src/magnet/tests/judy_test.cpp:38:1: required from here src/magnet/magnet/containers/judy.hpp:103:2: error: invalid conversion from ‘magnet::containers::JudySet::keytype* {aka unsigned int}’ to ‘Wordt {aka long unsigned int_}’ [-fpermissive] In file included from src/magnet/magnet/containers/judy.hpp:23:0, from src/magnet/tests/judy_test.cpp:5: /usr/local/include/Judy.h:237:17: error: initializing argument 2 of ‘int Judy1Next(Pcvoid_t, Wordt, PJError_t)’ [-fpermissive]

[toastedcrumpets]

Can you confirm if you're on 64bit or 32bit linux? I should say why. LibJudy only lets you store machine words in its structures, but sometimes I need to store a pair of integers. Therefore I think LibJudy support should only enable on 64bit systems. I'm currently adding in some casts which should fix your compile problems though.

[toastedcrumpets]

Ok, I don't think your problem is to do with 64/32bit systems. I think your compiler is simply being more strict on the conversion between certain types. I've added a patch 6c62d17ad5968dce0477018741fb66fe3f7ff5a3 which explicitly states the conversions required which should allow it compile on your system. Please confirm if the patch above allows you to use the Judy arrays so I can close the issue.

[mpcmpc]

Hi Marcus, Thanks. libjudy is now found, and make test returns no errors. However, I am now unable to compile dynamo because of some judy related issues. I receive many errors - all of them alike the one pasted below.

Thanks, Massimo

In file included from src/dynamo/dynamo/interactions/captures.hpp:24:0, from src/dynamo/dynamo/dynamics/multicanonical_contactmap.hpp:20, from src/dynamo/dynamo/dynamics/include.hpp:22, from src/dynamo/dynamo/dynamics/dynamics.cpp:18: src/magnet/magnet/containers/judy.hpp: In instantiation of ‘class magnet::containers::JudyMap<dynamo::detail::PairKey, unsigned int>’: src/dynamo/dynamo/interactions/captures.hpp:100:30: required from here src/magnet/magnet/containers/judy.hpp:156:7: error: static assertion failed: Must use a machine word as KeyType

[toastedcrumpets]

Are you on 32bit linux? If your'e unsure, please type uname -a and past the results here. Marcus

[mpcmpc]

32bit. Can try tomorrow at work on a 64bit linux. Thanks.

[toastedcrumpets]

One of the main reasons why LibJudy is optional (despite the large speed increases) is the requirement of a 64 bit system. I could make it work on a 32bit system, but DynamO would be limited to 65k particles in this case.

I will add an additional test for 64bit support to the LibJudy test to make sure this is clearer in future. Thanks for reporting the bug and taking the time to work through it, I think I can close this, but please reopen it if you find it doesn't work on your 64 bit system.

[mpcmpc]

Ok. Thanks! Massimo

Sent from Samsung Mobile

-------- Original message --------

From: Marcus Bannerman notifications@github.com

Date:16/03/2014 21:30 (GMT+08:00)

To: toastedcrumpets/DynamO DynamO@noreply.github.com

Cc: mpcmpc picaciam@gmail.com

Subject: Re: [DynamO] judy not found (#42)

One of the main reasons why LibJudy is optional (despite the large speed increases) is the requirement of a 64 bit system. I could make it work on a 32bit system, but DynamO would be limited to 65k particles in this case. I will add an additional test for 64bit support to the LibJudy test to make sure this is clearer in future. Thanks for reporting the bug and taking the time to work through it, I think I can close this, but please reopen it if you find it doesn't work on your 64 bit system. — Reply to this email directly or view it on GitHub.

[mpcmpc]

Hi Marcus, sadly I have also problems with the 64 bit version.

Just to check, this is the output of uname -a Linux pcmassimo 3.11.0-18-generic #32~precise1-Ubuntu SMP Thu Feb 20 17:52:10 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux version.

I past below the relevant error messages returned by make test.

Thanks, Massimo

gcc.compile.c++ /home/massimo/Simulations/DynamO/build-dir/gcc-4.6/release/coil-integration-yes/magnet-OpenCL-no/threading-multi/tests/judy_test.o In file included from src/magnet/tests/judy_test.cpp:5:0: src/magnet/magnet/containers/judy.hpp:79:34: sorry, unimplemented: non-static data member initializers src/magnet/magnet/containers/judy.hpp:79:34: error: ‘constexpr’ needed for in-class initialization of static data member ‘_array’ of non-integral type src/magnet/magnet/containers/judy.hpp:80:23: sorry, unimplemented: non-static data member initializers src/magnet/magnet/containers/judy.hpp:80:23: error: ISO C++ forbids in-class initialization of non-const static member ‘_count’ src/magnet/magnet/containers/judy.hpp: In member function ‘magnet::containers::JudySet<KeyType, endIndex>::const_iterator magnet::containers::JudySet<KeyType, endIndex>::find(const key_type&) const’: src/magnet/magnet/containers/judy.hpp:86:16: error: ‘_array’ was not declared in this scope src/magnet/magnet/containers/judy.hpp: In member function ‘magnet::containers::JudySet<KeyType, endIndex>::const_iterator magnet::containers::JudySet<KeyType, endIndex>::lower_bound(magnet::containers::JudySet<KeyType, endIndex>::key_type) const’: src/magnet/magnet/containers/judy.hpp:93:17: error: ‘_array’ was not declared in this scope src/magnet/magnet/containers/judy.hpp: In member function ‘magnet::containers::JudySet<KeyType, endIndex>::const_iterator magnet::containers::JudySet<KeyType, endIndex>::next(const const_iterator&) const’: src/magnet/magnet/containers/judy.hpp:103:16: error: ‘_array’ was not declared in this scope src/magnet/magnet/containers/judy.hpp: In member function ‘magnet::containers::JudySet<KeyType, endIndex>::const_iterator magnet::containers::JudySet<KeyType, endIndex>::prev(const const_iterator&) const’: src/magnet/magnet/containers/judy.hpp:111:16: error: ‘_array’ was not declared in this scope src/magnet/magnet/containers/judy.hpp: In member function ‘magnet::containers::JudySet<KeyType, endIndex>::const_iterator magnet::containers::JudySet<KeyType, endIndex>::findNth(size_t)’: src/magnet/magnet/containers/judy.hpp:127:19: error: ‘_array’ was not declared in this scope src/magnet/magnet/containers/judy.hpp: In member function ‘void magnet::containers::JudySet<KeyType, endIndex>::clear()’: src/magnet/magnet/containers/judy.hpp:138:38: error: ‘_array’ was not declared in this scope src/magnet/magnet/containers/judy.hpp: In member function ‘void magnet::containers::JudySet<KeyType, endIndex>::insert(magnet::containers::JudySet<KeyType, endIndex>::key_type)’: src/magnet/magnet/containers/judy.hpp:141:61: error: ‘_array’ was not declared in this scope src/magnet/magnet/containers/judy.hpp: In member function ‘void magnet::containers::JudySet<KeyType, endIndex>::erase(magnet::containers::JudySet<KeyType, endIndex>::key_type)’: src/magnet/magnet/containers/judy.hpp:142:62: error: ‘_array’ was not declared in this scope src/magnet/magnet/containers/judy.hpp: At global scope: src/magnet/magnet/containers/judy.hpp:163:34: sorry, unimplemented: non-static data member initializers src/magnet/magnet/containers/judy.hpp:163:34: error: ‘constexpr’ needed for in-class initialization of static data member ‘_array’ of non-integral type src/magnet/magnet/containers/judy.hpp:164:23: sorry, unimplemented: non-static data member initializers src/magnet/magnet/containers/judy.hpp:164:23: error: ISO C++ forbids in-class initialization of non-const static member ‘_count’ src/magnet/magnet/containers/judy.hpp: In member function ‘magnet::containers::JudyMap<KeyType, MappedType, endIndex>::const_iterator magnet::containers::JudyMap<KeyType, MappedType, endIndex>::find(magnet::containers::JudyMap<KeyType, MappedType, endIndex>::key_type) const’: src/magnet/magnet/containers/judy.hpp:170:44: error: ‘_array’ was not declared in this scope src/magnet/magnet/containers/judy.hpp: In member function ‘magnet::containers::JudyMap<KeyType, MappedType, endIndex>::const_iterator magnet::containers::JudyMap<KeyType, MappedType, endIndex>::lower_bound(magnet::containers::JudyMap<KeyType, MappedType, endIndex>::key_type) const’: src/magnet/magnet/containers/judy.hpp:178:46: error: ‘_array’ was not declared in this scope src/magnet/magnet/containers/judy.hpp: In member function ‘magnet::containers::JudyMap<KeyType, MappedType, endIndex>::const_iterator magnet::containers::JudyMap<KeyType, MappedType, endIndex>::next(magnet::containers::JudyMap<KeyType, MappedType, endIndex>::key_type) const’: src/magnet/magnet/containers/judy.hpp:191:45: error: ‘_array’ was not declared in this scope src/magnet/magnet/containers/judy.hpp: In member function ‘magnet::containers::JudyMap<KeyType, MappedType, endIndex>::const_iterator magnet::containers::JudyMap<KeyType, MappedType, endIndex>::prev(magnet::containers::JudyMap<KeyType, MappedType, endIndex>::key_type) const’: src/magnet/magnet/containers/judy.hpp:200:45: error: ‘_array’ was not declared in this scope src/magnet/magnet/containers/judy.hpp: In member function ‘void magnet::containers::JudyMap<KeyType, MappedType, endIndex>::erase(magnet::containers::JudyMap<KeyType, MappedType, endIndex>::key_type)’: src/magnet/magnet/containers/judy.hpp:212:22: error: ‘_array’ was not declared in this scope src/magnet/magnet/containers/judy.hpp: In member function ‘void magnet::containers::JudyMap<KeyType, MappedType, endIndex>::clear()’: src/magnet/magnet/containers/judy.hpp:227:38: error: ‘_array’ was not declared in this scope src/magnet/magnet/containers/judy.hpp: In member function ‘magnet::containers::JudyMap<KeyType, MappedType, endIndex>::const_iterator magnet::containers::JudyMap<KeyType, MappedType, endIndex>::findNth(size_t) const’: src/magnet/magnet/containers/judy.hpp:234:51: error: ‘_array’ was not declared in this scope src/magnet/magnet/containers/judy.hpp: In member function ‘magnet::containers::JudyMap<KeyType, MappedType, endIndex>::mapped_type* magnet::containers::JudyMap<KeyType, MappedType, endIndex>::getPtr(magnet::containers::JudyMap<KeyType, MappedType, endIndex>::key_type)’: src/magnet/magnet/containers/judy.hpp:267:47: error: ‘_array’ was not declared in this scope src/magnet/magnet/containers/judy.hpp: In member function ‘magnet::containers::JudySet<KeyType, endIndex>::const_iterator magnet::containers::JudySet<KeyType, endIndex>::findNth(size_t) [with KeyType = long unsigned int, long unsigned int endIndex = -1u, magnet::containers::JudySet<KeyType, endIndex>::const_iterator = magnet::containers::detail::ConstJudyIterator<magnet::containers::JudySet, long unsigned int>, size_t = long unsigned int]’: src/magnet/magnet/containers/judy.hpp:131:7: warning: control reaches end of non-void function [-Wreturn-type] src/magnet/magnet/containers/judy.hpp: In member function ‘magnet::containers::JudySet<KeyType, endIndex>::const_iterator magnet::containers::JudySet<KeyType, endIndex>::next(const const_iterator&) const [with KeyType = long unsigned int, long unsigned int endIndex = -1u, magnet::containers::JudySet<KeyType, endIndex>::const_iterator = magnet::containers::detail::ConstJudyIterator<magnet::containers::JudySet, long unsigned int>]’: src/magnet/magnet/containers/judy.hpp:107:7: warning: control reaches end of non-void function [-Wreturn-type] src/magnet/magnet/containers/judy.hpp: In member function ‘magnet::containers::JudySet<KeyType, endIndex>::const_iterator magnet::containers::JudySet<KeyType, endIndex>::lower_bound(magnet::containers::JudySet<KeyType, endIndex>::key_type) const [with KeyType = long unsigned int, long unsigned int endIndex = -1u, magnet::containers::JudySet<KeyType, endIndex>::const_iterator = magnet::containers::detail::ConstJudyIterator<magnet::containers::JudySet, long unsigned int>, magnet::containers::JudySet<KeyType, endIndex>::key_type = long unsigned int]’: src/magnet/magnet/containers/judy.hpp:97:7: warning: control reaches end of non-void function [-Wreturn-type]

"g++"  -ftemplate-depth-128 -O3 -finline-functions -Wno-inline -Wall -pthread -fPIC -std=c++0x -DNDEBUG  -I"src/magnet" -c -o "/home/massimo/Simulations/DynamO/build-dir/gcc-4.6/release/coil-integration-yes/magnet-OpenCL-no/threading-multi/tests/judy_test.o" "src/magnet/tests/judy_test.cpp"

...failed gcc.compile.c++ /home/massimo/Simulations/DynamO/build-dir/gcc-4.6/release/coil-integration-yes/magnet-OpenCL-no/threading-multi/tests/judy_test.o... ...skipped <p/home/massimo/Simulations/DynamO/build-dir/gcc-4.6/release/coil-integration-yes/magnet-OpenCL-no/threading-multi>judy-test for lack of <p/home/massimo/Simulations/DynamO/build-dir/gcc-4.6/release/coil-integration-yes/magnet-OpenCL-no/threading-multi>tests/judy_test.o... ...skipped <p/home/massimo/Simulations/DynamO/build-dir/gcc-4.6/release/coil-integration-yes/magnet-OpenCL-no/threading-multi>judy-test.passed for lack of <p/home/massimo/Simulations/DynamO/build-dir/gcc-4.6/release/coil-integration-yes/magnet-OpenCL-no/threading-multi>judy-test... ...failed updating 1 target...

[toastedcrumpets]

OK, it seems the version of GCC on Precise Pangolin doesn't support some features I've started using in the code (data member initializers). Normally I catch this through the nightly builds, but these don't have libJudy enabled on these (due to it being a 64-bit feature only)

I've just pushed a patch which should fix these errors, please confirm if they do, there may be a line I've missed somewhere.

[mpcmpc]

It works. Many thanks!!

[toastedcrumpets]

No problem, just please remember to cite DynamO in any published work where you use it Marcus