Including headers causes compilation failure

[dnbaker]

When I ran into your manuscript on arxiv, I was quite excited by your results and wanted to try your tables, but I am unable to build any files including either definitions.h or tsxdefinitions.h. The only difference between this and a branch of our project which is building as expected is the include line. tsxdefinitions seems to die much sooner.

It outputs:

In file included from growt/data-structures/tsxdefinitions.h:18:0,
                 from lib/db.h:6,
                 from lib/db.cpp:1:
growt/data-structures/simpleelement.h:24:18: warning: ISO C++ does not support ‘__int128’ for ‘type name’ [-Wpedantic]
 using int128_t = __int128;
                  ^~~~~~~~
In file included from growt/data-structures/tsxdefinitions.h:19:0,
                 from lib/db.h:6,
                 from lib/db.cpp:1:
growt/data-structures/markableelement.h:28:18: warning: ISO C++ does not support ‘__int128’ for ‘type name’ [-Wpedantic]
 using int128_t = __int128;
                  ^~~~~~~~
growt/data-structures/markableelement.h: In member function ‘int128_t& growt::MarkableElement::as128i()’:
growt/data-structures/markableelement.h:152:37: warning: ISO C++ does not support ‘__int128’ for ‘type name’ [-Wpedantic]
 { return *reinterpret_cast<__int128 *>(this); }
                                     ^
growt/data-structures/markableelement.h: In member function ‘const int128_t& growt::MarkableElement::as128i() const’:
growt/data-structures/markableelement.h:155:43: warning: ISO C++ does not support ‘__int128’ for ‘type name’ [-Wpedantic]
 { return *reinterpret_cast<const __int128 *>(this); }
                                           ^
In file included from growt/data-structures/tsxdefinitions.h:26:0,
                 from lib/db.h:6,
                 from lib/db.cpp:1:
growt/data-structures/growtable.h: At global scope:
growt/data-structures/growtable.h:280:2: warning: extra ‘;’ [-Wpedantic]
 };
  ^
growt/data-structures/growtable.h:292:2: warning: extra ‘;’ [-Wpedantic]
 };
  ^
In file included from lib/db.h:6:0,
                 from lib/db.cpp:1:
growt/data-structures/tsxdefinitions.h:30:99: error: ‘E’ was not declared in this scope
 template<class HashFct = std::hash<typename SimpleElement::Key>, class Allocator = std::allocator<E> >
                                                                                                   ^
growt/data-structures/tsxdefinitions.h:30:100: error: template argument 1 is invalid
 template<class HashFct = std::hash<typename SimpleElement::Key>, class Allocator = std::allocator<E> >
                                                                                                    ^
make: *** [lib/db.o] Error 1

From definitions.h, I get the following errors:

lib/db.o: In function `growt::SimpleElement::CAS(growt::SimpleElement&, growt::SimpleElement const&)':
db.cpp:(.text+0xf5): undefined reference to `__sync_bool_compare_and_swap_16'
lib/db.o: In function `growt::SimpleElement::atomicDelete(growt::SimpleElement const&)':
db.cpp:(.text+0x123): undefined reference to `__sync_bool_compare_and_swap_16'
lib/db.o: In function `growt::MarkableElement::atomicDelete(growt::MarkableElement const&)':
db.cpp:(.text+0x173): undefined reference to `__sync_bool_compare_and_swap_16'
collect2: error: ld returned 1 exit status
make: *** [taxmap] Error 1
make: *** Waiting for unfinished jobs....
src/mapmake.o: In function `growt::successful(growt::ReturnCode)':
mapmake.cpp:(.text+0xff0): multiple definition of `growt::successful(growt::ReturnCode)'
lib/db.o:db.cpp:(.text+0x0): first defined here
src/mapmake.o: In function `growt::SimpleElement::SimpleElement()':
mapmake.cpp:(.text+0x1000): multiple definition of `growt::SimpleElement::SimpleElement()'
lib/db.o:db.cpp:(.text+0x10): first defined here
src/mapmake.o: In function `growt::SimpleElement::SimpleElement()':
mapmake.cpp:(.text+0x1000): multiple definition of `growt::SimpleElement::SimpleElement()'
lib/db.o:db.cpp:(.text+0x10): first defined here
src/mapmake.o: In function `growt::SimpleElement::SimpleElement(unsigned long, unsigned long)':
mapmake.cpp:(.text+0x1010): multiple definition of `growt::SimpleElement::SimpleElement(unsigned long, unsigned long)'
lib/db.o:db.cpp:(.text+0x20): first defined here
src/mapmake.o: In function `growt::SimpleElement::SimpleElement(unsigned long, unsigned long)':
mapmake.cpp:(.text+0x1010): multiple definition of `growt::SimpleElement::SimpleElement(unsigned long, unsigned long)'
lib/db.o:db.cpp:(.text+0x20): first defined here
src/mapmake.o: In function `growt::SimpleElement::SimpleElement(growt::SimpleElement const&&)':
mapmake.cpp:(.text+0x1020): multiple definition of `growt::SimpleElement::SimpleElement(growt::SimpleElement const&&)'
lib/db.o:db.cpp:(.text+0x30): first defined here
src/mapmake.o: In function `growt::SimpleElement::SimpleElement(growt::SimpleElement const&&)':
mapmake.cpp:(.text+0x1020): multiple definition of `growt::SimpleElement::SimpleElement(growt::SimpleElement const&&)'
lib/db.o:db.cpp:(.text+0x30): first defined here
src/mapmake.o: In function `growt::SimpleElement::SimpleElement(growt::SimpleElement const&)':
mapmake.cpp:(.text+0x1030): multiple definition of `growt::SimpleElement::SimpleElement(growt::SimpleElement const&)'
lib/db.o:db.cpp:(.text+0x40): first defined here
src/mapmake.o: In function `growt::SimpleElement::SimpleElement(growt::SimpleElement const&)':
mapmake.cpp:(.text+0x1030): multiple definition of `growt::SimpleElement::SimpleElement(growt::SimpleElement const&)'
lib/db.o:db.cpp:(.text+0x40): first defined here
src/mapmake.o: In function `growt::SimpleElement::operator=(growt::SimpleElement const&)':
mapmake.cpp:(.text+0x1040): multiple definition of `growt::SimpleElement::operator=(growt::SimpleElement const&)'
lib/db.o:db.cpp:(.text+0x50): first defined here
src/mapmake.o: In function `growt::SimpleElement::isEmpty() const':
mapmake.cpp:(.text+0x1050): multiple definition of `growt::SimpleElement::isEmpty() const'
lib/db.o:db.cpp:(.text+0x60): first defined here
src/mapmake.o: In function `growt::SimpleElement::isDeleted() const':
mapmake.cpp:(.text+0x1060): multiple definition of `growt::SimpleElement::isDeleted() const'
lib/db.o:db.cpp:(.text+0x70): first defined here
src/mapmake.o: In function `growt::SimpleElement::isMarked() const':
mapmake.cpp:(.text+0x1080): multiple definition of `growt::SimpleElement::isMarked() const'
lib/db.o:db.cpp:(.text+0x90): first defined here
src/mapmake.o: In function `growt::SimpleElement::compareKey(unsigned long const&) const':
mapmake.cpp:(.text+0x1090): multiple definition of `growt::SimpleElement::compareKey(unsigned long const&) const'
lib/db.o:db.cpp:(.text+0xa0): first defined here
src/mapmake.o: In function `growt::SimpleElement::atomicMark(growt::SimpleElement&)':
mapmake.cpp:(.text+0x10a0): multiple definition of `growt::SimpleElement::atomicMark(growt::SimpleElement&)'
lib/db.o:db.cpp:(.text+0xb0): first defined here
src/mapmake.o: In function `growt::SimpleElement::getKey() const':
mapmake.cpp:(.text+0x10b0): multiple definition of `growt::SimpleElement::getKey() const'
lib/db.o:db.cpp:(.text+0xc0): first defined here
src/mapmake.o: In function `growt::SimpleElement::getData() const':
mapmake.cpp:(.text+0x10c0): multiple definition of `growt::SimpleElement::getData() const'
lib/db.o:db.cpp:(.text+0xd0): first defined here
src/mapmake.o: In function `growt::SimpleElement::CAS(growt::SimpleElement&, growt::SimpleElement const&)':
mapmake.cpp:(.text+0x10d0): multiple definition of `growt::SimpleElement::CAS(growt::SimpleElement&, growt::SimpleElement const&)'
lib/db.o:db.cpp:(.text+0xe0): first defined here
src/mapmake.o: In function `growt::SimpleElement::atomicDelete(growt::SimpleElement const&)':
mapmake.cpp:(.text+0x10f0): multiple definition of `growt::SimpleElement::atomicDelete(growt::SimpleElement const&)'
lib/db.o:db.cpp:(.text+0x100): first defined here
src/mapmake.o: In function `growt::SimpleElement::as128i()':
mapmake.cpp:(.text+0x1120): multiple definition of `growt::SimpleElement::as128i()'
lib/db.o:db.cpp:(.text+0x130): first defined here
src/mapmake.o: In function `growt::SimpleElement::as128i() const':
mapmake.cpp:(.text+0x1130): multiple definition of `growt::SimpleElement::as128i() const'
lib/db.o:db.cpp:(.text+0x140): first defined here
src/mapmake.o: In function `growt::MarkableElement::atomicDelete(growt::MarkableElement const&)':
mapmake.cpp:(.text+0x1140): multiple definition of `growt::MarkableElement::atomicDelete(growt::MarkableElement const&)'
lib/db.o:db.cpp:(.text+0x150): first defined here
src/mapmake.o: In function `growt::SimpleElement::CAS(growt::SimpleElement&, growt::SimpleElement const&)':
mapmake.cpp:(.text+0x10e5): undefined reference to `__sync_bool_compare_and_swap_16'
src/mapmake.o: In function `growt::SimpleElement::atomicDelete(growt::SimpleElement const&)':
mapmake.cpp:(.text+0x1113): undefined reference to `__sync_bool_compare_and_swap_16'
src/mapmake.o: In function `growt::MarkableElement::atomicDelete(growt::MarkableElement const&)':
mapmake.cpp:(.text+0x1163): undefined reference to `__sync_bool_compare_and_swap_16'
lib/db.o: In function `growt::SimpleElement::CAS(growt::SimpleElement&, growt::SimpleElement const&)':
db.cpp:(.text+0xf5): undefined reference to `__sync_bool_compare_and_swap_16'
lib/db.o: In function `growt::SimpleElement::atomicDelete(growt::SimpleElement const&)':
db.cpp:(.text+0x123): undefined reference to `__sync_bool_compare_and_swap_16'
lib/db.o:db.cpp:(.text+0x173): more undefined references to `__sync_bool_compare_and_swap_16' follow
collect2: error: ld returned 1 exit status
make: *** [mapmake] Error 1

Now, the tsx might die sooner because I may not have transactional memory on these Xeon R processors. This is with gcc6.2.1 on RedHat.

Edit: Adding -latomic and -lgcc_s and -lgcc seem to have eliminated the undefineed __sync_bool_compare_and_swap_16s, but I still am encountering the multiple definitions. Maybe a static declaration is needed?

Any idea why that might be?

Thanks!

[TooBiased]

Hi, first of all thank you for your interest.

I am looking into the issues you uncovered, the tsxdefinitions.h apparently had an error, causing it to fail during compilation. I have fixed this one issue.

The second problem seems to be a Linker issue. To fix this it would be really helpful to know more about your projects structure. On first glance it looks like the hash table is included within multiple compile units. Apparently there is some problem with then linking them into a single file. Can you give me an intuition, how db.cpp and mapmake.cpp interact in your project architecture?

Greetings Tobias

[dnbaker]

Hi Tobias! Thanks for the help.

db.h was actually almost empty, and db.cpp had nothing but namespace kpg {}:

#ifndef __DB_H__
#define __DB_H__
#include "feature_min.h"
#include "encoder.h"
#include "data-structures/tsxdefinitions.h"
namespace kpg {
}
#endif // #ifdef __DB_H__

mapmake.cpp was the only file that included db.h except for db.cpp, which was causing the problems, as I was linking the two together. I found that by removing db.o, I was able to get it to compile.

It would be nice to be able to be able to have functions working with them in multiple input files, though that seems to be only be an issue for the non-template classes. I was considering using them for the core for the core of a database for a classification project. I think that could be solved by templating the non-template classes, declaring their functions as inline, or by providing implementations in matched .cpp files. I expect many projects would want to share the declarations between multiple files without sharing the implementations.

Either way, thanks for the quick response, and best of luck!

[TooBiased]

You were right apparently, I missed marking the functions in SimpleElement with inline. I hope everything works now.

[dnbaker]

It compiles just fine now. Thank you!