Compilation Linking Issue wih cpp_stl Runtime

[glass-ships]

KSC is compiling and generating code but is unable to properly locate kaitai cpp_stl runtime files:

Generated header file:

``` $ cat animal_reference.h #include "kaitai/kaitaistruct.h" #include "awkward/builder/ArrayBuilder.h" namespace ak = awkward; ak::ArrayBuilder read_animal(kaitai::kstream* ks); ```

KT struct header file (up to date with master):

``` $ ll ./kaitai/ total 56 drwxrwxr-x 2 josh josh 4096 Aug 14 11:35 ./ drwxrwxr-x 4 josh josh 4096 Aug 14 12:00 ../ -rw-rw-r-- 1 josh josh 236 Aug 12 17:33 custom_decoder.h -rw-rw-r-- 1 josh josh 5279 Aug 12 17:33 exceptions.h -rw-rw-r-- 1 josh josh 18998 Aug 12 17:33 kaitaistream.cpp -rw-rw-r-- 1 josh josh 8353 Aug 12 17:33 kaitaistream.h -rw-rw-r-- 1 josh josh 286 Aug 14 11:35 kaitaistruct.h $ cat kaitai/kaitaistruct.h #ifndef KAITAI_STRUCT_H #define KAITAI_STRUCT_H #include namespace kaitai { class kstruct { public: kstruct(kstream *_io) { m__io = _io; } virtual ~kstruct() {} protected: kstream *m__io; public: kstream *_io() { return m__io; } }; } #endif ```

Compilation command and output:

``` $ g++ -Ikaitai -I/opt/awkward-1.0/include -L/opt/awkward-1.0/build/lib.linux-x86_64-3.7/awkward1 main.cpp -lawkward -lawkward-cpu-kernels -o kaitaiAnimalReader In file included from animal_reference.h:1, from main.cpp:3: kaitai/kaitaistruct.h:4:10: fatal error: kaitai/kaitaistream.h: No such file or directory 4 | #include | ^~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. ```

Tried changing kaitaistruct.h line 4 from #include <kaitai/kaitaistream.h> to #include <kaitaistream.h> (because relative to kaitaistruct.h this file is in the same directory) which then results in:

/usr/bin/ld: /tmp/ccX7rU9S.o: in function `main':
main.cpp:(.text+0x9f): undefined reference to `kaitai::kstream::kstream(std::istream*)'
/usr/bin/ld: main.cpp:(.text+0xe0): undefined reference to `read_animal(kaitai::kstream*)'
collect2: error: ld returned 1 exit status

Any suggestions of how to compile properly against kaitai generated code + cpp_stl runtime library?

[webbnh]

You shouldn't need to modify the path in the #include directive if you specify the correct path in the command line -I switch.

As for your second set of diagnostics, by this point your compilation has succeeded but the linker (ld) is unable to locate the c'tor's for kstream and read_animal in any of the object files and libraries that you've given it. I see two -l switches on your command line, but no .o files and no other .cpp files...where are those c'tor's defined?

You need to build the KS runtime and place a reference to it on your main command line. And, presumably, you need to build your generated code and add a reference to it, as well. (I presume that you're not planning on putting these objects into your awkward libraries....)

[glass-ships]

In that case, if I don't modify the #include line in kaitai/kaitaistruct.h then the error I receive is simply a "file not found":

In file included from animal_reference.h:1,
                 from main.cpp:3:
kaitai/kaitaistruct.h:4:10: fatal error: kaitai/kaitaistream.h: No such file or directory
    4 | #include <kaitai/kaitaistream.h>
      |          ^~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.

However these files exist relative to the build directory. The chain is essentially main.cpp references animal_reference.h which includes <kaitai/kaitaistruct.h> which is where the faulty call to <kaitai/kaitaistream.h> is. I'm a bit unclear on whether I do indeed need to build the runtime, and where I would place the .so file created, as I remember having a working version of this in which I never ran cmake . in the runtime library. Perhaps you have some thoughts on this? Apologies for not including the code context earlier

[KOLANICH]

I advise you to just use CMake and not to mess with compiler command line directly.

[glass-ships]

Thanks @KOLANICH

Could you clarify what you mean?

I'm not familiar with CMake, but I have a makefile for the project which results in the same error as above.

[KOLANICH]

1. Have you tried to use absolute paths in all your compiler command lines?

2. I'm not familiar with CMake

CMake is a de-facto industry standard, used by many libs, including KS runtime for C++, you cannot avoid getting familiar it, if you plan to continue software development in C or C++ especially. You really have no choice.

I also recommend you to install ninja and use it instead of GNU Make in conjunction with CMake.

[glass-ships]

I have indeed tried absolute paths to the same effect.

Again, the compiler can find main.cpp and animal_reference.h and kaitai/kaitaistruct.h. The faulty call is from kaitaistruct.h to kaitaistream.h.

If you think creating a CMakelists.txt would be useful for this project, I'm definitely open to trying it.

However, perhaps since I've never worked with CMake you might append to your suggestion some particular lines/points of reference for me to begin?

[KOLANICH]

I canot currently give you an example of CMake usage with KS C++ runtime (I have left it on another PC and haven't uploaded to GH because didn't feel like it worth it), but here is a link directly messing with compiler command line you can find helpful: https://github.com/KOLANICH/research_compiler_optimizations_comparison/blob/master/kaitai/generate.sh

[webbnh]

@glass-ships, CMake is very handy and not hard to learn, especially for simple cases. It is a "makefile generator" (although, as KOLANICH points out, you can plug in other generators, which is what makes it so handy -- you can use the same "description of your build" (i.e., CMakeLists.txt) to build your project on Linux, Windows, etc. with different compilers and builders.

However, you don't necessarily need to learn it or use it to build your code in order to solve your problem.

Nevertheless, you do need to build the KS runtime and link your code with it. But, you don't have to build the runtime into an .so -- it's only a single module, so you can just build its .o and link that into your executable. (For arcane reasons, in my project's build, we build a .a from it and link against that.)

So, if you're comfortable working with make, then you just need to add a target which compiles kaitaistream.cpp, and then include that in the dependencies for your main module. If that doesn't make sense to you, you should strongly consider learning CMake, as it's much simpler to use that to express your requirements.

If you're working "by hand", then just compile kaitaistream.cpp into a .o and then provide the .o on the g++ commmand line for your main, and you should be in business.

The chain is essentially main.cpp references animal_reference.h which includes <kaitai/kaitaistruct.h> which is where the faulty call to <kaitai/kaitaistream.h> is.

The fact that the code uses <> around the include file means that the file is expected to be found relative to the "system include path" or in the "standard include directories". If you didn't place the target include files there (which you almost certainly didn't...), then you have to use a -I switch on the compiler command line to give the compiler additional directories to search. (This is one of the things which you can have CMake manage for you....)

You tried -Ikaitai, but that (presumably) won't help find the file kaitai/kaitaistream.h, because it will cause the compiler to look in ./kaitai/kaitai/kaitaistream.h. So, perhaps what you need is -I..

[KOLANICH]

@glass-ships https://github.com/KOLANICH/QtInstallerFrameworkExtractor