Build for Raspberry Pi failed

[takaomoriyama]

Build of Intu on my RasPi stopped as follows:

$ sudo apt-get update
$ sudo apt-get install python-pip cmake
$ sudo pip install qibuild
$ sudo apt-get install libssl-dev libopencv-dev libboost-python-dev libboost-filesystem-dev libboost-thread-dev
$ git clone —recursive git@github.com:watson-intu/self.git
$ cd self
$ ./scripts/build_raspi.sh
...
[ 17%] Building CXX object lib/cpp-sdk/src/CMakeFiles/utils.dir/utils/Library.cpp.o
/home/pi/dev/watson-intu/self/lib/cpp-sdk/src/utils/Crypt.cpp:27:23: error: aggregate ?EVP_CIPHER_CTX g_EncodeContext? has incomplete type and cannot be defined
 static EVP_CIPHER_CTX g_EncodeContext;
                       ^~~~~~~~~~~~~~~
/home/pi/dev/watson-intu/self/lib/cpp-sdk/src/utils/Crypt.cpp:28:23: error: aggregate ?EVP_CIPHER_CTX g_DecodeContext? has incomplete type and cannot be defined
 static EVP_CIPHER_CTX g_DecodeContext;
                       ^~~~~~~~~~~~~~~
lib/cpp-sdk/src/CMakeFiles/utils.dir/build.make:158: recipe for target 'lib/cpp-sdk/src/CMakeFiles/utils.dir/utils/Crypt.cpp.o' failed
make[2]: *** [lib/cpp-sdk/src/CMakeFiles/utils.dir/utils/Crypt.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
CMakeFiles/Makefile2:571: recipe for target 'lib/cpp-sdk/src/CMakeFiles/utils.dir/all' failed
make[1]: *** [lib/cpp-sdk/src/CMakeFiles/utils.dir/all] Error 2
Makefile:116: recipe for target 'all' failed
make: *** [all] Error 2
[ERROR]: BuildFailed Error occurred when building project self

I noticed that my RasPi has openssl 1.1 installed, while Mac/Linux platform has openssl 1.0.2, and probably this is the reason of the error above.

$ dpkg -l openssl
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                                  Version                 Architecture            Description
+++-=====================================-=======================-=======================-================================================================================
ii  openssl                               1.1.0f-3                armhf                   Secure Sockets Layer toolkit - cryptographic utility

[takaomoriyama]

Similar issue is reported in https://github.com/openssl/openssl/issues/962#issuecomment-208792020. The essence of the change is

#if (OPENSSL_VERSION_NUMBER >= 0x10100000L)
static EVP_CIPHER_CTX g_EncodeContext;
#else // (OPENSSL_VERSION_NUMBER >= 0x10100000L)
static EVP_CIPHER_CTX *g_EncodeContext;
EVP_EncryptInit_ex( g_EncodeContext, NULL, NULL, NULL, NULL );
#endif // (OPENSSL_VERSION_NUMBER >= 0x10100000L)

etc. in in lib/cpp-sdk/src/utils/Crypt.cpp.

Fix:

commit 6fd2af755431f7767561462dc5b95c42dab041cd
Author: Takao Moriyama <moriyama@jp.ibm.com>
Date:   Wed Oct 18 13:29:26 2017 +0900

    Add compatibility to OpenSSL v1.1

[takaomoriyama]

Another build error.

[ 15%] Building CXX object lib/cpp-sdk/src/CMakeFiles/utils.dir/utils/UniqueID.cpp.o
/home/pi/dev/watson-intu/self/lib/cpp-sdk/src/utils/UniqueID.cpp: In static member function ‘static int UniqueID::Decode64(char)’:
/home/pi/dev/watson-intu/self/lib/cpp-sdk/src/utils/UniqueID.cpp:164:61: error: narrowing conversion of ‘-1’ from ‘int’ to ‘char’ inside { } [-Wnarrowing]
   37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51};
                                                             ^

This code, which assumes 'char' is 'signed char', does not work with Rasp because 'char' is 'unsigned char' on Raspi's compiler. The same applies to lib/cpp-sdk/lib/base64/cdecode.c.

The solution is to instruct Raspi's compiler to use 'signed char' by specifying following code in CMakeLists.txt.

if("$ENV{TARGET}" STREQUAL "raspi")
        set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fsigned-char")
        set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -fsigned-char")
endif()

Fix:

commit f040d0ea6244290ba361b8d2c065f1a993720005
Author: Takao Moriyama <moriyama@jp.ibm.com>
Date:   Fri Oct 20 16:26:07 2017 +0900

    Use 'singed char' by default on Raspi since some crypt codes are assuming so.

[takaomoriyama]

Yet another build error.

Scanning dependencies of target self_instance
[ 70%] Linking CXX executable sdk/bin/self_instance
sdk/lib/libself.so:  undefined reference to `cv::CascadeClassifier::load(std::string const&)'
collect2: error: ld returned 1 exit status
CMakeFiles/self_instance.dir/build.make:137: recipe for target 'sdk/bin/self_instance' failed
make[2]: *** [sdk/bin/self_instance] Error 1
CMakeFiles/Makefile2:76: recipe for target 'CMakeFiles/self_instance.dir/all' failed
make[1]: *** [CMakeFiles/self_instance.dir/all] Error 2
Makefile:116: recipe for target 'all' failed
make: *** [all] Error 2
[ERROR]: BuildFailed Error occurred when building project self

The linker could not find cv::CascadeClassifier::load(std::string const&) in libopencv_objdetect. Actually there exists cv::CascadeClassidier() in libopencv_objdetect.so.2.4.9, but with different function signature.

$ nm -C -D /usr/lib/arm-linux-gnueabihf/libopencv_objdetect.so.2.4.9 | grep CascadeClassifier::load
0001b47c T cv::CascadeClassifier::load(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)

The difference comes from introduction of 2011 C++ standard ABI. Intu is configured to build with older ABI by setting "-D_GLIBCXX_USE_CXX11_ABI=0" in CMakeLists.txt for all platform in the following commit for Linux.

commit adfd7924bc0c93517a581c36bf10d585c52d16b3
Author: Richard Lyle <rlyle@cognitivescale.com>
Date:   Fri Aug 11 11:09:22 2017 -0500

    Fixing build for ubuntu 16.04

Unfortunately, opencv library for Raspi supports only new ABI, while one for Linux supports only old ABI, and ones for macOS and Nao/Pepper support both. So proposed solution is to specify "-D_GLIBCXX_USE_CXX11_ABI=0" only for Linux and Sota platforms.

if("$ENV{TARGET}" STREQUAL "linux" OR "$ENV{TARGET}" STREQUAL "sota")
        add_definitions(-D_GLIBCXX_USE_CXX11_ABI=0)
endif()

Fix:

commit 167b6b754595c0a46635e65b4e6d3a155daae462
Author: Takao Moriyama <moriyama@jp.ibm.com>
Date:   Fri Oct 20 16:31:54 2017 +0900

    Use 2011 C++ standard ABI except for Linux and Sota to aovid link error on Raspi

[takaomoriyama]

Test environment.

$ sudo apt-get update
$ sudo apt-get upgrade
$ sudo apt-get install libssl-dev libopencv-dev libboost-python-dev libboost-filesystem-dev libboost-thread-dev
$ sudo apt-get install python-pip cmake
$ sudo pip install qibuild
$ uname -a
Linux raspberrypi 4.9.56-v7+ #1044 SMP Fri Oct 13 15:23:13 BST 2017 armv7l GNU/Linux
$ dpkg -l > dpkg-l.txt

dpkg-l.txt