Trying to compile

[amunhoz]

I'm kind of new in c++ and trying to create a nodejs addon. But I'm failing to compile your project. Here are my steps:

$ sudo apt-get install libboost-all-dev libmsgpack-dev
$ git clone --recursive https://github.com/Squadrick/shadesmar.git
$ cd ./vendors/shadesmar/
$ ./install_deps.sh
$ ./configure
$ ninja

But I'm getting:

FAILED: CMakeFiles/dragons_test.dir/test/dragons_test.cpp.o 
/usr/bin/c++  -DDEBUG_BUILD -Iinclude -O3 -DNDEBUG   -march=native -O2 -std=gnu++1z -MD -MT CMakeFiles/dragons_test.dir/test/dragons_test.cpp.o -MF CMakeFiles/dragons_test.dir/test/dragons_test.cpp.o.d -o CMakeFiles/dragons_test.dir/test/dragons_test.cpp.o -c test/dragons_test.cpp
In file included from /usr/lib/gcc/x86_64-linux-gnu/7/include/immintrin.h:43:0,
                 from include/shadesmar/memory/dragons.h:31,
                 from test/dragons_test.cpp:29:
/usr/lib/gcc/x86_64-linux-gnu/7/include/avx2intrin.h: In function ‘void shm::memory::dragons::_avx_async_cpy(void*, const void*, size_t)’:
/usr/lib/gcc/x86_64-linux-gnu/7/include/avx2intrin.h:920:1: error: inlining failed in call to always_inline ‘__m256i _mm256_stream_load_si256(const __m256i*)’: target specific option mismatch
 _mm256_stream_load_si256 (__m256i const *__X)
 ^~~~~~~~~~~~~~~~~~~~~~~~
In file included from test/dragons_test.cpp:29:0:
include/shadesmar/memory/dragons.h:111:55: note: called from here
     const __m256i temp = _mm256_stream_load_si256(sVec);
                                                       ^
[6/8] Building CXX object CMakeFiles/dragons_bench.dir/benchmark/dragons.cpp.o
FAILED: CMakeFiles/dragons_bench.dir/benchmark/dragons.cpp.o 
/usr/bin/c++  -DDEBUG_BUILD -Iinclude -isystem /usr/local/include -O3 -DNDEBUG   -march=native -O2 -std=gnu++1z -MD -MT CMakeFiles/dragons_bench.dir/benchmark/dragons.cpp.o -MF CMakeFiles/dragons_bench.dir/benchmark/dragons.cpp.o.d -o CMakeFiles/dragons_bench.dir/benchmark/dragons.cpp.o -c benchmark/dragons.cpp
In file included from /usr/lib/gcc/x86_64-linux-gnu/7/include/immintrin.h:43:0,
                 from include/shadesmar/memory/dragons.h:31,
                 from benchmark/dragons.cpp:23:
/usr/lib/gcc/x86_64-linux-gnu/7/include/avx2intrin.h: In function ‘void shm::memory::dragons::_avx_async_cpy(void*, const void*, size_t)’:
/usr/lib/gcc/x86_64-linux-gnu/7/include/avx2intrin.h:920:1: error: inlining failed in call to always_inline ‘__m256i _mm256_stream_load_si256(const __m256i*)’: target specific option mismatch
 _mm256_stream_load_si256 (__m256i const *__X)
 ^~~~~~~~~~~~~~~~~~~~~~~~
In file included from benchmark/dragons.cpp:23:0:
include/shadesmar/memory/dragons.h:111:55: note: called from here
     const __m256i temp = _mm256_stream_load_si256(sVec);
                                                       ^
ninja: build stopped: subcommand failed.

Any help? Thanks in advance.

[amunhoz]

When using as a header in another project:

$ /usr/bin/g++ -g ./main.cpp ./src/*.cpp  -o ./main-I ./ -I ./vendor/shadesmar/include -pthread

In file included from ./vendor/shadesmar/include/shadesmar/memory/memory.h:45:0,
                 from ./vendor/shadesmar/include/shadesmar/pubsub/topic.h:39,
                 from ./vendor/shadesmar/include/shadesmar/pubsub/publisher.h:38,
                 from ./main.cpp:12:
./vendor/shadesmar/include/shadesmar/memory/tmp.h: In function ‘void shm::memory::tmp::write(const string&)’:
./vendor/shadesmar/include/shadesmar/memory/tmp.h:77:22: error: request for member ‘c_str’ in ‘shm::memory::tmp::tmp_prefix’, which is of non-class type ‘const char [10]’
     mkdir(tmp_prefix.c_str(), opts);
                      ^~~~~
./vendor/shadesmar/include/shadesmar/memory/tmp.h: In function ‘std::vector<std::__cxx11::basic_string<char> > shm::memory::tmp::get_tmp_names()’:
./vendor/shadesmar/include/shadesmar/memory/tmp.h:105:27: error: request for member ‘c_str’ in ‘shm::memory::tmp::tmp_prefix’, which is of non-class type ‘const char [10]’
   dp = opendir(tmp_prefix.c_str());
                           ^~~~~
./vendor/shadesmar/include/shadesmar/memory/tmp.h:109:28: error: invalid operands of types ‘const char [10]’ and ‘char [256]’ to binary ‘operator+’
       file.open(tmp_prefix + entry->d_name, std::ios::in);
                 ~~~~~~~~~~~^~~~~~~~~~~~~~~
./vendor/shadesmar/include/shadesmar/memory/tmp.h: In function ‘void shm::memory::tmp::delete_topics()’:
./vendor/shadesmar/include/shadesmar/memory/tmp.h:135:21: error: invalid operands of types ‘const char [8]’ and ‘const char [10]’ to binary ‘operator+’
   system(("rm -rf " + tmp_prefix).c_str());
           ~~~~~~~~~~^~~~~~~~~~~~
In file included from ./main.cpp:13:0:
./vendor/shadesmar/include/shadesmar/pubsub/subscriber.h: In constructor ‘shm::pubsub::SubscriberBase<queue_size>::SubscriberBase(std::__cxx11::string, shm::memory::Copier*, bool)’:
./vendor/shadesmar/include/shadesmar/pubsub/subscriber.h:99:56: error: no matching function for call to ‘forward<std::__cxx11::string>(std::__cxx11::string&, bool&)’
       std::forward<std::string>(topic_name_, extra_copy)));

[Squadrick]

@amunhoz What's your processor architecture? The issue might be due to unavailability of AVX instruction set on your hardware.

[Squadrick]

@amunhoz Your second error is due to the C++ version. You'll need to use c++17 for Shadesmar. Use:

$ /usr/bin/g++ -std=c++17 ...

[amunhoz]

@amunhoz What's your processor architecture? The issue might be due to unavailability of AVX instruction set on your hardware.

I'm using a vmware machine in a amd cpu fx 8350.

[Squadrick]

I'm using a vmware machine in a amd cpu fx 8350.

I've pushed a few commits that add compile guards for checking for supported hardware (AVX, etc.). Can you pull from master and try building again?

Also, pass the -march=native while compiling, like so:

$ /usr/bin/g++ -std=c++17 -march=native ...

Created #27 to add the flags to CMakeLists.txt

[amunhoz]

It worked with -std=c++17 -march=native -lrt -lstdc++fs

But now, im my test, it is using a LOT of cpu, is that normal? A test with another ipc lib like nng uses 4% of one core, this test is using 100%. My test:

void functest(shm::memory::Ptr *msg){
    char * txt = (char*)msg->ptr;// (char*)((char*)msg->ptr + msg->size);
    //std::cout << txt << std::endl;     
}
int main()
{   
    //publisher
    shm::memory::DefaultCopier cpyp;
    shm::pubsub::PublisherBin<1024 /* buffer size */ > pub("topic_name", &cpyp);

    //subscriber
    shm::memory::DefaultCopier cpys;
    shm::pubsub::SubscriberBin<1024 /* buffer size */ > sub("topic_name", &cpys, functest);

    // OR
    // Using `spin`

    std::thread nativeThread = std::thread( [&sub] {                  
        sub.spin();
    });
    //string txt = "lalala";
    char * msg = "lalala";
    const uint32_t data_size = 6;

    for(int i=0;i<9999999999;++i) {
        if (i % 10000 ==0) std::cout << i << std::endl;     
        pub.publish(msg, data_size);
        std::this_thread::sleep_for(std::chrono::milliseconds(20)); //delay audio a little, just to sync        
    }

    std::this_thread::sleep_for(std::chrono::milliseconds(100000)); //delay audio a little, just to sync
    exit(1);
}

[Squadrick]

@amunhoz Yeah, the high CPU usage is expected since sub.spin() runs in a while(true) loop polling for new messages. Two ways to tackle the issue:

1. If you know the approx rate of communication, you can use spinOnce(). Like so:

while(true) {
    sub.spinOnce();
    std::this_thread::sleep_for(std::chrono::milliseconds(1)); // checks every millisecond
}

I think a nice feature would be to pass an optional rate param to spin() to control the rate of polling.

2. There is currently an open ticket to use conditional variables instead of polling (#20) to minimize high CPU usage.

[Squadrick]

@amunhoz A word of advice: Shadesmar is currently pre-alpha, and I would not recommend using it for anything important. Secondly, shadesmar is better suited for transferring large message sizes (~kB, ~MB).