Using wdt as a library

[NasrinJaleel93]

I know this functionality exists. Pointers to how to actually enable this feature in another project. Say transfer a file?

[ldemailly]

Sure, something like (from https://github.com/facebook/wdt/blob/master/Wdt.h ) :

Wdt &wdt = Wdt:: initializeWdt("test");
// Receiving side:
WdtTransferRequest req;
req.directory = "/tmp/receivedirectory";
wdt.wdtReceiveStart("testN", req);
// Make a string out of req and give it to the sender:
String uri = req.genWdtUrlWithSecret();

// Sender side:
WdtTransferRequest reqS(uri);
// add files to send or directory to the request
reqS.directory= "/tmp/dirtosend";
wdt.wdtSend(reqS);

// finish receiver side
wdt.wdtReceiveFinish("testN");

[NasrinJaleel93]

I added this snippet. And included the wdt.h files as #include <wdt/Wdt.h> after installing wdt (made a subdirectory in my project directory and ran all the instructions in build.md)

I get the following error when I build my project.

[ 0%] Building CXX object src/CMakeFiles/streamer.dir/folly/folly/Benchmark.cpp.o /home/nasrinj/streamer_v2/streamer/src/folly/folly/Benchmark.cpp:19:29: fatal error: folly/Benchmark.h: No such file or directory

include <folly/Benchmark.h>

                         ^

compilation terminated. make[2]: [src/CMakeFiles/streamer.dir/folly/folly/Benchmark.cpp.o] Error 1 make[1]: [src/CMakeFiles/streamer.dir/all] Error 2 make: *** [all] Error 2

Not sure why it happens. Do I need to install folly again explicitly? I tried doing that too, but it throws some error regarding gflags I believe, gflags is already installed according to me. (Is there a way to check?)

[NasrinJaleel93]

I tried installing folly using ./configure CPPFLAGS="-I/usr/local/include" LDFLAGS="-L/usr/local/lib" instead of just ./configure. I was able to install folly I believe but with some errors

When I try to build my project it progresses a little but fails at following point :

[ 0%] Building CXX object src/CMakeFiles/streamer.dir/folly/folly/Benchmark.cpp.o [ 0%] Building CXX object src/CMakeFiles/streamer.dir/folly/folly/Format.cpp.o [ 1%] Building CXX object src/CMakeFiles/streamer.dir/folly/folly/Uri.cpp.o [ 1%] Building CXX object src/CMakeFiles/streamer.dir/folly/folly/json.cpp.o [ 1%] Building CXX object src/CMakeFiles/streamer.dir/folly/folly/init/Init.cpp.o [ 1%] Building CXX object src/CMakeFiles/streamer.dir/folly/folly/tracing/test/StaticTracepointTest.cpp.o In file included from /home/nasrinj/streamer_v2/streamer/src/folly/folly/tracing/test/StaticTracepointTest.cpp:34:0: /usr/local/include/folly/tracing/StaticTracepoint.h:20:51: fatal error: folly/tracing/StaticTracepoint-ELFx86.h: No such file or directory

include <folly/tracing/StaticTracepoint-ELFx86.h>

                                               ^

compilation terminated. make[2]: [src/CMakeFiles/streamer.dir/folly/folly/tracing/test/StaticTracepointTest.cpp.o] Error 1 make[1]: [src/CMakeFiles/streamer.dir/all] Error 2 make: *** [all] Error 2

[ldemailly]

you can remove -DBUILD_TESTING if you don't want to build that benchmark which you don't need for the file transfer

[NasrinJaleel93]

But folly has to installed on the machine where I want to use the wdt library?

[ldemailly]

no - we build the subset of folly we use from source

[ldemailly]

did you solve your problem ?

[NasrinJaleel93]

Unfortunately no, these are some of the errors I am getting :

../src/libstreamer.so: undefined reference to facebook::wdt::EncryptionParams::~EncryptionParams()' ../src/libstreamer.so: undefined reference tofacebook::wdt::WdtTransferRequest::WdtTransferRequest(std::string const&)' ../src/libstreamer.so: undefined reference to `facebook::wdt::Wdt::initializeWdt(std::string const&)'

I wrote a test program that runs as a receiver, it gives this error on compilation :

nasrinj@node5:~/test_code$ g++ -I/home/nasrinj/ -L/usr/local/lib -lwdt hello.cpp /tmp/cchuHzww.o: In function main': hello.cpp:(.text+0x32): undefined reference tofacebook::wdt::Wdt::initializeWdt(std::string const&)' collect2: error: ld returned 1 exit status

[ldemailly]

did making libwdt succeed first ? please attach the full output of cmake + make + make install

[NasrinJaleel93]

I believe everything was installed correctly.

nasrinj@node7:~/wdt$ cmake . -- Boost version: 1.54.0 -- Found the following Boost libraries: -- system -- Configuring done -- Generating done -- Build files have been written to: /home/nasrinj/wdt nasrinj@node7:~/wdt$ make -j [ 11%] Built target folly4wdt [ 86%] Built target wdt_min [ 91%] Built target wdt [100%] Built target wdtbin nasrinj@node7:~/wdt$ sudo make install [sudo] password for nasrinj: [ 11%] Built target folly4wdt [ 86%] Built target wdt_min [ 91%] Built target wdt [100%] Built target wdtbin Install the project... -- Install configuration: "Release" -- Up-to-date: /usr/local/bin/wdt -- Up-to-date: /usr/local/lib/libwdt.so -- Up-to-date: /usr/local/lib/libwdt_min.so -- Up-to-date: /usr/local/lib/libfolly4wdt.so -- Up-to-date: /usr/local/include/wdt/AbortChecker.h -- Up-to-date: /usr/local/include/wdt/WdtOptions.h -- Up-to-date: /usr/local/include/wdt/ByteSource.h -- Up-to-date: /usr/local/include/wdt/WdtBase.h -- Up-to-date: /usr/local/include/wdt/ErrorCodes.h -- Up-to-date: /usr/local/include/wdt/Protocol.h -- Up-to-date: /usr/local/include/wdt/WdtThread.h -- Up-to-date: /usr/local/include/wdt/WdtConfig.h -- Up-to-date: /usr/local/include/wdt/SourceQueue.h -- Up-to-date: /usr/local/include/wdt/SenderThread.h -- Up-to-date: /usr/local/include/wdt/Writer.h -- Up-to-date: /usr/local/include/wdt/Sender.h -- Up-to-date: /usr/local/include/wdt/WdtResourceController.h -- Up-to-date: /usr/local/include/wdt/Reporting.h -- Up-to-date: /usr/local/include/wdt/Wdt.h -- Up-to-date: /usr/local/include/wdt/WdtTransferRequest.h -- Up-to-date: /usr/local/include/wdt/ReceiverThread.h -- Up-to-date: /usr/local/include/wdt/Receiver.h -- Up-to-date: /usr/local/include/wdt/Throttler.h -- Up-to-date: /usr/local/include/wdt/util -- Up-to-date: /usr/local/include/wdt/util/WdtFlags.h -- Up-to-date: /usr/local/include/wdt/util/EncryptionUtils.h -- Up-to-date: /usr/local/include/wdt/util/FileByteSource.h -- Up-to-date: /usr/local/include/wdt/util/TransferLogManager.h -- Up-to-date: /usr/local/include/wdt/util/SerializationUtil.h -- Up-to-date: /usr/local/include/wdt/util/WdtFlagsMacros.h -- Up-to-date: /usr/local/include/wdt/util/FileCreator.h -- Up-to-date: /usr/local/include/wdt/util/ServerSocket.h -- Up-to-date: /usr/local/include/wdt/util/ThreadTransferHistory.h -- Up-to-date: /usr/local/include/wdt/util/CommonImpl.h -- Up-to-date: /usr/local/include/wdt/util/ClientSocket.h -- Up-to-date: /usr/local/include/wdt/util/WdtSocket.h -- Up-to-date: /usr/local/include/wdt/util/Stats.h -- Up-to-date: /usr/local/include/wdt/util/ThreadsController.h -- Up-to-date: /usr/local/include/wdt/util/DirectorySourceQueue.h -- Up-to-date: /usr/local/include/wdt/util/FileWriter.h -- Up-to-date: /usr/local/bin/wcp

This is how I am explicitly linking the library : g++ -L/usr/local/lib/ hello.cpp -lwdt

I get the following error : /usr/bin/ld: /tmp/ccTjK0JB.o: undefined reference to symbol '_ZN8facebook3wdt16EncryptionParamsD1Ev' //usr/local/lib/libwdt_min.so: error adding symbols: DSO missing from command line collect2: error: ld returned 1 exit status