Docker build is failing

[sumitjhaMindtickle]

Hi , I am trying to install bistro in below environment . os : ubuntu:16.04 gcc : 5 This is blocking ,please suggest any quick around for installation of any stable version of bistro .

Below is the error i am getting in log file :

Step 124/126 : RUN make -j '1' ---> Running in bfef425bf6ae Scanning dependencies of target bistro_lib [ 0%] Building CXX object CMakeFiles/bistro_lib.dir/Bistro.cpp.o [ 0%] Linking CXX static library libbistro_lib.a [ 0%] Built target bistro_lib Scanning dependencies of target gtest [ 1%] Building CXX object cmake/deps/gtest-1.7.0/CMakeFiles/gtest.dir/src/gtest-all.cc.o [ 1%] Linking CXX static library libgtest.a [ 1%] Built target gtest Scanning dependencies of target gtest_main [ 1%] Building CXX object cmake/deps/gtest-1.7.0/CMakeFiles/gtest_main.dir/src/gtest_main.cc.o [ 2%] Linking CXX static library libgtest_main.a [ 2%] Built target gtest_main Scanning dependencies of target subprocess_with_timeout [ 2%] Building CXX object processes/CMakeFiles/subprocess_with_timeout.dir/SubprocessOutputWithTimeout.cpp.o [ 3%] Linking CXX static library libsubprocess_with_timeout.a [ 3%] Built target subprocess_with_timeout Scanning dependencies of target processes [ 4%] Building CXX object processes/CMakeFiles/processes.dir/AsyncCGroupReaper.cpp.o [ 4%] Building CXX object processes/CMakeFiles/processes.dir/AsyncReadPipeRateLimiter.cpp.o [ 4%] Building CXX object processes/CMakeFiles/processes.dir/CGroupSetup.cpp.o [ 5%] Building CXX object processes/CMakeFiles/processes.dir/TaskSubprocessQueue.cpp.o [ 5%] Linking CXX static library libprocesses.a [ 5%] Built target processes Scanning dependencies of target test_cgroup_setup [ 5%] Building CXX object processes/tests/CMakeFiles/test_cgroup_setup.dir/test_cgroup_setup.cpp.o [ 6%] Linking CXX executable test_cgroup_setup ^[[91mCMakeFiles/test_cgroup_setup.dir/test_cgroup_setup.cpp.o: In function TestCGroupSetup_TestSetup_Test::TestBody()::{lambda()#1}::operator()() const': /home/bistro/bistro/processes/tests/test_cgroup_setup.cpp:72: undefined reference tofacebook::bistro::cgroupSetup(std::cxx11::basic_string<char, std::char_traits, std::allocator > const&, facebook::bistro::cpp2::CGroupOptions const&)' CMakeFiles/test_cgroup_setup.dir/test_cgroup_setup.cpp.o: In function TestCGroupSetup_TestSetup_Test::TestBody()::{lambda()#4}::operator()() const': /home/bistro/bistro/processes/tests/test_cgroup_setup.cpp:93: undefined reference tofacebook::bistro::cgroupSetup(std::cxx11::basic_string<char, std::char_traits, std::allocator > const&, facebook::bistro::cpp2::CGroupOptions const&)' CMakeFiles/test_cgroup_setup.dir/test_cgroup_setup.cpp.o: In function TestCGroupSetup_TestSetup_Test::TestBody()::{lambda()#6}::operator()() const': /home/bistro/bistro/processes/tests/test_cgroup_setup.cpp:118: undefined reference tofacebook::bistro::cgroupSetup(std::cxx11::basic_string<char, std::char_traits, std::allocator > const&, facebook::bistro::cpp2::CGroupOptions const&)' CMakeFiles/test_cgroup_setup.dir/test_cgroup_setup.cpp.o: In function TestCGroupSetup_TestSetup_Test::TestBody()::{lambda()#7}::operator()() const': /home/bistro/bistro/processes/tests/test_cgroup_setup.cpp:139: undefined reference tofacebook:^[[0m^[[91m:bistro::cgroupSetup(std::cxx11::basic_string<char, std::char_traits, std::allocator > const&, facebook::bistro::cpp2::CGroupOptions const&)' CMakeFiles/test_cgroup_setup.dir/test_cgroup_setup.cpp.o: In function TestCGroupSetup_TestSetup_Test::TestBody()': /home/bistro/bistro/processes/tests/test_cgroup_setup.cpp:63: undefined reference tofacebook::bistro::cpp2::CGroupOptions::CGroupOptions()' /home/bistro/bistro/processes/tests/test_cgroup_setup.cpp:67: undefined reference to `facebook::bistro::cgroupSetup(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, facebook::bistro::cpp2::CGroupOptions const&)'

[ribaptista]

Hello @sumitjhaMindtickle

I've got a similar error. Please see #24.

As a workaround I was able to build Bistro from a previous commit (044cd9f as suggested on #18):

git checkout 044cd9f
cd build 
./fbcode_builder/make_docker_context.py --os-image ubuntu:16.04 --gcc-version 5 --local-repo-dir <path of bistro source>

Then into dir returned by command above:

docker build .

Some tests fail but it should not be a problem as stated on #18.

Cheers

[sumitjhaMindtickle]

Thanks @ribaptista , I was looking for a commit which would be stable enough to go in production .I doubt stability of bistro as of now but looking forward to test it in own environment . @ribaptista , are you gonna use it in production or just trying it for some benchmark ?

[ribaptista]

Hey @sumitjhaMindtickle Yeah, I intend to use it in production for a new product I'm working on. I found some of its features to fit my needs, like dynamic node creation (through the script fetcher). Anyways it would be great to hear from Bistro developers what updates we could expect within the next months. Or whether Bistro is even going to evolve as an open source project.

[snarkmaster]

build broken

Sorry about this, Bistro's dependencies move fast, and the continuous integration that tells developers when they broke Bistro had been imperfect. In the last month or so, I did some work to make that more robust, and Travis actually looks green pretty consistently. Even if nobody breaks the build by accident, it cannot be green 100% of the time, because Travis limits builds to 45 minutes. A from-scratch build Bistro + all dependencies on Travis takes longer than that. We use ccache — which usually makes it fast enough to finish. However, if there is a change that triggers a full rebuild, it takes a smaller commit (or two) for the build to turn green again.

does it work in production?

Bistro has been very stable for the use-cases I'm familiar with. It runs workloads continuously on thousands of machines, and has run a few tens of billions of tasks in the last few years.

Despite the mottled appearance of Travis, Bistro's test suite is actually pretty good, so that helps a lot.

any updates expected?

I am personally not doing significant feature work on the C++ core of the project — there's one small scheduler feature that should land soon, but no bigger plans as of now.

The project has been mature and "ready" for many years. I do have a "potential future roadmap" for the project, if a clear need arises, but at the moment Bistro serves the existing deployments well enough.

That said, I'm happy to discuss features you want, and to give pointers on how to add the good ideas to the codebase. So if you're looking to contribute, please open a new issue to discuss your suggestion.

There is also some unreleased code, which could be helpful for open-source Bistro users. It consists of various CLI tooling to make it easier to schedule and manage ad-hoc jobs. It would not be much work to put that code up on Github, but significantly more work to turn it into a high-quality release.

That said, if there is clear demand for that tooling, maybe done is better than perfect, and I should start with a "your mileage may vary" release of those tools, with a few hundred words of markdown to explain how they work. Last year, my impression was that the demand is not there. If I'm wrong this year, please let me know.

So thanks for writing, @sumitjhaMindtickle & @ribaptista. I will do my best to reply more promptly on Github from here on out.