search

robust-rosin / robust

A dataset of 200+ bugs in the Robot Operating System for BugZoo
30 stars 9 forks source link

Failed to build Docker image: robustrosin/robust:ca23e58 #20

Open git-afsantos opened 6 years ago

git-afsantos commented 6 years ago

I believe I have the test case done, so all that remains is to build the image and test it there. Below follows the output.

Step 24/29 : RUN mkdir src  && git clone "${REPO_FORK_URL}" "src/repo-under-test"  && cd src/repo-under-test  && git checkout robust_buggy_released

 ---> Running in 1ad05a302d42
fatal: repository '' does not exist

Traceback (most recent call last):
  File "/home/andre/robust/bin/bugzoo", line 11, in <module>
    sys.exit(main())
  File "/home/andre/robust/lib/python3.5/site-packages/bugzoo/cli/__init__.py", line 532, in main
    args.func(args)
  File "/home/andre/robust/lib/python3.5/site-packages/bugzoo/cli/__init__.py", line 487, in <lambda>
    cmd.set_defaults(func=lambda args: build_bug(rbox, args.bug, args.force))
  File "/home/andre/robust/lib/python3.5/site-packages/bugzoo/cli/__init__.py", line 101, in build_bug
    rbox.bugs.build(bug, force=force)
  File "/home/andre/robust/lib/python3.5/site-packages/bugzoo/mgr/bug.py", line 83, in build
    quiet=quiet)
  File "/home/andre/robust/lib/python3.5/site-packages/bugzoo/mgr/build.py", line 124, in build
    raise ImageBuildFailed(name, log)
bugzoo.exceptions.ImageBuildFailed: failed to build Docker image: robustrosin/robust:ca23e58

Someone is missing something. Is it me? Is it the bugzoo/docker file? @ChrisTimperley

gavanderhoorn commented 6 years ago

This may be something with the Dockerfile or how args are passed to it by bugzoo.

I've just run into the same problem, but for b166c93 (added in #19).

ChrisTimperley commented 6 years ago

The reason for the failure is due to missing build-time arguments in the BugZoo manifest file. In this case, REPO_FORK_URL is missing. (I don't think the fork existed when I wrote the manifest file?) Anyway, I'll fix that up.

ChrisTimperley commented 6 years ago

The build now gets much further:

Step 27/29 : RUN if [ "${IS_BUILD_FAILURE}" = "False" ]; then ./build.sh ; fi

 ---> Running in 6239111c51e4
-- The C compiler identification is GNU 5.4.0
-- The CXX compiler identification is GNU 5.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Using CATKIN_DEVEL_PREFIX: /ros_ws/devel
-- Using CMAKE_PREFIX_PATH: /opt/ros
-- This workspace overlays: /opt/ros
-- Found PythonInterp: /usr/bin/python (found version "2.7.12")
-- Using PYTHON_EXECUTABLE: /usr/bin/python
-- Using Debian Python package layout
-- Using empy: /usr/bin/empy
-- Using CATKIN_ENABLE_TESTING: ON
-- Call enable_testing()
-- Using CATKIN_TEST_RESULTS_DIR: /ros_ws/build/test_results
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Found gtest sources under '/usr/src/gtest': gtests will be built
-- Using Python nosetests: /usr/bin/nosetests-2.7
-- catkin 0.7.0
-- BUILD_SHARED_LIBS is on
-- Using CATKIN_WHITELIST_PACKAGES: rosconsole;roscpp;xmlrpcpp
-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-- ~~  traversing 3 packages in topological order:
-- ~~  - rosconsole
-- ~~  - xmlrpcpp
-- ~~  - roscpp
-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-- +++ processing catkin package: 'rosconsole'
-- ==> add_subdirectory(repo-under-test/tools/rosconsole)
-- Boost version: 1.58.0
-- Found the following Boost libraries:
--   regex
--   system
--   thread
--   chrono
--   date_time
--   atomic
-- rosconsole backend: log4cxx
-- +++ processing catkin package: 'xmlrpcpp'
-- ==> add_subdirectory(repo-under-test/utilities/xmlrpcpp)
-- +++ processing catkin package: 'roscpp'
-- ==> add_subdirectory(repo-under-test/clients/roscpp)
-- Using these message generators: gencpp;geneus;genlisp;genpy
-- Boost version: 1.58.0
-- Found the following Boost libraries:
--   signals
--   filesystem
--   system
-- roscpp: 1 messages, 3 services
-- Looking for include file ifaddrs.h
-- Looking for include file ifaddrs.h - found
-- Looking for trunc
-- Looking for trunc - not found
CMake Error at repo-under-test/clients/roscpp/CMakeLists.txt:151 (find_package):
  By not providing "Findrostest.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "rostest", but
  CMake did not find one.

  Could not find a package configuration file provided by "rostest" with any
  of the following names:

    rostestConfig.cmake
    rostest-config.cmake

  Add the installation prefix of "rostest" to CMAKE_PREFIX_PATH or set
  "rostest_DIR" to a directory containing one of the above files.  If
  "rostest" provides a separate development package or SDK, be sure it has
  been installed.

-- Configuring incomplete, errors occurred!
See also "/ros_ws/build/CMakeFiles/CMakeOutput.log".
See also "/ros_ws/build/CMakeFiles/CMakeError.log".
Invoking "cmake" failed
gavanderhoorn commented 6 years ago

I just did a manual build of the Docker container for this bug and it actually finishes (well, it failed on test.sh: no such file or directory, but that was expected). I didn't use bugzoo just to see whether the Dockerfile was maybe not working correctly.

I used the following commandline:

docker build \
  -t robustrosin/test-ca23e58 \
  --build-arg UBUNTU_VERSION=xenial \
  --build-arg IS_BUILD_FAILURE=no \
  --build-arg USE_APT_OLD_RELEASES=no \
  --build-arg ROS_DISTRO=kinetic \
  --build-arg CATKIN_PKG=roscpp \
  --build-arg REPO_FORK_URL=https://github.com/robust-rosin/ca23e58_ros_comm \
  .

I don't get the missing rostest error that @ChrisTimperley quoted.

@git-afsantos: so it looks like the Docker side of things is alright. Perhaps the bugzoo metadata is not ok yet? I thought that @ChrisTimperley fixed that in 030479b?

git-afsantos commented 6 years ago

I just tried this now. Running bugzoo still fails on the (supposedly) missing rostest, but running your command (replacing ROS_DISTRO=xenial with UBUNTU_VERSION=xenial) does build successfully.

git-afsantos commented 6 years ago

Running the container, however, results in another error.

$ docker run -it 7f64891a128d
/entrypoint.sh: line 4: /ros_ws/devel/setup.bash: No such file or directory

Is there a missing step in the build process, perhaps, where catkin_make is not being called?

gavanderhoorn commented 6 years ago

It could be that the Dockerfile is expecting some tags or branches to exist which may not be true for the REPO_FORK_URL.

We'd really have to ask @ChrisTimperley.

ChrisTimperley commented 6 years ago

The BugZoo metadata looks correct:

blueprints:
  - type: docker
    tag: robustrosin/robust:ca23e58
    file: ../Dockerfile
    context: ca23e58
    arguments:
      IS_BUILD_FAILURE: no
      USE_APT_OLD_RELEASES: no
      UBUNTU_VERSION: xenial
      ROS_DISTRO: kinetic
      CATKIN_PKG: roscpp
      REPO_FORK_URL: https://github.com/robust-rosin/ca23e58_ros_comm
ChrisTimperley commented 6 years ago

I'll try to build it manually to see if I can spot the difference.

ChrisTimperley commented 6 years ago

I can confirm that building manually works on my machine. That means that BugZoo must be passing its build arguments to Docker incorrectly; I'll investigate further.

ChrisTimperley commented 6 years ago

I found the problem!

The following property in the YAML file is evaluated to a boolean:

      IS_BUILD_FAILURE: no

When BugZoo gets round to sending the build arguments to docker, Python silently converts the boolean into a string (either "True" or "False"). For ROBUST, the fix is to quote yes and no in the YAML files. In BugZoo, the fix is to throw an error if a build argument is provided as anything other than a string. I'll fix this up and close the issue :-)

gavanderhoorn commented 6 years ago

N1.

http://yaml.org/type/bool.html

gavanderhoorn commented 6 years ago

In general strings do not need quotes btw. It's just this special case.

Ref: SO#22235064.

ChrisTimperley commented 6 years ago

Bad news. After fixing another typo in the Dockerfile, it would appear that once again this image doesn't build successfully. Attempting to build manually yields the same results:

docker build \
  -t robustrosin/test-ca23e58 \
  -f ../Dockerfile \
  --build-arg UBUNTU_VERSION=xenial \
  --build-arg IS_BUILD_FAILURE=no \
  --build-arg USE_APT_OLD_RELEASES=no \
  --build-arg ROS_DISTRO=kinetic \
  --build-arg CATKIN_PKG=roscpp \
  --build-arg REPO_FORK_URL=https://github.com/robust-rosin/ros_comm \
  --build-arg REPO_BUG_COMMIT=b64b72664f6d64ba0278382de49b1821165a3966 \
  --build-arg REPO_FIX_COMMIT=348cc17ccb3689febe9adaeeb781a3abc9dbb0f6 \
  ca23e58
...

Step 37/39 : RUN echo "[ROBUST] attempting to build PUT..."  && echo "[ROBUST] is a build failure expected? ${IS_BUILD_FAILURE}."  && ./build.sh || [ "${IS_BUILD_FAILURE}" = "yes" ]
 ---> Running in 028538e0a163
[ROBUST] attempting to build PUT...
[ROBUST] is a build failure expected? no.
-- The C compiler identification is GNU 5.4.0
-- The CXX compiler identification is GNU 5.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Using CATKIN_DEVEL_PREFIX: /ros_ws/devel
-- Using CMAKE_PREFIX_PATH: /opt/ros/kinetic
-- This workspace overlays: /opt/ros/kinetic
-- Found PythonInterp: /usr/bin/python (found version "2.7.12") 
-- Using PYTHON_EXECUTABLE: /usr/bin/python
-- Using Debian Python package layout
-- Using empy: /usr/bin/empy
-- Using CATKIN_ENABLE_TESTING: ON
-- Call enable_testing()
-- Using CATKIN_TEST_RESULTS_DIR: /ros_ws/build/test_results
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
-- Found gtest sources under '/usr/src/gtest': gtests will be built
-- Using Python nosetests: /usr/bin/nosetests-2.7
-- catkin 0.7.0
-- BUILD_SHARED_LIBS is on
-- Using CATKIN_WHITELIST_PACKAGES: rosconsole;roscpp;xmlrpcpp
-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-- ~~  traversing 3 packages in topological order:
-- ~~  - rosconsole
-- ~~  - xmlrpcpp
-- ~~  - roscpp
-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-- +++ processing catkin package: 'rosconsole'
-- ==> add_subdirectory(repo-under-test/tools/rosconsole)
-- Boost version: 1.58.0
-- Found the following Boost libraries:
--   regex
--   system
--   thread
--   chrono
--   date_time
--   atomic
-- rosconsole backend: log4cxx
-- +++ processing catkin package: 'xmlrpcpp'
-- ==> add_subdirectory(repo-under-test/utilities/xmlrpcpp)
-- +++ processing catkin package: 'roscpp'
-- ==> add_subdirectory(repo-under-test/clients/roscpp)
-- Using these message generators: gencpp;geneus;genlisp;genpy
-- Boost version: 1.58.0
-- Found the following Boost libraries:
--   signals
--   filesystem
--   system
-- roscpp: 1 messages, 3 services
-- Looking for include file ifaddrs.h
-- Looking for include file ifaddrs.h - found
-- Looking for trunc
-- Looking for trunc - not found
CMake Error at repo-under-test/clients/roscpp/CMakeLists.txt:151 (find_package):
  By not providing "Findrostest.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "rostest", but
  CMake did not find one.

  Could not find a package configuration file provided by "rostest" with any
  of the following names:

    rostestConfig.cmake
    rostest-config.cmake

  Add the installation prefix of "rostest" to CMAKE_PREFIX_PATH or set
  "rostest_DIR" to a directory containing one of the above files.  If
  "rostest" provides a separate development package or SDK, be sure it has
  been installed.

-- Configuring incomplete, errors occurred!
See also "/ros_ws/build/CMakeFiles/CMakeOutput.log".
See also "/ros_ws/build/CMakeFiles/CMakeError.log".