search

robust-rosin / robust

A dataset of 200+ bugs in the Robot Operating System for BugZoo
30 stars 9 forks source link

New bugs exposed while building images #236

Open hsd-dev opened 5 years ago

hsd-dev commented 5 years ago

I had this issue while building care-o-bot/ac6a181 (mentioned in https://github.com/robust-rosin/robust/issues/214) and care-o-bot/c8091b6.

  1. In case of care-o-bot/ac6a181, the docker image building failed: 
Step 42/44 : RUN echo "[ROBUST] attempting to build PUT..."  && echo "[ROBUST] is a build failure expected? ${IS_BUILD_FAILURE}."  && ./build.sh || [ "${IS_BUILD_FAILURE}" = "yes" ]
 ---> Running in acfe3340c7c7
[ROBUST] attempting to build PUT...
[ROBUST] is a build failure expected? yes.
Error(s) in package '/ros_ws/src/repo-under-test/cob_teleop_cob4/package.xml':
The license tag must neither be empty nor only contain whitespaces

Base path: /ros_ws
Source space: /ros_ws/src
Build space: /ros_ws/build
Devel space: /ros_ws/devel
Install space: /ros_ws/install
Creating symlink "/ros_ws/src/CMakeLists.txt" pointing to "/opt/ros/hydro/share/catkin/cmake/toplevel.cmake"
 ---> 6e581e697565
Removing intermediate container acfe3340c7c7
Step 43/44 : COPY test.sh .
 ---> 30569d285e05

This is because in package.xml :

The license tag must neither be empty nor only contain whitespaces

  1. In case of care-o-bot/c8091b6, after building the image, the "problematic" script fails with the error:

no module named gtk

This error was fixed by installing python-gtk within the container and only then I got the expected bug of missing dependency of pynotify.

@wasowski mentioned in today's meeting that these kind of issues were faced earlier where new bugs are exposed while building the images. Is it possible to automate these fixes or should be the responsibility of whoever is building the image?

gavanderhoorn commented 5 years ago

If there are errors in packages that are not the PUT(s), then it could be that the maintainers/devs were using a source checkout around the time the bug was reported in which the "unrelated issues" were already fixed (we've seen something like this in care-o-bot packages before, which caused me to log rosin-project/rosinstall_generator_time_machine#11).

The released version would not have the fixes yet, which would lead to the kind of situation that you encountered (ie: fix seemingly unrelated bug before getting to the one that the report is about).

Is it possible to automate these fixes or should be the responsibility of whoever is building the image?

Do you have any suggestions for how we could automate things like this? It would seem rather difficult to me.

gavanderhoorn commented 5 years ago

re: license tag empty issue: that was fixed in https://github.com/ipa320/cob_command_tools/commit/dd1ab323078e6ad1088d38e6bbadfb2e29c38b67, which was committed on Jul 11, 2014. The package.xml you refer to is from May 13, 2014. That's pretty close.

andersfischernielsen commented 5 years ago

@ipa-hsd How would I go about building dd1ab32? Or have you gotten that to build? It seems the bug is still present in dd1ab32 (where the license section of the package.xml was added), which would enable building a buggy version of the package.

andersfischernielsen commented 5 years ago

After having checked out your fork and modified the .bug to contain the commit where the bug is present and the license field is filled out I get the following error when building the image: 

bugzoo bug build robust:c8091b6                                                               
building bug: robust:c8091b6
Step 1/43 : ARG UBUNTU_VERSION

Step 2/43 : FROM alpine:3.7 as fork

================================================================================
An error occurred during execution:

  failed to build Docker image: robustrosin/robust:c8091b6

See log file for details: /Users/fischer/.bugzoo/logs/2019-03-06_09:32:06.bugzoo.log

The log contains no useful information: 

INFO:bugzoo.mgr.source:2019-03-06 09:41:31,595: refreshed sources
DEBUG:bugzoo.mgr.container:2019-03-06 09:41:31,595: initialising container manager
DEBUG:bugzoo.mgr.container:2019-03-06 09:41:31,595: connecting to low-level Docker API
DEBUG:bugzoo.mgr.container:2019-03-06 09:41:31,599: connected to low-level Docker API
DEBUG:bugzoo.mgr.container:2019-03-06 09:41:31,599: initialised container manager
DEBUG:bugzoo.mgr.build:2019-03-06 09:41:31,607: request to build image: robustrosin/robust:c8091b6
INFO:bugzoo.mgr.build:2019-03-06 09:41:31,607: building image: robustrosin/robust:c8091b6
ERROR:bugzoo.cli.app:2019-03-06 09:41:33,637: An error occurred
Traceback (most recent call last):
  File "/Users/fischer/SQUARE/ipa-hsd/robust/bugzoo_venv/lib/python3.7/site-packages/bugzoo/cli/app.py", line 72, in main
    app.run()
  File "/Users/fischer/SQUARE/ipa-hsd/robust/bugzoo_venv/lib/python3.7/site-packages/cement/core/foundation.py", line 916, in run
    return_val = self.controller._dispatch()
  File "/Users/fischer/SQUARE/ipa-hsd/robust/bugzoo_venv/lib/python3.7/site-packages/cement/ext/ext_argparse.py", line 808, in _dispatch
    return func()
  File "/Users/fischer/SQUARE/ipa-hsd/robust/bugzoo_venv/lib/python3.7/site-packages/bugzoo/cli/controllers/bug.py", line 100, in build
    bugs.build(bug)
  File "/Users/fischer/SQUARE/ipa-hsd/robust/bugzoo_venv/lib/python3.7/site-packages/bugzoo/mgr/bug.py", line 94, in build
    quiet=quiet)
  File "/Users/fischer/SQUARE/ipa-hsd/robust/bugzoo_venv/lib/python3.7/site-packages/bugzoo/mgr/build.py", line 130, in build
    raise ImageBuildFailed(name, log)
bugzoo.exceptions.ImageBuildFailed: failed to build Docker image: robustrosin/robust:c8091b6

I'm unsure if this is a bugzoo or a configuration issue. There is no Dockerfile in the folder for c8091b6. Could that be what's preventing the build?

gavanderhoorn commented 5 years ago

/Users/fischer/SQUARE -> are you running this on OSX?

andersfischernielsen commented 5 years ago

@gavanderhoorn Yes.

gavanderhoorn commented 5 years ago

It's all Python, so theoretically it could work, but you're "on your own", as none of us use OSX, so we haven't tested it.

gavanderhoorn commented 5 years ago

Oh and the Dockerfile not being present in the bug dir is not an issue. The tool reuses a single - parameterised - Dockerfile for all bugs.

Perhaps @ChrisTimperley can shed some light on the errors in the log.

andersfischernielsen commented 5 years ago

@gavanderhoorn Okay, sweet. Thanks! 👍 I'll spin up an Ubuntu VM and see how it looks there.

Regarding the logs, I tried to add extra login in the bugzoo Python build scripts, but the response from the docker daemon is pretty useless (at least the output I got).

ChrisTimperley commented 5 years ago

After having checked out your fork and modified the .bug to contain the commit where the bug is present and the license field is filled out I get the following error when building the image:

bugzoo bug build robust:c8091b6                                                               
building bug: robust:c8091b6
Step 1/43 : ARG UBUNTU_VERSION

Step 2/43 : FROM alpine:3.7 as fork

================================================================================
An error occurred during execution:

  failed to build Docker image: robustrosin/robust:c8091b6

See log file for details: /Users/fischer/.bugzoo/logs/2019-03-06_09:32:06.bugzoo.log

The log contains no useful information:

INFO:bugzoo.mgr.source:2019-03-06 09:41:31,595: refreshed sources
DEBUG:bugzoo.mgr.container:2019-03-06 09:41:31,595: initialising container manager
DEBUG:bugzoo.mgr.container:2019-03-06 09:41:31,595: connecting to low-level Docker API
DEBUG:bugzoo.mgr.container:2019-03-06 09:41:31,599: connected to low-level Docker API
DEBUG:bugzoo.mgr.container:2019-03-06 09:41:31,599: initialised container manager
DEBUG:bugzoo.mgr.build:2019-03-06 09:41:31,607: request to build image: robustrosin/robust:c8091b6
INFO:bugzoo.mgr.build:2019-03-06 09:41:31,607: building image: robustrosin/robust:c8091b6
ERROR:bugzoo.cli.app:2019-03-06 09:41:33,637: An error occurred
Traceback (most recent call last):
  File "/Users/fischer/SQUARE/ipa-hsd/robust/bugzoo_venv/lib/python3.7/site-packages/bugzoo/cli/app.py", line 72, in main
    app.run()
  File "/Users/fischer/SQUARE/ipa-hsd/robust/bugzoo_venv/lib/python3.7/site-packages/cement/core/foundation.py", line 916, in run
    return_val = self.controller._dispatch()
  File "/Users/fischer/SQUARE/ipa-hsd/robust/bugzoo_venv/lib/python3.7/site-packages/cement/ext/ext_argparse.py", line 808, in _dispatch
    return func()
  File "/Users/fischer/SQUARE/ipa-hsd/robust/bugzoo_venv/lib/python3.7/site-packages/bugzoo/cli/controllers/bug.py", line 100, in build
    bugs.build(bug)
  File "/Users/fischer/SQUARE/ipa-hsd/robust/bugzoo_venv/lib/python3.7/site-packages/bugzoo/mgr/bug.py", line 94, in build
    quiet=quiet)
  File "/Users/fischer/SQUARE/ipa-hsd/robust/bugzoo_venv/lib/python3.7/site-packages/bugzoo/mgr/build.py", line 130, in build
    raise ImageBuildFailed(name, log)
bugzoo.exceptions.ImageBuildFailed: failed to build Docker image: robustrosin/robust:c8091b6

I'm unsure if this is a bugzoo or a configuration issue. There is no Dockerfile in the folder for c8091b6. Could that be what's preventing the build?

It looks like you might be running a slightly older version of Docker that doesn't support multi-stage builds (https://docs.docker.com/develop/develop-images/multistage-build/).

andersfischernielsen commented 5 years ago

My docker engine on macOS is version 18.09.2, build 6247962. Multi-stage builds should be supported on that. It does seem to bee a macOS issue, though. I can build the bug just fine on my Ubuntu VM. Oh, well.