Setup new environment before running tests in a devel/prerelease job

[meyerj]

The tests of the rtt_ros_integration devel and prerelease jobs currently fail. I assume the reason is that the tests are run directly after the build without sourcing devel/setup.sh before. We are using env-hooks to set some environment variables that are required during run-time to find the generated Orocos RTT plugin and typekit libraries.

There is a line

Retrieve the ROS build environment by sourcing /tmp/test_repositories/build_repository/devel/setup.bash

in the console output, but it is executed right after the tests and before the depends-on repositories are tested. Is there anything we can do so that Jenkins properly sets up the new workspace environment first before make run_tests is executed?

Additionally, there is a number of recipients in the list of e-mail addresses that are notified about the failed/unstable devel job for each commit. How is this list generated?

[tfoote]

When that gets printed, it is getting the environment from the setup file: https://github.com/ros-infrastructure/jenkins_scripts/blob/master/common.py#L99

It's done twice, once before the main test and once before the depends_on. However in the middle of the run this is all inside of the build. I'll have to look at how catkin handles this. If you have a clean workspace with one of these packages, can you immediately call "make run_tests" without building before hand?

The recipients are the maintainers as defined in the package.xml + the committers detected by jenkins since the last successful build. As well as a global notification address where we log all notifications. The flags are set here: https://github.com/ros/rosdistro/blob/master/hydro/source-build.yaml

[tfoote]

I've confirmed that the below unit tests pass by calling make run_tests from the build directory without externally sourcing the setup file, and the environment hooks get called. So I think there's another issue here.

C++:

#include <gtest/gtest.h>
#include <stdlib.h>

TEST(env, read)
{
  char* env_val;
  env_val = getenv ("TEST_VALUE");
  char* expected = "exists";
  EXPECT_STREQ(env_val, expected);
}

int main(int argc, char **argv){
  testing::InitGoogleTest(&argc, argv);
  return RUN_ALL_TESTS();
}

Python:

#!/usr/bin/env python                                                           

import os
import unittest

class TestEnv(unittest.TestCase):
    def test_env(self):
        self.assertEqual(os.environ.get('TEST_VALUE'), 'exists')

I have the following hook.sh registered

export TEST_VALUE='exists'

tfoote@BigFoote:/tmp/test_env/build$ env | grep TEST
tfoote@BigFoote:/tmp/test_env/build$ rm -rf *
tfoote@BigFoote:/tmp/test_env/build$ cmake ..
-- The C compiler identification is GNU
-- The CXX compiler identification is GNU
-- Check for working C compiler: /usr/lib/ccache/gcc
-- Check for working C compiler: /usr/lib/ccache/gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /usr/lib/ccache/c++
-- Check for working CXX compiler: /usr/lib/ccache/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Using CATKIN_DEVEL_PREFIX: /tmp/test_env/build/devel
-- Using CMAKE_PREFIX_PATH: /opt/ros/hydro
-- This workspace overlays: /opt/ros/hydro
-- Found PythonInterp: /usr/bin/python (found version "2.7.3")
-- Using Debian Python package layout
-- Using CATKIN_ENABLE_TESTING: ON
-- Call enable_testing()
-- Using CATKIN_TEST_RESULTS_DIR: /tmp/test_env/build/test_results
-- Looking for include files CMAKE_HAVE_PTHREAD_H
-- Looking for include files CMAKE_HAVE_PTHREAD_H - found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE 
-- Found gtest sources under '/usr/src/gtest': gtests will be built
-- catkin 0.5.77
-- Configuring done
-- Generating done
-- Build files have been written to: /tmp/test_env/build
tfoote@BigFoote:/tmp/test_env/build$ make run_tests
Scanning dependencies of target gtest
[ 33%] Building CXX object gtest/CMakeFiles/gtest.dir/src/gtest-all.cc.o
Linking CXX shared library libgtest.so
[ 33%] Built target gtest
Scanning dependencies of target gtest_main
[ 66%] Building CXX object gtest/CMakeFiles/gtest_main.dir/src/gtest_main.cc.o
Linking CXX shared library libgtest_main.so
[ 66%] Built target gtest_main
Scanning dependencies of target test_env-test
[100%] Building CXX object CMakeFiles/test_env-test.dir/test/test_test_env.cpp.o
/tmp/test_env/test/test_test_env.cpp: In member function ‘virtual void env_read_Test::TestBody()’:
/tmp/test_env/test/test_test_env.cpp:9:20: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
Linking CXX executable devel/lib/test_env/test_env-test
[100%] Built target test_env-test
Scanning dependencies of target tests
[100%] Built target tests
Scanning dependencies of target clean_test_results
[100%] Built target clean_test_results
Scanning dependencies of target _run_tests_test_env_gtest_test_env-test
-- run_tests.py: execute commands
  /tmp/test_env/build/devel/lib/test_env/test_env-test --gtest_output=xml:/tmp/test_env/build/test_results/test_env/gtest-test_env-test.xml
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from env
[ RUN      ] env.read
[       OK ] env.read (0 ms)
[----------] 1 test from env (0 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test case ran. (0 ms total)
[  PASSED  ] 1 test.
-- run_tests.py: verify result "/tmp/test_env/build/test_results/test_env/gtest-test_env-test.xml"
[100%] Built target _run_tests_test_env_gtest_test_env-test
Scanning dependencies of target _run_tests_test_env_gtest
[100%] Built target _run_tests_test_env_gtest
Scanning dependencies of target _run_tests_test_env_nosetests_test
-- run_tests.py: execute commands
  /usr/bin/cmake -E make_directory /tmp/test_env/build/test_results/test_env
  /usr/bin/nosetests -P --process-timeout=60 --where=/tmp/test_env/test --with-xunit --xunit-file=/tmp/test_env/build/test_results/test_env/nosetests-test.xml
.
----------------------------------------------------------------------
Ran 1 test in 0.002s

OK
-- run_tests.py: verify result "/tmp/test_env/build/test_results/test_env/nosetests-test.xml"
[100%] Built target _run_tests_test_env_nosetests_test
Scanning dependencies of target _run_tests_test_env_nosetests
[100%] Built target _run_tests_test_env_nosetests
Scanning dependencies of target _run_tests_test_env
[100%] Built target _run_tests_test_env
Scanning dependencies of target run_tests
[100%] Built target run_tests

For reproducibility here's my CMakeLists.txt additions to the basic template:

catkin_add_gtest(${PROJECT_NAME}-test test/test_test_env.cpp)
catkin_add_nosetests(test)
catkin_add_env_hooks(hooks SHELLS sh DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR})

[dirk-thomas]

Just to clarify: each test is run with the environment being sourced before.

[meyerj]

This was my fault: I can confirm that the tests run fine locally with make run_tests if I use the head revisions of rtt, ocl and rtt_ros_integration, but the released version of ros-hydro-rtt does not contain the env-hook yet. It has been moved from the rtt_ros package in the rtt_ros_integration repo to rtt itself. We will release a new version of the rtt package soon and I assume that the tests will be successful afterwards.

Many thanks for your help, Tully and Dirk!