Base builds not running tests

[mdavidsaver]

https://travis-ci.org/github/epics-base/epics-base/jobs/691149598#L6796

grep: /configure/CONFIG_BASE_VERSION: No such file or directory

Looks like it comes from

https://github.com/epics-base/ci-scripts/blob/e66c1f748c76cb4db78a2fc4259885a20420e563/travis/build.sh#L40-L43

I'm not sure why. I see export BASE=SELF printed earlier, so I would expect it to be defined. Unless I'm missing a cd.

https://github.com/epics-base/ci-scripts/blob/e66c1f748c76cb4db78a2fc4259885a20420e563/travis/build.sh#L9-L12

[mdavidsaver]

https://github.com/epics-base/ci-scripts/blob/e66c1f748c76cb4db78a2fc4259885a20420e563/travis/build.sh#L36-L44

Maybe this logic could be simplified to:

 if [ "$TEST" != "NO" ] 
 then 
   if grep -q "^test-results:" $EPICS_BASE/configure/RULES_BUILD
   then 
     make -j2 tapfiles || ret=$? 
     make -sk test-results 
   else
     make runtests || ret=$? 
   fi 
 fi 

This would prefer make test-results, but fallback to make runtests.

[mdavidsaver]

I'm not sure why. I see export BASE=SELF printed earlier, so I would expect it to be defined. Unless I'm missing a cd.

Or if $CURDIR isn't defined. (as it is in prepare.sh)

https://github.com/epics-base/ci-scripts/blob/e66c1f748c76cb4db78a2fc4259885a20420e563/travis/prepare.sh#L23

[ralphlange]

Maybe this logic could be simplified to:

Been there. Done that. Your simplified logic was actually where I was coming from.

There are specific in-between versions of 3.14, which define a test-results target that does not work. The logic has to be: Run test-results only on 3.15 or later versions that define the target.

[mdavidsaver]

The present logic doesn't capture the fact that "make tapfiles" without "make test-results" is not very useful (cf. the most recent build). In this case, the single "make runtests" should be preferred, as it will actually show PASS/FAIL results.

Also, the 3.14 test pattern needs to account for whitespace.

$ git grep BASE_3_14 configure/
...
configure/CONFIG_BASE_VERSION:BASE_3_14 = NO

So maybe:

if [ "$TEST" != "NO" ]
then
  if grep -q "BASE_3_14\s*=\s*NO" $EPICS_BASE/configure/CONFIG_BASE_VERSION && grep -q "^test-results:" $EPICS_BASE/configure/RULES_BUILD
  then
    make -j2 tapfiles || ret=$?
    make -sk test-results
  else
    make runtests
  fi
fi

[mdavidsaver]

All good now. https://travis-ci.org/github/epics-base/epics-base/jobs/691423217#L4928

[anjohnson]

All good now.

Actually no it isn't because it still isn't running the tests on RTEMS (which is what I was looking for). The results you looked at were from the linux host build, but not from one of the RTEMS cross-builds:

make[4]: Entering directory '/home/travis/build/epics-base/epics-base/modules/pva2pva/testApp/O.linux-x86_64'
perl -CSD testweak.t -tap > testweak.tap
perl -CSD testtest.t -tap > testtest.tap
perl -CSD testpvif.t -tap > testpvif.tap
perl -CSD testpdb.t -tap > testpdb.tap
perl -CSD testpvalink.t -tap > testpvalink.tap
perl -CSD testgroupconfig.t -tap > testgroupconfig.tap
perl -CSD testdbf_copy.t -tap > testdbf_copy.tap
make[4]: Leaving directory '/home/travis/build/epics-base/epics-base/modules/pva2pva/testApp/O.linux-x86_64'
make -C O.RTEMS-pc386-qemu -f ../Makefile TOP=../.. \
    T_A=RTEMS-pc386-qemu tapfiles
make[4]: Entering directory '/home/travis/build/epics-base/epics-base/modules/pva2pva/testApp/O.RTEMS-pc386-qemu'
make[4]: Nothing to be done for 'tapfiles'.
make[4]: Leaving directory '/home/travis/build/epics-base/epics-base/modules/pva2pva/testApp/O.RTEMS-pc386-qemu'

I don't think the current build system will let us run the tests on a cross-target without also running them on the host, but I accept that as a limitation (I will take a look at what needs fixing to do that after the release).

[ralphlange]

How can 'make tapfiles' finding 'Nothing to be done' be related to ci-scripts? Out of scope, if you ask me. Also, pva2pva/testApp does not contain RTEMS tests. Closing.

[mdavidsaver]

I stand by my assertion of goodness. If you look, you'll see a result for epicsTypesTest.tap printed twice. So it's more a question of teaching make test-results to print some more context, like a directory name.

https://travis-ci.org/github/epics-base/epics-base/jobs/691423228#L6797

[anjohnson]

Okay, sorry – itchy trigger finger hopefully caused by my not having had lunch at the time, and Ralph's point that pva2pva/testApp/Makefile doesn't build any tests for RTEMS (it does contain IOC tests, but that's a different matter). I just happened to land on one of the few test dir's that don't.