Generate test/Makefile from test/Makefile.am with autoconf.

[fvalasiad]

Closes #185.

[zvr]

Eh, no. The desired outcome is not to simply copy the hand-crafted rules. It should be generated by automake, i.e. have some tests_single_SOURCES variables, etc.

[fvalasiad]

Well I will try.

[fvalasiad]

@zvr Well I struggle choosing a design for this.

But I have to ask, won't this entire system be replaced by #186?

Well you might argue the test folder is there to test that build-recorder produces correct results, while #186 will be to test the performance itself.

the current design is something like this:

bin_PROGRAMS = ex0 ex1 ex2 ex3 ex4
build_recorder_FILES = \
    compile_single_file.out \
    compile_link_file.out \
    compile_file_include.out \
    compile_file_sysinclude.out \
    compile_two_files.out

ex0_SOURCES = f1.c
ex1_SOURCES = f1.c
ex2_SOURCES = f2.c
ex3_SOURCES = f3.c
ex4_SOURCES = f4.c f5.c

ex0_CFLAGS = -c
ex2_CFLAGS = -c
ex3_CFLAGS = -c

BUILD_RECORDER = ../build-recorder

compile_single_file.out: ex0
    $(AM_V_GEN) $(BUILD_RECORDER) -o $@ $(MAKE) $^

compile_link_file.out: ex1
    $(AM_V_GEN) $(BUILD_RECORDER) -o $@ $(MAKE) $^

compile_file_include.out: ex2
    $(AM_V_GEN) $(BUILD_RECORDER) -o $@ $(MAKE) $^

compile_file_sysinclude.out: ex3
    $(AM_V_GEN) $(BUILD_RECORDER) -o $@ $(MAKE) $^

compile_two_files.out: ex4
    $(AM_V_GEN) $(BUILD_RECORDER) -o $@ $(MAKE) $^

.PHONY: all clean clobber

all: $(build_recorder_FILES)

clean:
    $(RM) *.o $(bin_PROGRAMS)

clobber: clean
    $(RM) $(build_recorder_FILES)

EXTRA_DIST = $(build_recorder_FILES)

It still has issues though, but is this what you were thinking? Any value in continuing on with it?

[zvr]

Very briefly:

• 186 will simply add more targets to the makefile (Makefile.am in our case)

• the location of the build-recorder executable should be inherited from the top-level Makefile
• the tests should be the compilations, as they are currently
• don't be confused that the test/benchmarking task is a compilation; this is not something that automake should change
• (maybe not even use _SOURCES, if this implies compilation)

[fvalasiad]

@zvr Autotools generally have a test suite, using the TESTS variable, have it point to a script.

Do we have any reason not to use it? In the past we spoke about how to test build recorder's correctness, and we concluded that using standard scripts and grep would be enough. Do we have any reason not to just create a script that runs the test cases and also test their output? test/tests.sh

#!/bin/bash

build_recorder=$(../build-recorder)

$build_recorder -o compile_single_file.out $(CC) f1.c
# Test compile_single_file.out
...

Toplevel Makefile.am:

TESTS = test/tests.sh

and to run the tests:

make check

[fvalasiad]

@zvr If not the above then I don't know... using _SOURCES doesn't really work all that well. Make tries to premature compile the targets specified in _SOURCES and those builds aren't traced by build-recorder.

As far as I can tell automake simplifies the whole process when the targets are executables that need to be built & linked. This isn't the case in our scenario. Does automake offer some other functionality I haven't taken into consideration that makes it a superior option to a simple Makefile or to the test suite included in autotools?

[fvalasiad]

Closing this as it's no longer relevant.