Build race

[rossburton]

Our autobuilder just failed with the following failure in a parallel build:

error: unable to open output file 'builddir/mem_buff_demo.o': 'No such file or directory'

Looks like a missing dependency somewhere.

[rossburton]

Happened again:

error: unable to open output file 'builddir/c_api_pkt_print_test.o': 'No such file or directory'

[rossburton]

So the basic problem is that this is not doing what you want:

https://github.com/Linaro/OpenCSD/blob/master/decoder/tests/build/linux/mem_buffer_eg/makefile#L54

all:  build_dir copy_libs

In a parallel build, build_dir and copy_libs are happening at the same time. Typically the mkdir in build_dir will happen fast but given a loaded build machine and that isn't guaranteed.

This is easily verified by making the build_dir target very slow:

build_dir:
    sleep 10
    mkdir -p $(BUILD_DIR)

make[1]: Entering directory '/home/ross/Code/OpenCSD/decoder/tests/build/linux/mem_buffer_eg'
sleep 10
g++  -c -Wall -DLINUX -Wno-switch -Wlogical-op -fPIC -std=c++11 -O2 -DNDEBUG -I/home/ross/Code/OpenCSD/decoder/tests/source -I/home/ross/Code/OpenCSD/decoder/include -I/home/ross/Code/OpenCSD/decoder/tests/snapshot_parser_lib/include -MMD /home/ross/Code/OpenCSD/decoder/tests/source/mem_buff_demo.cpp -o builddir/mem_buff_demo.o
/home/ross/Code/OpenCSD/decoder/tests/source/mem_buff_demo.cpp: In function ‘int initDataBuffers()’:
/home/ross/Code/OpenCSD/decoder/tests/source/mem_buff_demo.cpp:161:10: warning: ignoring return value of ‘size_t fread(void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result [-Wunused-result]
  161 |     fread(input_trace_data, 1, input_trace_data_size, fp);
      |     ~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/ross/Code/OpenCSD/decoder/tests/source/mem_buff_demo.cpp:181:10: warning: ignoring return value of ‘size_t fread(void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result [-Wunused-result]
  181 |     fread(program_image_buffer, 1, program_image_size, fp);
      |     ~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/ross/Code/OpenCSD/decoder/tests/source/mem_buff_demo.cpp: At global scope:
/home/ross/Code/OpenCSD/decoder/tests/source/mem_buff_demo.cpp:416:1: fatal error: opening dependency file builddir/mem_buff_demo.d: No such file or directory
  416 | }
      | ^
compilation terminated.

Basically, Makefiles are hard. Have you considered something like Meson as an alternative?

[mikel-armbb]

Hi,

Thanks for spotting this. We do do some parallel build testing, but have not encountered the issue. I presume that the distros that also build this have not encountered issues either. I'll look into this.

Regards

Mike

[mikel-armbb]

Fixed by adding order only pre-requisites of build_dir for compilation of object files. Added to all makefiles in tests build area. Will appear in the next release.

[rossburton]

Can I check that you haven't pushed this already? I was going to cherry-pick the commit to our packaging.

[mikel-armbb]

Hi

No this has not been pushed yet. There are a number of minor updates in line for the next release. This should occur during this week. You will see an update to the minor rev on github, and there is generally an announcement on the coresight mailing list.

Mike

On Mon, 16 Nov 2020 at 19:58, Ross Burton notifications@github.com wrote:

Can I check that you haven't pushed this already? I was going to cherry-pick the commit to our packaging.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/Linaro/OpenCSD/issues/32#issuecomment-728290754, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADF7BM32GADE3LWNTQ3UGXDSQF767ANCNFSM4TRGGFWQ .

-- Mike Leach Principal Engineer, ARM Ltd. Manchester Design Centre. UK