search

ocaml / ocamlbuild

The legacy OCamlbuild build manager
Other
122 stars 81 forks source link

Fix race conditions when invoking `ocamlbuild` in parallel from a makefile #302

Closed gasche closed 5 years ago

gasche commented 5 years ago

I'm creating this PR to track my progress on fixing #300 and related race-conditions issue. The current reproduction testcase is as follows:

DIR=foo
FILE=code.ml

setup:
    mkdir -p $(DIR)
    echo "let x = 1" > $(DIR)/$(FILE)
    echo "run 'make -j all' to reproduce the failure'"

clean:
    ocamlbuild -clean

dist-clean: clean
    rm $(DIR)/$(FILE)
    rmdir $(DIR)

.PHONY: all test1 test2
all: test1 test2

test1:
    $(MAKE) _build/$(DIR)/$(FILE)

test2:
    $(MAKE) _build/$(DIR)/$(FILE)

_build/$(DIR)/$(FILE):
    ocamlbuild $(DIR)/$(FILE)

running make setup; make -j all exposes races in ocamlbuild's parallel accesses to the filesystem.

I don't think we will ever try to guarantee that concurrent ocamlbuild invocations work robustly, but we should at least fix the issues that users encounter in the wild, when reasonably possible.

gasche commented 5 years ago

I tried to deal with a race in mkdir_p and another in rm_f, and now the reproduction script works reliably on my machine. I'm thinking of merging this, getting a minor release out, and waiting for more user reports to do more if needed.