premake / premake-core

Premake
https://premake.github.io/
BSD 3-Clause "New" or "Revised" License
3.13k stars 611 forks source link

Ignore mkdir errors on Windows with gmake2 generator #2181

Closed tuccio closed 4 months ago

tuccio commented 4 months ago

What does this PR do?

Two different paths are implemented for sh and Windows shells in gmake2, which output something like:

$(TARGETDIR):
    @echo Creating $(TARGETDIR)
ifeq (posix,$(SHELLTYPE))
    $(SILENT) mkdir -p $(TARGETDIR)
else
    $(SILENT) mkdir $(subst /,\\,$(TARGETDIR))
endif

There are some differences between the POSIX mkdir -p and mkdir on cmd, one of them is that mkdir on Windows fails if a path already exists. This is not much of a problem in a single-threaded scenario, since the Makefile target takes care of checking whether the directory exists, but you quickly run into problems when running make with multiple parallel jobs (e.g. -j16), and anything may happen between the check and the mkdir, including some other job creating the directory.

How does this PR change Premake's behavior?

I did not find a better solution than ignoring the mkdir errors in the branch for cmd, so this PR effectively changes the output of the previous snippet to:

$(TARGETDIR):
    @echo Creating $(TARGETDIR)
ifeq (posix,$(SHELLTYPE))
    $(SILENT) mkdir -p $(TARGETDIR)
else
    $(SILENT) -mkdir $(subst /,\\,$(TARGETDIR))
endif

Upon hitting the issue the output will now look like this and make will carry on instead of exiting:

make[1]: [foo.make:362: bin/release] Error 1 (ignored)

Anything else we should know?

Unfortunately all mkdir errors are ignored, not just the ones caused by the directory already existing. Although, while not ideal, it seemed better than building in a single thread for a large project.

Did you check all the boxes?

You can now support Premake on our OpenCollective. Your contributions help us spend more time responding to requests like these!

Jarod42 commented 4 months ago

run into problems when running make with multiple parallel jobs

Then, I would say the dependencies are wrong. Does echo Creating $(TARGETDIR) appears several time in that scenario?

tuccio commented 4 months ago

run into problems when running make with multiple parallel jobs

Then, I would say the dependencies are wrong. Does echo Creating $(TARGETDIR) appears several time in that scenario?

Yes, that is the case on Linux builds as well, they just don't fail because that's how mkdir -p works. Which is why I thought the easy fix could be ok to share: they already behave differently, I think my change makes the differences less "punishing" when trying to build on Windows.

I'm guessing the problem is because a few different projects, that don't depend on each other, all output binaries to the same directories.

I suppose one could also either get more creative with adding phony dependencies for target directories to make sure they are created before anything starts, or other solutions without changing premake, but the user experience of finding out that your build setup (that mostly seems to make sense) works on Linux but fails on Windows with a weird error is not the best, so I thought it could be helpful to share the workaround.