ninja-build / ninja

a small build system with a focus on speed
https://ninja-build.org/
Apache License 2.0
11.27k stars 1.6k forks source link

Multiple threads copy file error on Windows #2402

Open jiannanya opened 8 months ago

jiannanya commented 8 months ago

Hi there.

When using cmake to add custom command to copy files like dlls to target output directory, since Ninja works with default multiple threads, the copy operation will fail after one thread has copy the same file.

When run ninja -j 1 then it will success.

My current operating system is Windows.

So is there anyway to work around it?

digit-google commented 8 months ago

There is too little information here to answer you. E.g. what kind of failure are you talking exactly? Do you have a minimal reproduction case to share? So far, it looks like your CMake commands are running in parallel and lack the proper dependencies in your CMake declarations.

jiannanya commented 7 months ago

There is too little information here to answer you. E.g. what kind of failure are you talking exactly? Do you have a minimal reproduction case to share? So far, it looks like your CMake commands are running in parallel and lack the proper dependencies in your CMake declarations.

For example, this line in the CMakeLists.txt https://github.com/JoeyDeVries/LearnOpenGL/blob/f2dc2e1904fe920ec52c734bd56888ab870ff937/CMakeLists.txt#L255

When copy file in parallel with Ninja build command, cmake will run into error due to multiple threads are coping the same file in the same time.

How to copy files in parallel with Ninja like such? I know the key is to add judgement when coping files, but I still want to know if Ninja has anything like atomic operation in c++?

digit-google commented 7 months ago

This looks like a CMake-specific issue (e.g. you are using two PRE_BUILD add_custom_commands to write to the same location), not something related to Ninja itself. I recommend you ask on CMake-specific projects for help.

Ninja has multiple ways to ensure that commands only run in deterministic sequence when needed, here it looks like CMake didn't generate a Ninja build plan that used them properly.