NVlabs / nvdiffrast

Nvdiffrast - Modular Primitives for High-Performance Differentiable Rendering
Other
1.37k stars 146 forks source link

Lock file exists in build directory: '/root/.cache/torch_extensions/py38_cu113/nvdiffrast_plugin/lock' #67

Closed simon-cross closed 2 years ago

simon-cross commented 2 years ago

I have installed nvdiffrast by running pip3 install . at the root of the NVDiffRast repository.

When I run my code that imports NVDiffRast, I get the following message: Lock file exists in build directory: '/root/.cache/torch_extensions/py38_cu113/nvdiffrast_plugin/lock'

Nevertheless, my code seems to run without problems. What is this error message? how can I solve the problem related to the lock file?

s-laine commented 2 years ago

You may see this message when you're importing nvdiffrast in multiple threads or processes simultaneously, and the plugin compilation is being synchronized by PyTorch via the lock file. This is not an error but expected behavior, and the message is intended to be merely informative (although it is given via logger.warning() because otherwise it would be silent by default).

The reason for the message is that sometimes, if the plugin compilation is interrupted by e.g. terminating the process, PyTorch leaves a stale lock file in the compilation directory. If this happens, all future attempts to compile the plugin will hang, with no error given. The message tells where the lock file is so that if the compilation seems to hang, it can be deleted manually.

jpainam commented 7 months ago

what file should I delete? ./cache/torch_extensions/nvdiffrast_plugin/lock, only the lock file or all the nvdiffrast_plugin extension?

s-laine commented 7 months ago

I'm assuming you are stuck with PyTorch waiting on the lock file that was left in the directory.

Deleting just the lock file allows PyTorch to move on with the compilation. If the plugin doesn't seem to work, you can try deleting the entire extension directory to force a full recompilation.