search

NVlabs / stylegan3

Official PyTorch implementation of StyleGAN3
Other
6.43k stars 1.13k forks source link

Setting up PyTorch plugin "bias_act_plugin"... Failed! #86

Open Andredenise opened 2 years ago

Andredenise commented 2 years ago

Can someone please help me with this issue? I have set up everything correctly as in the readme file but I always get the same outcome like this: 

Constructing networks...
Setting up PyTorch plugin "bias_act_plugin"... Failed!
Traceback (most recent call last):
  File "C:\Users\student\Documents\thomas\stylegan3-main\env\lib\site-packages\torch\utils\cpp_extension.py", line 1666, in _run_ninja_build
    subprocess.run(
  File "C:\Program Files\Python38\lib\subprocess.py", line 516, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "train.py", line 286, in <module>
    main() # pylint: disable=no-value-for-parameter
  File "C:\Users\student\Documents\thomas\stylegan3-main\env\lib\site-packages\click\core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "C:\Users\student\Documents\thomas\stylegan3-main\env\lib\site-packages\click\core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "C:\Users\student\Documents\thomas\stylegan3-main\env\lib\site-packages\click\core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Users\student\Documents\thomas\stylegan3-main\env\lib\site-packages\click\core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "train.py", line 281, in main
    launch_training(c=c, desc=desc, outdir=opts.outdir, dry_run=opts.dry_run)
  File "train.py", line 96, in launch_training
    subprocess_fn(rank=0, c=c, temp_dir=temp_dir)
  File "train.py", line 47, in subprocess_fn
    training_loop.training_loop(rank=rank, **c)
  File "C:\Users\student\Documents\thomas\stylegan3-main\training\training_loop.py", line 168, in training_loop
    img = misc.print_module_summary(G, [z, c])
  File "C:\Users\student\Documents\thomas\stylegan3-main\torch_utils\misc.py", line 216, in print_module_summary
    outputs = module(*inputs)
  File "C:\Users\student\Documents\thomas\stylegan3-main\env\lib\site-packages\torch\nn\modules\module.py", line 1071, in _call_impl
    result = forward_call(*input, **kwargs)
  File "C:\Users\student\Documents\thomas\stylegan3-main\training\networks_stylegan3.py", line 511, in forward
    ws = self.mapping(z, c, truncation_psi=truncation_psi, truncation_cutoff=truncation_cutoff, update_emas=update_emas)
  File "C:\Users\student\Documents\thomas\stylegan3-main\env\lib\site-packages\torch\nn\modules\module.py", line 1071, in _call_impl
    result = forward_call(*input, **kwargs)
  File "C:\Users\student\Documents\thomas\stylegan3-main\training\networks_stylegan3.py", line 151, in forward
    x = getattr(self, f'fc{idx}')(x)
  File "C:\Users\student\Documents\thomas\stylegan3-main\env\lib\site-packages\torch\nn\modules\module.py", line 1071, in _call_impl
    result = forward_call(*input, **kwargs)
  File "C:\Users\student\Documents\thomas\stylegan3-main\training\networks_stylegan3.py", line 100, in forward
    x = bias_act.bias_act(x, b, act=self.activation)
  File "C:\Users\student\Documents\thomas\stylegan3-main\torch_utils\ops\bias_act.py", line 84, in bias_act
    if impl == 'cuda' and x.device.type == 'cuda' and _init():
  File "C:\Users\student\Documents\thomas\stylegan3-main\torch_utils\ops\bias_act.py", line 41, in _init
    _plugin = custom_ops.get_plugin(
  File "C:\Users\student\Documents\thomas\stylegan3-main\torch_utils\custom_ops.py", line 136, in get_plugin
    torch.utils.cpp_extension.load(name=module_name, build_directory=cached_build_dir,
  File "C:\Users\student\Documents\thomas\stylegan3-main\env\lib\site-packages\torch\utils\cpp_extension.py", line 1080, in load
    return _jit_compile(
  File "C:\Users\student\Documents\thomas\stylegan3-main\env\lib\site-packages\torch\utils\cpp_extension.py", line 1293, in _jit_compile
    _write_ninja_file_and_build_library(
  File "C:\Users\student\Documents\thomas\stylegan3-main\env\lib\site-packages\torch\utils\cpp_extension.py", line 1405, in _write_ninja_file_and_build_library
    _run_ninja_build(
  File "C:\Users\student\Documents\thomas\stylegan3-main\env\lib\site-packages\torch\utils\cpp_extension.py", line 1682, in _run_ninja_build
    raise RuntimeError(message) from e
RuntimeError: Error building extension 'bias_act_plugin': [1/3] C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin\nvcc --generate-dependencies-with-compile --dependency-output bias_act.cuda.o.d -Xcudafe --diag_suppress=dll_interface_conflict_dllexport_assumed -Xcudafe --diag_suppress=dll_interface_conflict_none_assumed -Xcudafe --diag_suppress=field_without_dll_interface -Xcudafe --diag_suppress=base_class_has_different_dll_interface -Xcompiler /EHsc -Xcompiler /wd4190 -Xcompiler /wd4018 -Xcompiler /wd4275 -Xcompiler /wd4267 -Xcompiler /wd4244 -Xcompiler /wd4251 -Xcompiler /wd4819 -Xcompiler /MD -DTORCH_EXTENSION_NAME=bias_act_plugin -DTORCH_API_INCLUDE_EXTENSION_H -IC:\Users\student\Documents\thomas\stylegan3-main\env\lib\site-packages\torch\include -IC:\Users\student\Documents\thomas\stylegan3-main\env\lib\site-packages\torch\include\torch\csrc\api\include -IC:\Users\student\Documents\thomas\stylegan3-main\env\lib\site-packages\torch\include\TH -IC:\Users\student\Documents\thomas\stylegan3-main\env\lib\site-packages\torch\include\THC "-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include" "-IC:\Program Files\Python38\Include" -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_75,code=compute_75 -gencode=arch=compute_75,code=sm_75 --use_fast_math -c C:\Users\student\AppData\Local\torch_extensions\torch_extensions\Cache\bias_act_plugin\3cb576a0039689487cfba59279dd6d46-nvidia-geforce-rtx-2080-ti\bias_act.cu -o bias_act.cuda.o
FAILED: bias_act.cuda.o
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin\nvcc --generate-dependencies-with-compile --dependency-output bias_act.cuda.o.d -Xcudafe --diag_suppress=dll_interface_conflict_dllexport_assumed -Xcudafe --diag_suppress=dll_interface_conflict_none_assumed -Xcudafe --diag_suppress=field_without_dll_interface -Xcudafe --diag_suppress=base_class_has_different_dll_interface -Xcompiler /EHsc -Xcompiler /wd4190 -Xcompiler /wd4018 -Xcompiler /wd4275 -Xcompiler /wd4267 -Xcompiler /wd4244 -Xcompiler /wd4251 -Xcompiler /wd4819 -Xcompiler /MD -DTORCH_EXTENSION_NAME=bias_act_plugin -DTORCH_API_INCLUDE_EXTENSION_H -IC:\Users\student\Documents\thomas\stylegan3-main\env\lib\site-packages\torch\include -IC:\Users\student\Documents\thomas\stylegan3-main\env\lib\site-packages\torch\include\torch\csrc\api\include -IC:\Users\student\Documents\thomas\stylegan3-main\env\lib\site-packages\torch\include\TH -IC:\Users\student\Documents\thomas\stylegan3-main\env\lib\site-packages\torch\include\THC "-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include" "-IC:\Program Files\Python38\Include" -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_75,code=compute_75 -gencode=arch=compute_75,code=sm_75 --use_fast_math -c C:\Users\student\AppData\Local\torch_extensions\torch_extensions\Cache\bias_act_plugin\3cb576a0039689487cfba59279dd6d46-nvidia-geforce-rtx-2080-ti\bias_act.cu -o bias_act.cuda.o
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include\crt/host_config.h(160): fatal error C1189: #error:  -- unsupported Microsoft Visual Studio version! Only the versions between 2017 and 2019 (inclusive) are supported! The nvcc flag '-allow-unsupported-compiler' can be used to override this version check; however, using an unsupported host compiler may cause compilation failure or incorrect run time execution. Use at your own risk.
bias_act.cu
[2/3] cl /showIncludes -DTORCH_EXTENSION_NAME=bias_act_plugin -DTORCH_API_INCLUDE_EXTENSION_H -IC:\Users\student\Documents\thomas\stylegan3-main\env\lib\site-packages\torch\include -IC:\Users\student\Documents\thomas\stylegan3-main\env\lib\site-packages\torch\include\torch\csrc\api\include -IC:\Users\student\Documents\thomas\stylegan3-main\env\lib\site-packages\torch\include\TH -IC:\Users\student\Documents\thomas\stylegan3-main\env\lib\site-packages\torch\include\THC "-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include" "-IC:\Program Files\Python38\Include" -D_GLIBCXX_USE_CXX11_ABI=0 /MD /wd4819 /wd4251 /wd4244 /wd4267 /wd4275 /wd4018 /wd4190 /EHsc -c C:\Users\student\AppData\Local\torch_extensions\torch_extensions\Cache\bias_act_plugin\3cb576a0039689487cfba59279dd6d46-nvidia-geforce-rtx-2080-ti\bias_act.cpp /Fobias_act.o
Microsoft (R) C/C++ Optimizing Compiler Version 19.30.30706 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.
felkoh commented 2 years ago

I'm getting same error on a new PC setup, any luck resolving it?

Andredenise commented 2 years ago

No, but I changed projects and did it with the stylegan2-ada-pytorch. I checked this tutorial for it and he is now training without any issues. link: https://www.youtube.com/watch?v=BCde68k6KXg&t=904s Good luck

cazubris commented 2 years ago

Same issue here, curious what does the prof from WU do? does he use the docker image?

(venv) D:\DATA\stylegan3>python gen_images.py --outdir=out --trunc=1 --seeds=2 --network=https://api.ngc.nvidia.co m/v2/models/nvidia/research/stylegan3/versions/1/files/stylegan3-r-afhqv2-512x512.pkl Loading networks from "https://api.ngc.nvidia.com/v2/models/nvidia/research/stylegan3/versions/1/files/stylegan3-r -afhqv2-512x512.pkl"... Generating image for seed 2 (0/1) ... Setting up PyTorch plugin "bias_act_plugin"... Failed! Traceback (most recent call last): File "D:\DATA\stylegan3\venv\lib\site-packages\torch\utils\cpp_extension.py", line 1717, in _run_ninja_build subprocess.run( File "C:\Python39\lib\subprocess.py", line 524, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "D:\DATA\stylegan3\gen_images.py", line 143, in generate_images() # pylint: disable=no-value-for-parameter File "D:\DATA\stylegan3\venv\lib\site-packages\click\core.py", line 1128, in call return self.main(args, kwargs) File "D:\DATA\stylegan3\venv\lib\site-packages\click\core.py", line 1053, in main rv = self.invoke(ctx) File "D:\DATA\stylegan3\venv\lib\site-packages\click\core.py", line 1395, in invoke return ctx.invoke(self.callback, ctx.params) File "D:\DATA\stylegan3\venv\lib\site-packages\click\core.py", line 754, in invoke return __callback(args, kwargs) File "D:\DATA\stylegan3\gen_images.py", line 135, in generate_images img = G(z, label, truncation_psi=truncation_psi, noise_mode=noise_mode) File "D:\DATA\stylegan3\venv\lib\site-packages\torch\nn\modules\module.py", line 1102, in _call_impl return forward_call(*input, *kwargs) File "", line 503, in forward File "D:\DATA\stylegan3\venv\lib\site-packages\torch\nn\modules\module.py", line 1102, in _call_impl return forward_call(input, kwargs) File "", line 143, in forward File "D:\DATA\stylegan3\venv\lib\site-packages\torch\nn\modules\module.py", line 1102, in _call_impl return forward_call(*input, **kwargs) File "", line 92, in forward File "D:\DATA\stylegan3\torch_utils\ops\bias_act.py", line 84, in bias_act if impl == 'cuda' and x.device.type == 'cuda' and _init(): File "D:\DATA\stylegan3\torch_utils\ops\bias_act.py", line 41, in _init _plugin = custom_ops.get_plugin( File "D:\DATA\stylegan3\torch_utils\custom_ops.py", line 136, in get_plugin torch.utils.cpp_extension.load(name=module_name, build_directory=cached_build_dir, File "D:\DATA\stylegan3\venv\lib\site-packages\torch\utils\cpp_extension.py", line 1124, in load return _jit_compile( File "D:\DATA\stylegan3\venv\lib\site-packages\torch\utils\cpp_extension.py", line 1337, in _jit_compile _write_ninja_file_and_build_library( File "D:\DATA\stylegan3\venv\lib\site-packages\torch\utils\cpp_extension.py", line 1449, in _write_ninja_file_an d_build_library _run_ninja_build( File "D:\DATA\stylegan3\venv\lib\site-packages\torch\utils\cpp_extension.py", line 1733, in _run_ninja_build raise RuntimeError(message) from e RuntimeError: Error building extension 'bias_act_plugin': [1/1] "C:\Program Files (x86)\Microsoft Visual Studio\20 19\BuildTools\VC\Tools\MSVC\14.28.29333\bin\Hostx64\x64/link.exe" bias_act.o bias_act.cuda.o /nologo /DLL c10.lib c10_cuda.lib torch_cpu.lib torch_cuda_cu.lib -INCLUDE:?searchsorted_cuda@native@at@@YA?AVTensor@2@AEBV32@0_N1@Z to rch_cuda_cpp.lib -INCLUDE:?warp_size@cuda@at@@YAHXZ torch.lib /LIBPATH:D:\DATA\stylegan3\venv\lib\site-packages\to rch\lib torch_python.lib /LIBPATH:D:\DATA\stylegan3\venv\Scripts\libs "/LIBPATH:C:\Program Files\NVIDIA GPU Comput ing Toolkit\CUDA\v11.5\lib/x64" cudart.lib /out:bias_act_plugin.pyd FAILED: bias_act_plugin.pyd "C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.28.29333\bin\Hostx64\x64/link.exe " bias_act.o bias_act.cuda.o /nologo /DLL c10.lib c10_cuda.lib torch_cpu.lib torch_cuda_cu.lib -INCLUDE:?searchsor ted_cuda@native@at@@YA?AVTensor@2@AEBV32@0_N1@Z torch_cuda_cpp.lib -INCLUDE:?warp_size@cuda@at@@YAHXZ torch.lib /L IBPATH:D:\DATA\stylegan3\venv\lib\site-packages\torch\lib torch_python.lib /LIBPATH:D:\DATA\stylegan3\venv\Scripts \libs "/LIBPATH:C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.5\lib/x64" cudart.lib /out:bias_act_plugin. pyd LINK : fatal error LNK1104: cannot open file 'python39.lib' ninja: build stopped: subcommand failed.

felkoh commented 2 years ago

This post helped me. Not sure if it's the same issue but it now runs fine when executed through the command line

BiPaulEr commented 2 years ago

Fatal error LNK1181: cannot open input file 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\lib\x64.obj' for me

johannwyh commented 2 years ago

I am working on Linux platform. As for me, the problem is that the compiler cannot load libmpc.so.2. This file is not included in standard GCC library. To include these, I additionally included gmp-4.3.2, mpc-0.8.1 and mpfr-2.4.2 into LD_LIBRARY_PATH. Then the problem is solved. Hope things above can help.

zhywanna commented 2 years ago

@johannwyh

I am working on Linux platform. As for me, the problem is that the compiler cannot load . This file is not included in standard GCC library. To include these, I additionally included , and into . Then the problem is solved. Hope things above can help.libmpc.so.2``gmp-4.3.2``mpc-0.8.1``mpfr-2.4.2``LD_LIBRARY_PATH

could you please tell me how to do that?

johannwyh commented 2 years ago

@johannwyh

I am working on Linux platform. As for me, the problem is that the compiler cannot load . This file is not included in standard GCC library. To include these, I additionally included , and into . Then the problem is solved. Hope things above can help.libmpc.so.2``gmp-4.3.2``mpc-0.8.1``mpfr-2.4.2``LD_LIBRARY_PATH

could you please tell me how to do that?

You can refer to this link. Just do things that are necessary, for example I think you do not need to reinstall the gcc. You should focus on installing the three packages and link it in LD_LIBRARY_PATH

ileniTudor commented 9 months ago

I managed to run on Windows 11 the stylegan3 as following :

create the conda env: install CUDA 11.3 on the system (not sure if necessary) - make sure you have a single CUDA installed (in my case I had both 11.3 and 21.3) and I have to uninstall version 21.3 (and remove all paths to 21.3 form Env. Variables)

install cudatoolkit == 11.3.1(using conda)

install torch 1.9.0+cu111 using (pip install torch==1.9.0+cu111 torchvision==0.10.0+cu111 torchaudio==0.9.0 -f https://download.pytorch.org/whl/torch_stable.html)

install the rest of the requirements from the environment.yml

Install Visual Studio Community 2019. Make sure you check "Desktop development with C++". "Universal Windows Platform development" was not required in my cased.

run the following bat: "C:\Program Files (x86)\Microsoft Visual Studio\\Community\VC\Auxiliary\Build\vcvars64.bat"

for more explanation watch this video