search

Fictionarry / ER-NeRF

[ICCV'23] Efficient Region-Aware Neural Radiance Fields for High-Fidelity Talking Portrait Synthesis
https://fictionarry.github.io/ER-NeRF/
MIT License
906 stars 125 forks source link

Is AU45 required ? #3

Closed mortal-Zero closed 12 months ago

mortal-Zero commented 12 months ago

Hi, Thanks for your project.

I want to know is the data/<ID>/au.csv from runing FeatureExtraction in OpenFace required?

If I don't run the FeatureExtraction, can I run the training process normally?

Fictionarry commented 12 months ago

Hi, we use AU45 to describe the degree of eye blinking in our implementation. You can modify the code at about line 485 in nerf_triplane/provider.py to enable the original eye blinking representation from RAD-NeRF that is no need for AU45. However, the performance may drop a bit, especially for the eye area.

mortal-Zero commented 12 months ago

Hi, we use AU45 to describe the degree of eye blinking in our implementation. You can modify the code at about line 485 in nerf_triplane/provider.py to enable the original eye blinking representation from RAD-NeRF that is no need for AU45. However, the performance may drop a bit, especially for the eye area.

Thanks for your reply. And, When I run the main.py, I got an error:

Traceback (most recent call last):
  File "/workspace/baiyixuan/ER-NeRF/raymarching/raymarching.py", line 10, in <module>
    import _raymarching_face as _backend
ModuleNotFoundError: No module named '_raymarching_face'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1808, in _run_ninja_build
    subprocess.run(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workspace/baiyixuan/ER-NeRF/main.py", line 6, in <module>
    from nerf_triplane.network import NeRFNetwork
  File "/workspace/baiyixuan/ER-NeRF/nerf_triplane/network.py", line 6, in <module>
    from .renderer import NeRFRenderer
  File "/workspace/baiyixuan/ER-NeRF/nerf_triplane/renderer.py", line 10, in <module>
    import raymarching
  File "/workspace/baiyixuan/ER-NeRF/raymarching/__init__.py", line 1, in <module>
    from .raymarching import *
  File "/workspace/baiyixuan/ER-NeRF/raymarching/raymarching.py", line 12, in <module>
    from .backend import _backend
  File "/workspace/baiyixuan/ER-NeRF/raymarching/backend.py", line 31, in <module>
    _backend = load(name='_raymarching_face',
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1202, in load
    return _jit_compile(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1425, in _jit_compile
    _write_ninja_file_and_build_library(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1537, in _write_ninja_file_and_build_library
    _run_ninja_build(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1824, in _run_ninja_build
    raise RuntimeError(message) from e
RuntimeError: Error building extension '_raymarching_face': [1/1] c++ raymarching.cuda.o bindings.o -shared -L/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/lib -lc10 -lc10_cuda -ltorch_cpu -ltorch_cuda_cu -ltorch_cuda_cpp -ltorch -ltorch_python -L/workspace/miniconda3/envs/ernerf/lib64 -lcudart -o _raymarching_face.so
FAILED: _raymarching_face.so 
c++ raymarching.cuda.o bindings.o -shared -L/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/lib -lc10 -lc10_cuda -ltorch_cpu -ltorch_cuda_cu -ltorch_cuda_cpp -ltorch -ltorch_python -L/workspace/miniconda3/envs/ernerf/lib64 -lcudart -o _raymarching_face.so
/usr/bin/ld: cannot find -lcudart: No such file or directory
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

It looks like an error when do orch.utils.cpp_extension.load. My env is: Ubuntu22.04, gcc-9, g++-9, cuda11.3, python3.10, torch1.12.1, and the pytorch3d is installed.

mortal-Zero commented 12 months ago

Hi, we use AU45 to describe the degree of eye blinking in our implementation. You can modify the code at about line 485 in nerf_triplane/provider.py to enable the original eye blinking representation from RAD-NeRF that is no need for AU45. However, the performance may drop a bit, especially for the eye area.

Thanks for your reply. And, When I run the main.py, I got an error:

Traceback (most recent call last):
  File "/workspace/baiyixuan/ER-NeRF/raymarching/raymarching.py", line 10, in <module>
    import _raymarching_face as _backend
ModuleNotFoundError: No module named '_raymarching_face'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1808, in _run_ninja_build
    subprocess.run(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workspace/baiyixuan/ER-NeRF/main.py", line 6, in <module>
    from nerf_triplane.network import NeRFNetwork
  File "/workspace/baiyixuan/ER-NeRF/nerf_triplane/network.py", line 6, in <module>
    from .renderer import NeRFRenderer
  File "/workspace/baiyixuan/ER-NeRF/nerf_triplane/renderer.py", line 10, in <module>
    import raymarching
  File "/workspace/baiyixuan/ER-NeRF/raymarching/__init__.py", line 1, in <module>
    from .raymarching import *
  File "/workspace/baiyixuan/ER-NeRF/raymarching/raymarching.py", line 12, in <module>
    from .backend import _backend
  File "/workspace/baiyixuan/ER-NeRF/raymarching/backend.py", line 31, in <module>
    _backend = load(name='_raymarching_face',
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1202, in load
    return _jit_compile(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1425, in _jit_compile
    _write_ninja_file_and_build_library(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1537, in _write_ninja_file_and_build_library
    _run_ninja_build(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1824, in _run_ninja_build
    raise RuntimeError(message) from e
RuntimeError: Error building extension '_raymarching_face': [1/1] c++ raymarching.cuda.o bindings.o -shared -L/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/lib -lc10 -lc10_cuda -ltorch_cpu -ltorch_cuda_cu -ltorch_cuda_cpp -ltorch -ltorch_python -L/workspace/miniconda3/envs/ernerf/lib64 -lcudart -o _raymarching_face.so
FAILED: _raymarching_face.so 
c++ raymarching.cuda.o bindings.o -shared -L/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/lib -lc10 -lc10_cuda -ltorch_cpu -ltorch_cuda_cu -ltorch_cuda_cpp -ltorch -ltorch_python -L/workspace/miniconda3/envs/ernerf/lib64 -lcudart -o _raymarching_face.so
/usr/bin/ld: cannot find -lcudart: No such file or directory
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

It looks like an error when do orch.utils.cpp_extension.load. My env is: Ubuntu22.04, gcc-9, g++-9, cuda11.3, python3.10, torch1.12.1, and the pytorch3d is installed.

Sorry, my bad, I have solved the problem.

shenriver commented 10 months ago

Hi, we use AU45 to describe the degree of eye blinking in our implementation. You can modify the code at about line 485 in nerf_triplane/provider.py to enable the original eye blinking representation from RAD-NeRF that is no need for AU45. However, the performance may drop a bit, especially for the eye area.

Thanks for your reply. And, When I run the main.py, I got an error:

Traceback (most recent call last):
  File "/workspace/baiyixuan/ER-NeRF/raymarching/raymarching.py", line 10, in <module>
    import _raymarching_face as _backend
ModuleNotFoundError: No module named '_raymarching_face'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1808, in _run_ninja_build
    subprocess.run(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workspace/baiyixuan/ER-NeRF/main.py", line 6, in <module>
    from nerf_triplane.network import NeRFNetwork
  File "/workspace/baiyixuan/ER-NeRF/nerf_triplane/network.py", line 6, in <module>
    from .renderer import NeRFRenderer
  File "/workspace/baiyixuan/ER-NeRF/nerf_triplane/renderer.py", line 10, in <module>
    import raymarching
  File "/workspace/baiyixuan/ER-NeRF/raymarching/__init__.py", line 1, in <module>
    from .raymarching import *
  File "/workspace/baiyixuan/ER-NeRF/raymarching/raymarching.py", line 12, in <module>
    from .backend import _backend
  File "/workspace/baiyixuan/ER-NeRF/raymarching/backend.py", line 31, in <module>
    _backend = load(name='_raymarching_face',
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1202, in load
    return _jit_compile(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1425, in _jit_compile
    _write_ninja_file_and_build_library(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1537, in _write_ninja_file_and_build_library
    _run_ninja_build(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1824, in _run_ninja_build
    raise RuntimeError(message) from e
RuntimeError: Error building extension '_raymarching_face': [1/1] c++ raymarching.cuda.o bindings.o -shared -L/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/lib -lc10 -lc10_cuda -ltorch_cpu -ltorch_cuda_cu -ltorch_cuda_cpp -ltorch -ltorch_python -L/workspace/miniconda3/envs/ernerf/lib64 -lcudart -o _raymarching_face.so
FAILED: _raymarching_face.so 
c++ raymarching.cuda.o bindings.o -shared -L/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/lib -lc10 -lc10_cuda -ltorch_cpu -ltorch_cuda_cu -ltorch_cuda_cpp -ltorch -ltorch_python -L/workspace/miniconda3/envs/ernerf/lib64 -lcudart -o _raymarching_face.so
/usr/bin/ld: cannot find -lcudart: No such file or directory
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

It looks like an error when do orch.utils.cpp_extension.load. My env is: Ubuntu22.04, gcc-9, g++-9, cuda11.3, python3.10, torch1.12.1, and the pytorch3d is installed.

Sorry, my bad, I have solved the problem.

how did you solve this problem?

mortal-Zero commented 10 months ago

Hi, we use AU45 to describe the degree of eye blinking in our implementation. You can modify the code at about line 485 in nerf_triplane/provider.py to enable the original eye blinking representation from RAD-NeRF that is no need for AU45. However, the performance may drop a bit, especially for the eye area.

Thanks for your reply. And, When I run the main.py, I got an error:

Traceback (most recent call last):
  File "/workspace/baiyixuan/ER-NeRF/raymarching/raymarching.py", line 10, in <module>
    import _raymarching_face as _backend
ModuleNotFoundError: No module named '_raymarching_face'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1808, in _run_ninja_build
    subprocess.run(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workspace/baiyixuan/ER-NeRF/main.py", line 6, in <module>
    from nerf_triplane.network import NeRFNetwork
  File "/workspace/baiyixuan/ER-NeRF/nerf_triplane/network.py", line 6, in <module>
    from .renderer import NeRFRenderer
  File "/workspace/baiyixuan/ER-NeRF/nerf_triplane/renderer.py", line 10, in <module>
    import raymarching
  File "/workspace/baiyixuan/ER-NeRF/raymarching/__init__.py", line 1, in <module>
    from .raymarching import *
  File "/workspace/baiyixuan/ER-NeRF/raymarching/raymarching.py", line 12, in <module>
    from .backend import _backend
  File "/workspace/baiyixuan/ER-NeRF/raymarching/backend.py", line 31, in <module>
    _backend = load(name='_raymarching_face',
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1202, in load
    return _jit_compile(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1425, in _jit_compile
    _write_ninja_file_and_build_library(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1537, in _write_ninja_file_and_build_library
    _run_ninja_build(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1824, in _run_ninja_build
    raise RuntimeError(message) from e
RuntimeError: Error building extension '_raymarching_face': [1/1] c++ raymarching.cuda.o bindings.o -shared -L/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/lib -lc10 -lc10_cuda -ltorch_cpu -ltorch_cuda_cu -ltorch_cuda_cpp -ltorch -ltorch_python -L/workspace/miniconda3/envs/ernerf/lib64 -lcudart -o _raymarching_face.so
FAILED: _raymarching_face.so 
c++ raymarching.cuda.o bindings.o -shared -L/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/lib -lc10 -lc10_cuda -ltorch_cpu -ltorch_cuda_cu -ltorch_cuda_cpp -ltorch -ltorch_python -L/workspace/miniconda3/envs/ernerf/lib64 -lcudart -o _raymarching_face.so
/usr/bin/ld: cannot find -lcudart: No such file or directory
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

It looks like an error when do orch.utils.cpp_extension.load. My env is: Ubuntu22.04, gcc-9, g++-9, cuda11.3, python3.10, torch1.12.1, and the pytorch3d is installed.

Sorry, my bad, I have solved the problem.

how did you solve this problem?

what is your env?

shenriver commented 10 months ago

ubuntu20.04.1, torch==1.12.1+cu113, pytorch3d==0.7.2, Python==3.10.10, cuda-11.2

Hi, we use AU45 to describe the degree of eye blinking in our implementation. You can modify the code at about line 485 in nerf_triplane/provider.py to enable the original eye blinking representation from RAD-NeRF that is no need for AU45. However, the performance may drop a bit, especially for the eye area.

Thanks for your reply. And, When I run the main.py, I got an error:

Traceback (most recent call last):
  File "/workspace/baiyixuan/ER-NeRF/raymarching/raymarching.py", line 10, in <module>
    import _raymarching_face as _backend
ModuleNotFoundError: No module named '_raymarching_face'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1808, in _run_ninja_build
    subprocess.run(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workspace/baiyixuan/ER-NeRF/main.py", line 6, in <module>
    from nerf_triplane.network import NeRFNetwork
  File "/workspace/baiyixuan/ER-NeRF/nerf_triplane/network.py", line 6, in <module>
    from .renderer import NeRFRenderer
  File "/workspace/baiyixuan/ER-NeRF/nerf_triplane/renderer.py", line 10, in <module>
    import raymarching
  File "/workspace/baiyixuan/ER-NeRF/raymarching/__init__.py", line 1, in <module>
    from .raymarching import *
  File "/workspace/baiyixuan/ER-NeRF/raymarching/raymarching.py", line 12, in <module>
    from .backend import _backend
  File "/workspace/baiyixuan/ER-NeRF/raymarching/backend.py", line 31, in <module>
    _backend = load(name='_raymarching_face',
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1202, in load
    return _jit_compile(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1425, in _jit_compile
    _write_ninja_file_and_build_library(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1537, in _write_ninja_file_and_build_library
    _run_ninja_build(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1824, in _run_ninja_build
    raise RuntimeError(message) from e
RuntimeError: Error building extension '_raymarching_face': [1/1] c++ raymarching.cuda.o bindings.o -shared -L/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/lib -lc10 -lc10_cuda -ltorch_cpu -ltorch_cuda_cu -ltorch_cuda_cpp -ltorch -ltorch_python -L/workspace/miniconda3/envs/ernerf/lib64 -lcudart -o _raymarching_face.so
FAILED: _raymarching_face.so 
c++ raymarching.cuda.o bindings.o -shared -L/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/lib -lc10 -lc10_cuda -ltorch_cpu -ltorch_cuda_cu -ltorch_cuda_cpp -ltorch -ltorch_python -L/workspace/miniconda3/envs/ernerf/lib64 -lcudart -o _raymarching_face.so
/usr/bin/ld: cannot find -lcudart: No such file or directory
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

It looks like an error when do orch.utils.cpp_extension.load. My env is: Ubuntu22.04, gcc-9, g++-9, cuda11.3, python3.10, torch1.12.1, and the pytorch3d is installed.

Sorry, my bad, I have solved the problem.

how did you solve this problem?

what is your env?

https://github.com/Fictionarry/ER-NeRF/issues/3#issuecomment-1691384370

Hi, we use AU45 to describe the degree of eye blinking in our implementation. You can modify the code at about line 485 in nerf_triplane/provider.py to enable the original eye blinking representation from RAD-NeRF that is no need for AU45. However, the performance may drop a bit, especially for the eye area.

Thanks for your reply. And, When I run the main.py, I got an error:

Traceback (most recent call last):
  File "/workspace/baiyixuan/ER-NeRF/raymarching/raymarching.py", line 10, in <module>
    import _raymarching_face as _backend
ModuleNotFoundError: No module named '_raymarching_face'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1808, in _run_ninja_build
    subprocess.run(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workspace/baiyixuan/ER-NeRF/main.py", line 6, in <module>
    from nerf_triplane.network import NeRFNetwork
  File "/workspace/baiyixuan/ER-NeRF/nerf_triplane/network.py", line 6, in <module>
    from .renderer import NeRFRenderer
  File "/workspace/baiyixuan/ER-NeRF/nerf_triplane/renderer.py", line 10, in <module>
    import raymarching
  File "/workspace/baiyixuan/ER-NeRF/raymarching/__init__.py", line 1, in <module>
    from .raymarching import *
  File "/workspace/baiyixuan/ER-NeRF/raymarching/raymarching.py", line 12, in <module>
    from .backend import _backend
  File "/workspace/baiyixuan/ER-NeRF/raymarching/backend.py", line 31, in <module>
    _backend = load(name='_raymarching_face',
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1202, in load
    return _jit_compile(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1425, in _jit_compile
    _write_ninja_file_and_build_library(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1537, in _write_ninja_file_and_build_library
    _run_ninja_build(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1824, in _run_ninja_build
    raise RuntimeError(message) from e
RuntimeError: Error building extension '_raymarching_face': [1/1] c++ raymarching.cuda.o bindings.o -shared -L/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/lib -lc10 -lc10_cuda -ltorch_cpu -ltorch_cuda_cu -ltorch_cuda_cpp -ltorch -ltorch_python -L/workspace/miniconda3/envs/ernerf/lib64 -lcudart -o _raymarching_face.so
FAILED: _raymarching_face.so 
c++ raymarching.cuda.o bindings.o -shared -L/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/lib -lc10 -lc10_cuda -ltorch_cpu -ltorch_cuda_cu -ltorch_cuda_cpp -ltorch -ltorch_python -L/workspace/miniconda3/envs/ernerf/lib64 -lcudart -o _raymarching_face.so
/usr/bin/ld: cannot find -lcudart: No such file or directory
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

It looks like an error when do orch.utils.cpp_extension.load. My env is: Ubuntu22.04, gcc-9, g++-9, cuda11.3, python3.10, torch1.12.1, and the pytorch3d is installed.

Sorry, my bad, I have solved the problem.

how did you solve this problem?

what is your env?

ubuntu20.04.1, torch==1.12.1+cu113, pytorch3d==0.7.2, Python==3.10.10, cuda-11.2

Hi, we use AU45 to describe the degree of eye blinking in our implementation. You can modify the code at about line 485 in nerf_triplane/provider.py to enable the original eye blinking representation from RAD-NeRF that is no need for AU45. However, the performance may drop a bit, especially for the eye area.

Thanks for your reply. And, When I run the main.py, I got an error:

Traceback (most recent call last):
  File "/workspace/baiyixuan/ER-NeRF/raymarching/raymarching.py", line 10, in <module>
    import _raymarching_face as _backend
ModuleNotFoundError: No module named '_raymarching_face'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1808, in _run_ninja_build
    subprocess.run(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workspace/baiyixuan/ER-NeRF/main.py", line 6, in <module>
    from nerf_triplane.network import NeRFNetwork
  File "/workspace/baiyixuan/ER-NeRF/nerf_triplane/network.py", line 6, in <module>
    from .renderer import NeRFRenderer
  File "/workspace/baiyixuan/ER-NeRF/nerf_triplane/renderer.py", line 10, in <module>
    import raymarching
  File "/workspace/baiyixuan/ER-NeRF/raymarching/__init__.py", line 1, in <module>
    from .raymarching import *
  File "/workspace/baiyixuan/ER-NeRF/raymarching/raymarching.py", line 12, in <module>
    from .backend import _backend
  File "/workspace/baiyixuan/ER-NeRF/raymarching/backend.py", line 31, in <module>
    _backend = load(name='_raymarching_face',
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1202, in load
    return _jit_compile(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1425, in _jit_compile
    _write_ninja_file_and_build_library(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1537, in _write_ninja_file_and_build_library
    _run_ninja_build(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1824, in _run_ninja_build
    raise RuntimeError(message) from e
RuntimeError: Error building extension '_raymarching_face': [1/1] c++ raymarching.cuda.o bindings.o -shared -L/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/lib -lc10 -lc10_cuda -ltorch_cpu -ltorch_cuda_cu -ltorch_cuda_cpp -ltorch -ltorch_python -L/workspace/miniconda3/envs/ernerf/lib64 -lcudart -o _raymarching_face.so
FAILED: _raymarching_face.so 
c++ raymarching.cuda.o bindings.o -shared -L/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/lib -lc10 -lc10_cuda -ltorch_cpu -ltorch_cuda_cu -ltorch_cuda_cpp -ltorch -ltorch_python -L/workspace/miniconda3/envs/ernerf/lib64 -lcudart -o _raymarching_face.so
/usr/bin/ld: cannot find -lcudart: No such file or directory
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

It looks like an error when do orch.utils.cpp_extension.load. My env is: Ubuntu22.04, gcc-9, g++-9, cuda11.3, python3.10, torch1.12.1, and the pytorch3d is installed.

Sorry, my bad, I have solved the problem.

how did you solve this problem?

what is your env?

ubuntu20.04.1, torch==1.12.1+cu113, pytorch3d==0.7.2, Python==3.10.10, cuda-11.2

mortal-Zero commented 10 months ago

ubuntu20.04.1, torch==1.12.1+cu113, pytorch3d==0.7.2, Python==3.10.10, cuda-11.2

Hi, we use AU45 to describe the degree of eye blinking in our implementation. You can modify the code at about line 485 in nerf_triplane/provider.py to enable the original eye blinking representation from RAD-NeRF that is no need for AU45. However, the performance may drop a bit, especially for the eye area.

Thanks for your reply. And, When I run the main.py, I got an error:

Traceback (most recent call last):
  File "/workspace/baiyixuan/ER-NeRF/raymarching/raymarching.py", line 10, in <module>
    import _raymarching_face as _backend
ModuleNotFoundError: No module named '_raymarching_face'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1808, in _run_ninja_build
    subprocess.run(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workspace/baiyixuan/ER-NeRF/main.py", line 6, in <module>
    from nerf_triplane.network import NeRFNetwork
  File "/workspace/baiyixuan/ER-NeRF/nerf_triplane/network.py", line 6, in <module>
    from .renderer import NeRFRenderer
  File "/workspace/baiyixuan/ER-NeRF/nerf_triplane/renderer.py", line 10, in <module>
    import raymarching
  File "/workspace/baiyixuan/ER-NeRF/raymarching/__init__.py", line 1, in <module>
    from .raymarching import *
  File "/workspace/baiyixuan/ER-NeRF/raymarching/raymarching.py", line 12, in <module>
    from .backend import _backend
  File "/workspace/baiyixuan/ER-NeRF/raymarching/backend.py", line 31, in <module>
    _backend = load(name='_raymarching_face',
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1202, in load
    return _jit_compile(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1425, in _jit_compile
    _write_ninja_file_and_build_library(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1537, in _write_ninja_file_and_build_library
    _run_ninja_build(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1824, in _run_ninja_build
    raise RuntimeError(message) from e
RuntimeError: Error building extension '_raymarching_face': [1/1] c++ raymarching.cuda.o bindings.o -shared -L/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/lib -lc10 -lc10_cuda -ltorch_cpu -ltorch_cuda_cu -ltorch_cuda_cpp -ltorch -ltorch_python -L/workspace/miniconda3/envs/ernerf/lib64 -lcudart -o _raymarching_face.so
FAILED: _raymarching_face.so 
c++ raymarching.cuda.o bindings.o -shared -L/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/lib -lc10 -lc10_cuda -ltorch_cpu -ltorch_cuda_cu -ltorch_cuda_cpp -ltorch -ltorch_python -L/workspace/miniconda3/envs/ernerf/lib64 -lcudart -o _raymarching_face.so
/usr/bin/ld: cannot find -lcudart: No such file or directory
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

It looks like an error when do orch.utils.cpp_extension.load. My env is: Ubuntu22.04, gcc-9, g++-9, cuda11.3, python3.10, torch1.12.1, and the pytorch3d is installed.

Sorry, my bad, I have solved the problem.

how did you solve this problem?

what is your env?

#3 (comment)

Hi, we use AU45 to describe the degree of eye blinking in our implementation. You can modify the code at about line 485 in nerf_triplane/provider.py to enable the original eye blinking representation from RAD-NeRF that is no need for AU45. However, the performance may drop a bit, especially for the eye area.

Thanks for your reply. And, When I run the main.py, I got an error:

Traceback (most recent call last):
  File "/workspace/baiyixuan/ER-NeRF/raymarching/raymarching.py", line 10, in <module>
    import _raymarching_face as _backend
ModuleNotFoundError: No module named '_raymarching_face'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1808, in _run_ninja_build
    subprocess.run(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workspace/baiyixuan/ER-NeRF/main.py", line 6, in <module>
    from nerf_triplane.network import NeRFNetwork
  File "/workspace/baiyixuan/ER-NeRF/nerf_triplane/network.py", line 6, in <module>
    from .renderer import NeRFRenderer
  File "/workspace/baiyixuan/ER-NeRF/nerf_triplane/renderer.py", line 10, in <module>
    import raymarching
  File "/workspace/baiyixuan/ER-NeRF/raymarching/__init__.py", line 1, in <module>
    from .raymarching import *
  File "/workspace/baiyixuan/ER-NeRF/raymarching/raymarching.py", line 12, in <module>
    from .backend import _backend
  File "/workspace/baiyixuan/ER-NeRF/raymarching/backend.py", line 31, in <module>
    _backend = load(name='_raymarching_face',
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1202, in load
    return _jit_compile(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1425, in _jit_compile
    _write_ninja_file_and_build_library(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1537, in _write_ninja_file_and_build_library
    _run_ninja_build(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1824, in _run_ninja_build
    raise RuntimeError(message) from e
RuntimeError: Error building extension '_raymarching_face': [1/1] c++ raymarching.cuda.o bindings.o -shared -L/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/lib -lc10 -lc10_cuda -ltorch_cpu -ltorch_cuda_cu -ltorch_cuda_cpp -ltorch -ltorch_python -L/workspace/miniconda3/envs/ernerf/lib64 -lcudart -o _raymarching_face.so
FAILED: _raymarching_face.so 
c++ raymarching.cuda.o bindings.o -shared -L/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/lib -lc10 -lc10_cuda -ltorch_cpu -ltorch_cuda_cu -ltorch_cuda_cpp -ltorch -ltorch_python -L/workspace/miniconda3/envs/ernerf/lib64 -lcudart -o _raymarching_face.so
/usr/bin/ld: cannot find -lcudart: No such file or directory
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

It looks like an error when do orch.utils.cpp_extension.load. My env is: Ubuntu22.04, gcc-9, g++-9, cuda11.3, python3.10, torch1.12.1, and the pytorch3d is installed.

Sorry, my bad, I have solved the problem.

how did you solve this problem?

what is your env?

ubuntu20.04.1, torch==1.12.1+cu113, pytorch3d==0.7.2, Python==3.10.10, cuda-11.2

Hi, we use AU45 to describe the degree of eye blinking in our implementation. You can modify the code at about line 485 in nerf_triplane/provider.py to enable the original eye blinking representation from RAD-NeRF that is no need for AU45. However, the performance may drop a bit, especially for the eye area.

Thanks for your reply. And, When I run the main.py, I got an error:

Traceback (most recent call last):
  File "/workspace/baiyixuan/ER-NeRF/raymarching/raymarching.py", line 10, in <module>
    import _raymarching_face as _backend
ModuleNotFoundError: No module named '_raymarching_face'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1808, in _run_ninja_build
    subprocess.run(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workspace/baiyixuan/ER-NeRF/main.py", line 6, in <module>
    from nerf_triplane.network import NeRFNetwork
  File "/workspace/baiyixuan/ER-NeRF/nerf_triplane/network.py", line 6, in <module>
    from .renderer import NeRFRenderer
  File "/workspace/baiyixuan/ER-NeRF/nerf_triplane/renderer.py", line 10, in <module>
    import raymarching
  File "/workspace/baiyixuan/ER-NeRF/raymarching/__init__.py", line 1, in <module>
    from .raymarching import *
  File "/workspace/baiyixuan/ER-NeRF/raymarching/raymarching.py", line 12, in <module>
    from .backend import _backend
  File "/workspace/baiyixuan/ER-NeRF/raymarching/backend.py", line 31, in <module>
    _backend = load(name='_raymarching_face',
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1202, in load
    return _jit_compile(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1425, in _jit_compile
    _write_ninja_file_and_build_library(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1537, in _write_ninja_file_and_build_library
    _run_ninja_build(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1824, in _run_ninja_build
    raise RuntimeError(message) from e
RuntimeError: Error building extension '_raymarching_face': [1/1] c++ raymarching.cuda.o bindings.o -shared -L/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/lib -lc10 -lc10_cuda -ltorch_cpu -ltorch_cuda_cu -ltorch_cuda_cpp -ltorch -ltorch_python -L/workspace/miniconda3/envs/ernerf/lib64 -lcudart -o _raymarching_face.so
FAILED: _raymarching_face.so 
c++ raymarching.cuda.o bindings.o -shared -L/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/lib -lc10 -lc10_cuda -ltorch_cpu -ltorch_cuda_cu -ltorch_cuda_cpp -ltorch -ltorch_python -L/workspace/miniconda3/envs/ernerf/lib64 -lcudart -o _raymarching_face.so
/usr/bin/ld: cannot find -lcudart: No such file or directory
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

It looks like an error when do orch.utils.cpp_extension.load. My env is: Ubuntu22.04, gcc-9, g++-9, cuda11.3, python3.10, torch1.12.1, and the pytorch3d is installed.

Sorry, my bad, I have solved the problem.

how did you solve this problem?

what is your env?

ubuntu20.04.1, torch==1.12.1+cu113, pytorch3d==0.7.2, Python==3.10.10, cuda-11.2

You can verify that you have added the 'cuda/bin' directory and the 'cuda/lib' directory to your environment variables. And make sure your torch and cuda version environment correspond.