search

Fictionarry / ER-NeRF

[ICCV'23] Efficient Region-Aware Neural Radiance Fields for High-Fidelity Talking Portrait Synthesis
https://fictionarry.github.io/ER-NeRF/
MIT License
1.08k stars 139 forks source link

Is AU45 required ? #3

Closed mortal-Zero closed 1 year ago

mortal-Zero commented 1 year ago

Hi, Thanks for your project.

I want to know is the data/<ID>/au.csv from runing FeatureExtraction in OpenFace required?

If I don't run the FeatureExtraction, can I run the training process normally?

Fictionarry commented 1 year ago

Hi, we use AU45 to describe the degree of eye blinking in our implementation. You can modify the code at about line 485 in nerf_triplane/provider.py to enable the original eye blinking representation from RAD-NeRF that is no need for AU45. However, the performance may drop a bit, especially for the eye area.

mortal-Zero commented 1 year ago

Hi, we use AU45 to describe the degree of eye blinking in our implementation. You can modify the code at about line 485 in nerf_triplane/provider.py to enable the original eye blinking representation from RAD-NeRF that is no need for AU45. However, the performance may drop a bit, especially for the eye area.

Thanks for your reply. And, When I run the main.py, I got an error:

Traceback (most recent call last):
  File "/workspace/baiyixuan/ER-NeRF/raymarching/raymarching.py", line 10, in <module>
    import _raymarching_face as _backend
ModuleNotFoundError: No module named '_raymarching_face'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1808, in _run_ninja_build
    subprocess.run(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workspace/baiyixuan/ER-NeRF/main.py", line 6, in <module>
    from nerf_triplane.network import NeRFNetwork
  File "/workspace/baiyixuan/ER-NeRF/nerf_triplane/network.py", line 6, in <module>
    from .renderer import NeRFRenderer
  File "/workspace/baiyixuan/ER-NeRF/nerf_triplane/renderer.py", line 10, in <module>
    import raymarching
  File "/workspace/baiyixuan/ER-NeRF/raymarching/__init__.py", line 1, in <module>
    from .raymarching import *
  File "/workspace/baiyixuan/ER-NeRF/raymarching/raymarching.py", line 12, in <module>
    from .backend import _backend
  File "/workspace/baiyixuan/ER-NeRF/raymarching/backend.py", line 31, in <module>
    _backend = load(name='_raymarching_face',
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1202, in load
    return _jit_compile(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1425, in _jit_compile
    _write_ninja_file_and_build_library(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1537, in _write_ninja_file_and_build_library
    _run_ninja_build(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1824, in _run_ninja_build
    raise RuntimeError(message) from e
RuntimeError: Error building extension '_raymarching_face': [1/1] c++ raymarching.cuda.o bindings.o -shared -L/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/lib -lc10 -lc10_cuda -ltorch_cpu -ltorch_cuda_cu -ltorch_cuda_cpp -ltorch -ltorch_python -L/workspace/miniconda3/envs/ernerf/lib64 -lcudart -o _raymarching_face.so
FAILED: _raymarching_face.so 
c++ raymarching.cuda.o bindings.o -shared -L/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/lib -lc10 -lc10_cuda -ltorch_cpu -ltorch_cuda_cu -ltorch_cuda_cpp -ltorch -ltorch_python -L/workspace/miniconda3/envs/ernerf/lib64 -lcudart -o _raymarching_face.so
/usr/bin/ld: cannot find -lcudart: No such file or directory
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

It looks like an error when do orch.utils.cpp_extension.load. My env is: Ubuntu22.04, gcc-9, g++-9, cuda11.3, python3.10, torch1.12.1, and the pytorch3d is installed.

mortal-Zero commented 1 year ago

Hi, we use AU45 to describe the degree of eye blinking in our implementation. You can modify the code at about line 485 in nerf_triplane/provider.py to enable the original eye blinking representation from RAD-NeRF that is no need for AU45. However, the performance may drop a bit, especially for the eye area.

Thanks for your reply. And, When I run the main.py, I got an error:

Traceback (most recent call last):
  File "/workspace/baiyixuan/ER-NeRF/raymarching/raymarching.py", line 10, in <module>
    import _raymarching_face as _backend
ModuleNotFoundError: No module named '_raymarching_face'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1808, in _run_ninja_build
    subprocess.run(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workspace/baiyixuan/ER-NeRF/main.py", line 6, in <module>
    from nerf_triplane.network import NeRFNetwork
  File "/workspace/baiyixuan/ER-NeRF/nerf_triplane/network.py", line 6, in <module>
    from .renderer import NeRFRenderer
  File "/workspace/baiyixuan/ER-NeRF/nerf_triplane/renderer.py", line 10, in <module>
    import raymarching
  File "/workspace/baiyixuan/ER-NeRF/raymarching/__init__.py", line 1, in <module>
    from .raymarching import *
  File "/workspace/baiyixuan/ER-NeRF/raymarching/raymarching.py", line 12, in <module>
    from .backend import _backend
  File "/workspace/baiyixuan/ER-NeRF/raymarching/backend.py", line 31, in <module>
    _backend = load(name='_raymarching_face',
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1202, in load
    return _jit_compile(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1425, in _jit_compile
    _write_ninja_file_and_build_library(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1537, in _write_ninja_file_and_build_library
    _run_ninja_build(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1824, in _run_ninja_build
    raise RuntimeError(message) from e
RuntimeError: Error building extension '_raymarching_face': [1/1] c++ raymarching.cuda.o bindings.o -shared -L/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/lib -lc10 -lc10_cuda -ltorch_cpu -ltorch_cuda_cu -ltorch_cuda_cpp -ltorch -ltorch_python -L/workspace/miniconda3/envs/ernerf/lib64 -lcudart -o _raymarching_face.so
FAILED: _raymarching_face.so 
c++ raymarching.cuda.o bindings.o -shared -L/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/lib -lc10 -lc10_cuda -ltorch_cpu -ltorch_cuda_cu -ltorch_cuda_cpp -ltorch -ltorch_python -L/workspace/miniconda3/envs/ernerf/lib64 -lcudart -o _raymarching_face.so
/usr/bin/ld: cannot find -lcudart: No such file or directory
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

It looks like an error when do orch.utils.cpp_extension.load. My env is: Ubuntu22.04, gcc-9, g++-9, cuda11.3, python3.10, torch1.12.1, and the pytorch3d is installed.

Sorry, my bad, I have solved the problem.

shenriver commented 1 year ago

Hi, we use AU45 to describe the degree of eye blinking in our implementation. You can modify the code at about line 485 in nerf_triplane/provider.py to enable the original eye blinking representation from RAD-NeRF that is no need for AU45. However, the performance may drop a bit, especially for the eye area.

Thanks for your reply. And, When I run the main.py, I got an error:

Traceback (most recent call last):
  File "/workspace/baiyixuan/ER-NeRF/raymarching/raymarching.py", line 10, in <module>
    import _raymarching_face as _backend
ModuleNotFoundError: No module named '_raymarching_face'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1808, in _run_ninja_build
    subprocess.run(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workspace/baiyixuan/ER-NeRF/main.py", line 6, in <module>
    from nerf_triplane.network import NeRFNetwork
  File "/workspace/baiyixuan/ER-NeRF/nerf_triplane/network.py", line 6, in <module>
    from .renderer import NeRFRenderer
  File "/workspace/baiyixuan/ER-NeRF/nerf_triplane/renderer.py", line 10, in <module>
    import raymarching
  File "/workspace/baiyixuan/ER-NeRF/raymarching/__init__.py", line 1, in <module>
    from .raymarching import *
  File "/workspace/baiyixuan/ER-NeRF/raymarching/raymarching.py", line 12, in <module>
    from .backend import _backend
  File "/workspace/baiyixuan/ER-NeRF/raymarching/backend.py", line 31, in <module>
    _backend = load(name='_raymarching_face',
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1202, in load
    return _jit_compile(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1425, in _jit_compile
    _write_ninja_file_and_build_library(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1537, in _write_ninja_file_and_build_library
    _run_ninja_build(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1824, in _run_ninja_build
    raise RuntimeError(message) from e
RuntimeError: Error building extension '_raymarching_face': [1/1] c++ raymarching.cuda.o bindings.o -shared -L/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/lib -lc10 -lc10_cuda -ltorch_cpu -ltorch_cuda_cu -ltorch_cuda_cpp -ltorch -ltorch_python -L/workspace/miniconda3/envs/ernerf/lib64 -lcudart -o _raymarching_face.so
FAILED: _raymarching_face.so 
c++ raymarching.cuda.o bindings.o -shared -L/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/lib -lc10 -lc10_cuda -ltorch_cpu -ltorch_cuda_cu -ltorch_cuda_cpp -ltorch -ltorch_python -L/workspace/miniconda3/envs/ernerf/lib64 -lcudart -o _raymarching_face.so
/usr/bin/ld: cannot find -lcudart: No such file or directory
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

It looks like an error when do orch.utils.cpp_extension.load. My env is: Ubuntu22.04, gcc-9, g++-9, cuda11.3, python3.10, torch1.12.1, and the pytorch3d is installed.

Sorry, my bad, I have solved the problem.

how did you solve this problem?

mortal-Zero commented 1 year ago

Hi, we use AU45 to describe the degree of eye blinking in our implementation. You can modify the code at about line 485 in nerf_triplane/provider.py to enable the original eye blinking representation from RAD-NeRF that is no need for AU45. However, the performance may drop a bit, especially for the eye area.

Thanks for your reply. And, When I run the main.py, I got an error:

Traceback (most recent call last):
  File "/workspace/baiyixuan/ER-NeRF/raymarching/raymarching.py", line 10, in <module>
    import _raymarching_face as _backend
ModuleNotFoundError: No module named '_raymarching_face'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1808, in _run_ninja_build
    subprocess.run(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workspace/baiyixuan/ER-NeRF/main.py", line 6, in <module>
    from nerf_triplane.network import NeRFNetwork
  File "/workspace/baiyixuan/ER-NeRF/nerf_triplane/network.py", line 6, in <module>
    from .renderer import NeRFRenderer
  File "/workspace/baiyixuan/ER-NeRF/nerf_triplane/renderer.py", line 10, in <module>
    import raymarching
  File "/workspace/baiyixuan/ER-NeRF/raymarching/__init__.py", line 1, in <module>
    from .raymarching import *
  File "/workspace/baiyixuan/ER-NeRF/raymarching/raymarching.py", line 12, in <module>
    from .backend import _backend
  File "/workspace/baiyixuan/ER-NeRF/raymarching/backend.py", line 31, in <module>
    _backend = load(name='_raymarching_face',
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1202, in load
    return _jit_compile(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1425, in _jit_compile
    _write_ninja_file_and_build_library(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1537, in _write_ninja_file_and_build_library
    _run_ninja_build(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1824, in _run_ninja_build
    raise RuntimeError(message) from e
RuntimeError: Error building extension '_raymarching_face': [1/1] c++ raymarching.cuda.o bindings.o -shared -L/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/lib -lc10 -lc10_cuda -ltorch_cpu -ltorch_cuda_cu -ltorch_cuda_cpp -ltorch -ltorch_python -L/workspace/miniconda3/envs/ernerf/lib64 -lcudart -o _raymarching_face.so
FAILED: _raymarching_face.so 
c++ raymarching.cuda.o bindings.o -shared -L/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/lib -lc10 -lc10_cuda -ltorch_cpu -ltorch_cuda_cu -ltorch_cuda_cpp -ltorch -ltorch_python -L/workspace/miniconda3/envs/ernerf/lib64 -lcudart -o _raymarching_face.so
/usr/bin/ld: cannot find -lcudart: No such file or directory
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

It looks like an error when do orch.utils.cpp_extension.load. My env is: Ubuntu22.04, gcc-9, g++-9, cuda11.3, python3.10, torch1.12.1, and the pytorch3d is installed.

Sorry, my bad, I have solved the problem.

how did you solve this problem?

what is your env?

shenriver commented 1 year ago

ubuntu20.04.1, torch==1.12.1+cu113, pytorch3d==0.7.2, Python==3.10.10, cuda-11.2

Hi, we use AU45 to describe the degree of eye blinking in our implementation. You can modify the code at about line 485 in nerf_triplane/provider.py to enable the original eye blinking representation from RAD-NeRF that is no need for AU45. However, the performance may drop a bit, especially for the eye area.

Thanks for your reply. And, When I run the main.py, I got an error:

Traceback (most recent call last):
  File "/workspace/baiyixuan/ER-NeRF/raymarching/raymarching.py", line 10, in <module>
    import _raymarching_face as _backend
ModuleNotFoundError: No module named '_raymarching_face'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1808, in _run_ninja_build
    subprocess.run(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workspace/baiyixuan/ER-NeRF/main.py", line 6, in <module>
    from nerf_triplane.network import NeRFNetwork
  File "/workspace/baiyixuan/ER-NeRF/nerf_triplane/network.py", line 6, in <module>
    from .renderer import NeRFRenderer
  File "/workspace/baiyixuan/ER-NeRF/nerf_triplane/renderer.py", line 10, in <module>
    import raymarching
  File "/workspace/baiyixuan/ER-NeRF/raymarching/__init__.py", line 1, in <module>
    from .raymarching import *
  File "/workspace/baiyixuan/ER-NeRF/raymarching/raymarching.py", line 12, in <module>
    from .backend import _backend
  File "/workspace/baiyixuan/ER-NeRF/raymarching/backend.py", line 31, in <module>
    _backend = load(name='_raymarching_face',
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1202, in load
    return _jit_compile(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1425, in _jit_compile
    _write_ninja_file_and_build_library(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1537, in _write_ninja_file_and_build_library
    _run_ninja_build(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1824, in _run_ninja_build
    raise RuntimeError(message) from e
RuntimeError: Error building extension '_raymarching_face': [1/1] c++ raymarching.cuda.o bindings.o -shared -L/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/lib -lc10 -lc10_cuda -ltorch_cpu -ltorch_cuda_cu -ltorch_cuda_cpp -ltorch -ltorch_python -L/workspace/miniconda3/envs/ernerf/lib64 -lcudart -o _raymarching_face.so
FAILED: _raymarching_face.so 
c++ raymarching.cuda.o bindings.o -shared -L/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/lib -lc10 -lc10_cuda -ltorch_cpu -ltorch_cuda_cu -ltorch_cuda_cpp -ltorch -ltorch_python -L/workspace/miniconda3/envs/ernerf/lib64 -lcudart -o _raymarching_face.so
/usr/bin/ld: cannot find -lcudart: No such file or directory
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

It looks like an error when do orch.utils.cpp_extension.load. My env is: Ubuntu22.04, gcc-9, g++-9, cuda11.3, python3.10, torch1.12.1, and the pytorch3d is installed.

Sorry, my bad, I have solved the problem.

how did you solve this problem?

what is your env?

https://github.com/Fictionarry/ER-NeRF/issues/3#issuecomment-1691384370

Hi, we use AU45 to describe the degree of eye blinking in our implementation. You can modify the code at about line 485 in nerf_triplane/provider.py to enable the original eye blinking representation from RAD-NeRF that is no need for AU45. However, the performance may drop a bit, especially for the eye area.

Thanks for your reply. And, When I run the main.py, I got an error:

Traceback (most recent call last):
  File "/workspace/baiyixuan/ER-NeRF/raymarching/raymarching.py", line 10, in <module>
    import _raymarching_face as _backend
ModuleNotFoundError: No module named '_raymarching_face'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1808, in _run_ninja_build
    subprocess.run(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workspace/baiyixuan/ER-NeRF/main.py", line 6, in <module>
    from nerf_triplane.network import NeRFNetwork
  File "/workspace/baiyixuan/ER-NeRF/nerf_triplane/network.py", line 6, in <module>
    from .renderer import NeRFRenderer
  File "/workspace/baiyixuan/ER-NeRF/nerf_triplane/renderer.py", line 10, in <module>
    import raymarching
  File "/workspace/baiyixuan/ER-NeRF/raymarching/__init__.py", line 1, in <module>
    from .raymarching import *
  File "/workspace/baiyixuan/ER-NeRF/raymarching/raymarching.py", line 12, in <module>
    from .backend import _backend
  File "/workspace/baiyixuan/ER-NeRF/raymarching/backend.py", line 31, in <module>
    _backend = load(name='_raymarching_face',
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1202, in load
    return _jit_compile(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1425, in _jit_compile
    _write_ninja_file_and_build_library(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1537, in _write_ninja_file_and_build_library
    _run_ninja_build(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1824, in _run_ninja_build
    raise RuntimeError(message) from e
RuntimeError: Error building extension '_raymarching_face': [1/1] c++ raymarching.cuda.o bindings.o -shared -L/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/lib -lc10 -lc10_cuda -ltorch_cpu -ltorch_cuda_cu -ltorch_cuda_cpp -ltorch -ltorch_python -L/workspace/miniconda3/envs/ernerf/lib64 -lcudart -o _raymarching_face.so
FAILED: _raymarching_face.so 
c++ raymarching.cuda.o bindings.o -shared -L/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/lib -lc10 -lc10_cuda -ltorch_cpu -ltorch_cuda_cu -ltorch_cuda_cpp -ltorch -ltorch_python -L/workspace/miniconda3/envs/ernerf/lib64 -lcudart -o _raymarching_face.so
/usr/bin/ld: cannot find -lcudart: No such file or directory
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

It looks like an error when do orch.utils.cpp_extension.load. My env is: Ubuntu22.04, gcc-9, g++-9, cuda11.3, python3.10, torch1.12.1, and the pytorch3d is installed.

Sorry, my bad, I have solved the problem.

how did you solve this problem?

what is your env?

ubuntu20.04.1, torch==1.12.1+cu113, pytorch3d==0.7.2, Python==3.10.10, cuda-11.2

Hi, we use AU45 to describe the degree of eye blinking in our implementation. You can modify the code at about line 485 in nerf_triplane/provider.py to enable the original eye blinking representation from RAD-NeRF that is no need for AU45. However, the performance may drop a bit, especially for the eye area.

Thanks for your reply. And, When I run the main.py, I got an error:

Traceback (most recent call last):
  File "/workspace/baiyixuan/ER-NeRF/raymarching/raymarching.py", line 10, in <module>
    import _raymarching_face as _backend
ModuleNotFoundError: No module named '_raymarching_face'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1808, in _run_ninja_build
    subprocess.run(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workspace/baiyixuan/ER-NeRF/main.py", line 6, in <module>
    from nerf_triplane.network import NeRFNetwork
  File "/workspace/baiyixuan/ER-NeRF/nerf_triplane/network.py", line 6, in <module>
    from .renderer import NeRFRenderer
  File "/workspace/baiyixuan/ER-NeRF/nerf_triplane/renderer.py", line 10, in <module>
    import raymarching
  File "/workspace/baiyixuan/ER-NeRF/raymarching/__init__.py", line 1, in <module>
    from .raymarching import *
  File "/workspace/baiyixuan/ER-NeRF/raymarching/raymarching.py", line 12, in <module>
    from .backend import _backend
  File "/workspace/baiyixuan/ER-NeRF/raymarching/backend.py", line 31, in <module>
    _backend = load(name='_raymarching_face',
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1202, in load
    return _jit_compile(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1425, in _jit_compile
    _write_ninja_file_and_build_library(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1537, in _write_ninja_file_and_build_library
    _run_ninja_build(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1824, in _run_ninja_build
    raise RuntimeError(message) from e
RuntimeError: Error building extension '_raymarching_face': [1/1] c++ raymarching.cuda.o bindings.o -shared -L/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/lib -lc10 -lc10_cuda -ltorch_cpu -ltorch_cuda_cu -ltorch_cuda_cpp -ltorch -ltorch_python -L/workspace/miniconda3/envs/ernerf/lib64 -lcudart -o _raymarching_face.so
FAILED: _raymarching_face.so 
c++ raymarching.cuda.o bindings.o -shared -L/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/lib -lc10 -lc10_cuda -ltorch_cpu -ltorch_cuda_cu -ltorch_cuda_cpp -ltorch -ltorch_python -L/workspace/miniconda3/envs/ernerf/lib64 -lcudart -o _raymarching_face.so
/usr/bin/ld: cannot find -lcudart: No such file or directory
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

It looks like an error when do orch.utils.cpp_extension.load. My env is: Ubuntu22.04, gcc-9, g++-9, cuda11.3, python3.10, torch1.12.1, and the pytorch3d is installed.

Sorry, my bad, I have solved the problem.

how did you solve this problem?

what is your env?

ubuntu20.04.1, torch==1.12.1+cu113, pytorch3d==0.7.2, Python==3.10.10, cuda-11.2

mortal-Zero commented 1 year ago

ubuntu20.04.1, torch==1.12.1+cu113, pytorch3d==0.7.2, Python==3.10.10, cuda-11.2

Hi, we use AU45 to describe the degree of eye blinking in our implementation. You can modify the code at about line 485 in nerf_triplane/provider.py to enable the original eye blinking representation from RAD-NeRF that is no need for AU45. However, the performance may drop a bit, especially for the eye area.

Thanks for your reply. And, When I run the main.py, I got an error:

Traceback (most recent call last):
  File "/workspace/baiyixuan/ER-NeRF/raymarching/raymarching.py", line 10, in <module>
    import _raymarching_face as _backend
ModuleNotFoundError: No module named '_raymarching_face'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1808, in _run_ninja_build
    subprocess.run(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workspace/baiyixuan/ER-NeRF/main.py", line 6, in <module>
    from nerf_triplane.network import NeRFNetwork
  File "/workspace/baiyixuan/ER-NeRF/nerf_triplane/network.py", line 6, in <module>
    from .renderer import NeRFRenderer
  File "/workspace/baiyixuan/ER-NeRF/nerf_triplane/renderer.py", line 10, in <module>
    import raymarching
  File "/workspace/baiyixuan/ER-NeRF/raymarching/__init__.py", line 1, in <module>
    from .raymarching import *
  File "/workspace/baiyixuan/ER-NeRF/raymarching/raymarching.py", line 12, in <module>
    from .backend import _backend
  File "/workspace/baiyixuan/ER-NeRF/raymarching/backend.py", line 31, in <module>
    _backend = load(name='_raymarching_face',
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1202, in load
    return _jit_compile(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1425, in _jit_compile
    _write_ninja_file_and_build_library(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1537, in _write_ninja_file_and_build_library
    _run_ninja_build(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1824, in _run_ninja_build
    raise RuntimeError(message) from e
RuntimeError: Error building extension '_raymarching_face': [1/1] c++ raymarching.cuda.o bindings.o -shared -L/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/lib -lc10 -lc10_cuda -ltorch_cpu -ltorch_cuda_cu -ltorch_cuda_cpp -ltorch -ltorch_python -L/workspace/miniconda3/envs/ernerf/lib64 -lcudart -o _raymarching_face.so
FAILED: _raymarching_face.so 
c++ raymarching.cuda.o bindings.o -shared -L/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/lib -lc10 -lc10_cuda -ltorch_cpu -ltorch_cuda_cu -ltorch_cuda_cpp -ltorch -ltorch_python -L/workspace/miniconda3/envs/ernerf/lib64 -lcudart -o _raymarching_face.so
/usr/bin/ld: cannot find -lcudart: No such file or directory
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

It looks like an error when do orch.utils.cpp_extension.load. My env is: Ubuntu22.04, gcc-9, g++-9, cuda11.3, python3.10, torch1.12.1, and the pytorch3d is installed.

Sorry, my bad, I have solved the problem.

how did you solve this problem?

what is your env?

#3 (comment)

Hi, we use AU45 to describe the degree of eye blinking in our implementation. You can modify the code at about line 485 in nerf_triplane/provider.py to enable the original eye blinking representation from RAD-NeRF that is no need for AU45. However, the performance may drop a bit, especially for the eye area.

Thanks for your reply. And, When I run the main.py, I got an error:

Traceback (most recent call last):
  File "/workspace/baiyixuan/ER-NeRF/raymarching/raymarching.py", line 10, in <module>
    import _raymarching_face as _backend
ModuleNotFoundError: No module named '_raymarching_face'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1808, in _run_ninja_build
    subprocess.run(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workspace/baiyixuan/ER-NeRF/main.py", line 6, in <module>
    from nerf_triplane.network import NeRFNetwork
  File "/workspace/baiyixuan/ER-NeRF/nerf_triplane/network.py", line 6, in <module>
    from .renderer import NeRFRenderer
  File "/workspace/baiyixuan/ER-NeRF/nerf_triplane/renderer.py", line 10, in <module>
    import raymarching
  File "/workspace/baiyixuan/ER-NeRF/raymarching/__init__.py", line 1, in <module>
    from .raymarching import *
  File "/workspace/baiyixuan/ER-NeRF/raymarching/raymarching.py", line 12, in <module>
    from .backend import _backend
  File "/workspace/baiyixuan/ER-NeRF/raymarching/backend.py", line 31, in <module>
    _backend = load(name='_raymarching_face',
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1202, in load
    return _jit_compile(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1425, in _jit_compile
    _write_ninja_file_and_build_library(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1537, in _write_ninja_file_and_build_library
    _run_ninja_build(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1824, in _run_ninja_build
    raise RuntimeError(message) from e
RuntimeError: Error building extension '_raymarching_face': [1/1] c++ raymarching.cuda.o bindings.o -shared -L/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/lib -lc10 -lc10_cuda -ltorch_cpu -ltorch_cuda_cu -ltorch_cuda_cpp -ltorch -ltorch_python -L/workspace/miniconda3/envs/ernerf/lib64 -lcudart -o _raymarching_face.so
FAILED: _raymarching_face.so 
c++ raymarching.cuda.o bindings.o -shared -L/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/lib -lc10 -lc10_cuda -ltorch_cpu -ltorch_cuda_cu -ltorch_cuda_cpp -ltorch -ltorch_python -L/workspace/miniconda3/envs/ernerf/lib64 -lcudart -o _raymarching_face.so
/usr/bin/ld: cannot find -lcudart: No such file or directory
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

It looks like an error when do orch.utils.cpp_extension.load. My env is: Ubuntu22.04, gcc-9, g++-9, cuda11.3, python3.10, torch1.12.1, and the pytorch3d is installed.

Sorry, my bad, I have solved the problem.

how did you solve this problem?

what is your env?

ubuntu20.04.1, torch==1.12.1+cu113, pytorch3d==0.7.2, Python==3.10.10, cuda-11.2

Hi, we use AU45 to describe the degree of eye blinking in our implementation. You can modify the code at about line 485 in nerf_triplane/provider.py to enable the original eye blinking representation from RAD-NeRF that is no need for AU45. However, the performance may drop a bit, especially for the eye area.

Thanks for your reply. And, When I run the main.py, I got an error:

Traceback (most recent call last):
  File "/workspace/baiyixuan/ER-NeRF/raymarching/raymarching.py", line 10, in <module>
    import _raymarching_face as _backend
ModuleNotFoundError: No module named '_raymarching_face'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1808, in _run_ninja_build
    subprocess.run(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workspace/baiyixuan/ER-NeRF/main.py", line 6, in <module>
    from nerf_triplane.network import NeRFNetwork
  File "/workspace/baiyixuan/ER-NeRF/nerf_triplane/network.py", line 6, in <module>
    from .renderer import NeRFRenderer
  File "/workspace/baiyixuan/ER-NeRF/nerf_triplane/renderer.py", line 10, in <module>
    import raymarching
  File "/workspace/baiyixuan/ER-NeRF/raymarching/__init__.py", line 1, in <module>
    from .raymarching import *
  File "/workspace/baiyixuan/ER-NeRF/raymarching/raymarching.py", line 12, in <module>
    from .backend import _backend
  File "/workspace/baiyixuan/ER-NeRF/raymarching/backend.py", line 31, in <module>
    _backend = load(name='_raymarching_face',
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1202, in load
    return _jit_compile(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1425, in _jit_compile
    _write_ninja_file_and_build_library(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1537, in _write_ninja_file_and_build_library
    _run_ninja_build(
  File "/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1824, in _run_ninja_build
    raise RuntimeError(message) from e
RuntimeError: Error building extension '_raymarching_face': [1/1] c++ raymarching.cuda.o bindings.o -shared -L/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/lib -lc10 -lc10_cuda -ltorch_cpu -ltorch_cuda_cu -ltorch_cuda_cpp -ltorch -ltorch_python -L/workspace/miniconda3/envs/ernerf/lib64 -lcudart -o _raymarching_face.so
FAILED: _raymarching_face.so 
c++ raymarching.cuda.o bindings.o -shared -L/workspace/miniconda3/envs/ernerf/lib/python3.10/site-packages/torch/lib -lc10 -lc10_cuda -ltorch_cpu -ltorch_cuda_cu -ltorch_cuda_cpp -ltorch -ltorch_python -L/workspace/miniconda3/envs/ernerf/lib64 -lcudart -o _raymarching_face.so
/usr/bin/ld: cannot find -lcudart: No such file or directory
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

It looks like an error when do orch.utils.cpp_extension.load. My env is: Ubuntu22.04, gcc-9, g++-9, cuda11.3, python3.10, torch1.12.1, and the pytorch3d is installed.

Sorry, my bad, I have solved the problem.

how did you solve this problem?

what is your env?

ubuntu20.04.1, torch==1.12.1+cu113, pytorch3d==0.7.2, Python==3.10.10, cuda-11.2

You can verify that you have added the 'cuda/bin' directory and the 'cuda/lib' directory to your environment variables. And make sure your torch and cuda version environment correspond.