I got this error. eglInitialize failed. Segmentation fault

[altava-sgp]

I'm using RTX 4090 ( 24 GB VRAM ).

I run this command.

python launch.py --config custom/threestudio-progressive3d/configs/mvdream-progressive3d.yaml --train --gpu 0 system.prompt_processor.prompt="a medieval soldier with metal armor|a medieval soldier with metal armor riding a terracotta wolf" resume="outputs/mvdream-sd21-rescale0.5-shading/a_medieval_soldier_with_metal_armor@20240328-015548/ckpts/last.ckpt"

I got error.

Cannot import custom/Progressive3D module for custom nodes: [Errno 2] No such file or directory: '/home/dreamer/threestudio/custom/Progressive3D/__init__.py'

Import times for custom modules:
   0.0 seconds (IMPORT FAILED): custom/ai-webpost-api
   0.0 seconds (IMPORT FAILED): custom/Progressive3D
   0.0 seconds (IMPORT FAILED): custom/dreamgaussian
   0.0 seconds (IMPORT FAILED): custom/maker-ai-discord-bot-2-pro
   0.1 seconds: custom/threestudio-mvdream
   0.1 seconds: custom/threestudio-meshfitting
   0.1 seconds: custom/threestudio-dreamcraft3D
   0.2 seconds: custom/threestudio-3dgs
   0.9 seconds: custom/threestudio-progressive3d

Global seed set to 0
[INFO] Loading Multiview Diffusion ...
Loading model from config: sd-v2-base.yaml
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is None and using 5 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is 1024 and using 5 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is None and using 5 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is 1024 and using 5 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is None and using 10 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is 1024 and using 10 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is None and using 10 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is 1024 and using 10 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 20 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 1024 and using 20 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 20 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 1024 and using 20 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 20 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 1024 and using 20 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 20 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 1024 and using 20 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 20 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 1024 and using 20 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 20 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 1024 and using 20 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is None and using 10 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is 1024 and using 10 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is None and using 10 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is 1024 and using 10 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is None and using 10 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is 1024 and using 10 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is None and using 5 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is 1024 and using 5 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is None and using 5 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is 1024 and using 5 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is None and using 5 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is 1024 and using 5 heads.
making attention of type 'vanilla-xformers' with 512 in_channels
building MemoryEfficientAttnBlock with 512 in_channels...
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
making attention of type 'vanilla-xformers' with 512 in_channels
building MemoryEfficientAttnBlock with 512 in_channels...
Loading model from cache file: /home/dreamer/.cache/huggingface/hub/models--MVDream--MVDream/snapshots/d14ac9d78c48c266005729f2d5633f6c265da467/sd-v2.1-base-4view.pt
[INFO] Loaded Multiview Diffusion!
[INFO] Using prompt [a medieval soldier with metal armor] and negative prompt [ugly, bad anatomy, blurry, pixelated obscure, unnatural colors, poor lighting, dull, and unclear, cropped, lowres, low quality, artifacts, duplicate, morbid, mutilated, poorly drawn face, deformed, dehydrated, bad proportions]
[INFO] Using view-dependent prompts [side]:[a medieval soldier with metal armor, side view] [front]:[a medieval soldier with metal armor, front view] [back]:[a medieval soldier with metal armor, back view] [overhead]:[a medieval soldier with metal armor, overhead view]
[INFO] Using prompt [a medieval soldier with metal armor riding a terracotta wolf] and negative prompt [ugly, bad anatomy, blurry, pixelated obscure, unnatural colors, poor lighting, dull, and unclear, cropped, lowres, low quality, artifacts, duplicate, morbid, mutilated, poorly drawn face, deformed, dehydrated, bad proportions]
[INFO] Using view-dependent prompts [side]:[a medieval soldier with metal armor riding a terracotta wolf, side view] [front]:[a medieval soldier with metal armor riding a terracotta wolf, front view] [back]:[a medieval soldier with metal armor riding a terracotta wolf, back view] [overhead]:[a medieval soldier with metal armor riding a terracotta wolf, overhead view]
[INFO] ModelCheckpoint(save_last=True, save_top_k=-1, monitor=None) will duplicate the last checkpoint saved.
[INFO] Using 16bit Automatic Mixed Precision (AMP)
[INFO] GPU available: True (cuda), used: True
[INFO] TPU available: False, using: 0 TPU cores
[INFO] IPU available: False, using: 0 IPUs
[INFO] HPU available: False, using: 0 HPUs
[INFO] You are using a CUDA device ('NVIDIA GeForce RTX 4090') that has Tensor Cores. To properly utilize them, you should set `torch.set_float32_matmul_precision('medium' | 'high')` which will trade-off precision for performance. For more details, read https://pytorch.org/docs/stable/generated/torch.set_float32_matmul_precision.html#torch.set_float32_matmul_precision
[INFO] Restoring states from the checkpoint path at outputs/mvdream-sd21-rescale0.5-shading/a_medieval_soldier_with_metal_armor@20240328-015548/ckpts/last.ckpt
/home/dreamer/.local/lib/python3.10/site-packages/pytorch_lightning/trainer/call.py:250: UserWarning: Be aware that when using `ckpt_path`, callbacks used to create the checkpoint need to be provided during `Trainer` instantiation. Please add the following callbacks: ["ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 10000, 'every_n_epochs': 0, 'train_time_interval': None}"].
  rank_zero_warn(
[INFO] LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
[INFO] 
  | Name       | Type                           | Params
--------------------------------------------------------------
0 | geometry   | ImplicitVolume                 | 12.6 M
1 | material   | DiffuseWithPointLightMaterial  | 0     
2 | background | NeuralEnvironmentMapBackground | 448   
3 | renderer   | NeRFVolumeRenderer             | 0     
--------------------------------------------------------------
12.6 M    Trainable params
0         Non-trainable params
12.6 M    Total params
50.419    Total estimated model params size (MB)
[INFO] Validation results will be saved to outputs/mvdream-progressive3d/a_medieval_soldier_with_metal_armor|a_medieval_soldier_with_metal_armor_riding_a_terracotta_wolf@20240328-044123/save
[FINISH] outputs/mvdream-progressive3d/a_medieval_soldier_with_metal_armor|a_medieval_soldier_with_metal_armor_riding_a_terracotta_wolf@20240328-044123/save
[Open3D INFO] EGL headless mode enabled.
FEngine (64 bits) created at 0x5630e9bcec00 (threading is enabled)
eglInitialize failed
Segmentation fault

What can I do ?

[cxh0519]

Segmentation fault is caused by the open3d and does not influence the generation process. The running error seems caused by the IMPORT FAILED of our progressive3d extension, please check the importing stage.

[altava-sgp]

@cxh0519 There is no save folder. C:\workspace\threestudio\outputs\mvdream-progressive3d\a_medieval_soldier_with_metal_armora_medieval_soldier_with_metal_armor_riding_a_terracotta_wolf@20240328-044123

And I can't find "importing stage". Where is it ?

[cxh0519]

You might firstly run python launch.py --config custom/threestudio-mvdream/configs/mvdream-sd21-shading.yaml --train --gpu 0 system.prompt_processor.prompt="a medieval soldier with metal armor" And you can obtain a generated ckpt in outputs/mvdream-sd21-shading/name_of_your_generated_results/ckpts/last.ckpt And then run python launch.py --config custom/threestudio-progressive3d/configs/mvdream-progressive3d.yaml --train --gpu 0 system.prompt_processor.prompt="a medieval soldier with metal armor|a medieval soldier with metal armor riding a terracotta wolf" resume="outputs/mvdream-sd21-shading/name_of_your_generated_results/ckpts/last.ckpt"

[altava-sgp]

@cxh0519 I already run the first command you said. python launch.py --config custom/threestudio-mvdream/configs/mvdream-sd21-shading.yaml --train --gpu 0 system.prompt_processor.prompt="a medieval soldier with metal armor I got the ckpt files already.

You can ignore this log. This is the log for especially me.

Cannot import custom/Progressive3D module for custom nodes: [Errno 2] No such file or directory: '/home/dreamer/threestudio/custom/Progressive3D/__init__.py'

Import times for custom modules:
   0.0 seconds (IMPORT FAILED): custom/ai-webpost-api
   0.0 seconds (IMPORT FAILED): custom/Progressive3D
   0.0 seconds (IMPORT FAILED): custom/dreamgaussian
   0.0 seconds (IMPORT FAILED): custom/maker-ai-discord-bot-2-pro

Import success

   0.1 seconds: custom/threestudio-mvdream
   0.1 seconds: custom/threestudio-meshfitting
   0.1 seconds: custom/threestudio-dreamcraft3D
   0.2 seconds: custom/threestudio-3dgs
   0.9 seconds: custom/threestudio-progressive3d

ckpt file is here. C:\workspace\threestudio\outputs\mvdream-sd21-rescale0.5-shading\a_medieval_soldier_with_metal_armor@20240328-015548\ckpts

• not mvdream-sd21-shading but mvdream-sd21-rescale0.5-shading ( which is right ? )

Can you guess the issue ?

[cxh0519]

Sorry for the misunderstanding. The segment fault might be caused by eglInitialize failed in Open3D. Please check the installation of open3d, driven version, or other reasons.

[Kirito-Ausna]

Hi, I met the same issue. Have you solved it? Could you share your fix with me? Many thanks!

[altava-sgp]

@Kirito-Ausna I barely got a solution. But I don't remember. In fact, I no longer use it because the results are not good. 🥲

[Kirito-Ausna]

@altava-sgp Thanks for your reply! I am new to this field, would you like to share the best models you have tested according to your experience? I have tried some methods now like DreamFusion-SD, Magic-3D, ProlificDreamer, HiFA,etc but the results are all not good (worse than their demos). Many Thanks!

[Kirito-Ausna]

As for now, MVDream is the best for me (I tested MVDream just in this repo)

[Kirito-Ausna]

I need to find a decent baseline for text-based NeRF/Gaussian Splatting editing in Score-Distillation Framework (just like threestudio). Do you have any suggestions? Thank you very much! Sorry for any trouble.

[altava-sgp]

@Kirito-Ausna In my case this was the best. https://github.com/DSaurus/threestudio-dreamcraft3D ( feat. MVDream )

[Kirito-Ausna]

@altava-sgp Thanks for your sharing! The original implementation of 'view-dependent diffusion model' in dreamcraft3D is Zero123 ( for both threestudio version and original version), not MVDream. Did you modify this by yourself? Thanks again for your generous sharing!

[altava-sgp]

@Kirito-Ausna I meant that DreamCraft3D generates the 3d model with caption. I didn't modify about the detailed techniques. I remember that the feature with caption for generating is from MVdream. Anyway good luck ! 👍