soohyoen
commented
2 days ago @haoyuhsu
An additional question is that the error below appears, but why is this error occurring?
python edit_scene.py \
--source_path datasets/garden_large \
--model_path output/garden_large/ \
--gaussians_ckpt_path output/garden_large/coarse/sugarcoarse_3Dgs15000_densityestim02_sdfnorm02/22000.pt \
--custom_traj_name transforms_001 \
--anchor_frame_idx 0 \
--scene_scale 2.67 \
--edit_text "Drop 5 basketballs on the table." \
--scene_mesh_path datasets/garden_large/mesh/mesh.obj \
--blender_config_name blender_cfg_rigid_body_simulation.json \
--blender_output_dir_name blender_cfg_rigid_body_simulation \
--render_type MULTI_VIEW \
--deva_dino_threshold 0.45 \
--is_uv_mesh
2024-12-01 13:26:31.150854: I tensorflow/core/util/port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-12-01 13:26:31.154187: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2024-12-01 13:26:31.160906: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2024-12-01 13:26:31.177856: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:485] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-12-01 13:26:31.206143: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:8454] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-12-01 13:26:31.214027: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1452] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-12-01 13:26:31.235266: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-12-01 13:26:33.042132: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
Detectron v2 is not installed
(using cache) *** OpenAI API call took 0.00s ***
########################################
## "planner" generated code
########################################
# Query: Drop 5 basketballs on the table..
table_obj = detect_object(scene, 'table')
basketball = retrieve_asset(scene, 'basketball')
for i in range(5):
pos = sample_point_above_object(scene, table_obj)
basketball_copy = make_copy(basketball)
basketball_copy = translate_object(basketball_copy, pos)
basketball_copy = allow_physics(basketball_copy)
insert_object(scene, basketball_copy)
Loading the coarse SuGaR model from path
output/garden_large/coarse/sugarcoarse_3Dgs15000_densityestim02_sdfnorm02/22000.pt...
Loading custom cameras: 100%|██████████████████████████| 150/150 [00:00<00:00, 218.13it/s]
Loading the coarse SuGaR model from path
output/garden_large/coarse/sugarcoarse_3Dgs15000_densityestim02_sdfnorm02/22000.pt...
Rendering progress: 0it [00:00, ?it/s]/opt/conda/envs/autovfx_real/lib/python3.10/site-packages/torch/functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:3483.)
return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined]
Rendering progress: 150it [02:47, 1.12s/it]
Detecting object: table
Largest object instance id: 3221540
Extracting object: table with id 3221540
Extracting object: table with id 3221540 from view 0
Unprojecting...: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:04<00:00, 4.87s/it]
===== RATIO: 0.05 =====
Number of triangles that are close to the object: tensor(1973417, device='cuda:0')
Number of gaussians that are close to the object: tensor(3110894, device='cuda:0')
Rendering progress: 150it [00:00, 2450.42it/s]
Total missed pixels: 882234
===== RATIO: 0.1 =====
Number of triangles that are close to the object: tensor(1973417, device='cuda:0')
Number of gaussians that are close to the object: tensor(3110894, device='cuda:0')
Rendering progress: 150it [00:00, 852.10it/s]
Total missed pixels: 882234
===== RATIO: 0.2 =====
Number of triangles that are close to the object: tensor(1973417, device='cuda:0')
Number of gaussians that are close to the object: tensor(3110894, device='cuda:0')
Rendering progress: 150it [00:00, 4061.39it/s]
Total missed pixels: 882234
===== RATIO: 0.25 =====
Number of triangles that are close to the object: tensor(1973417, device='cuda:0')
Number of gaussians that are close to the object: tensor(3110894, device='cuda:0')
Rendering progress: 150it [00:00, 4332.36it/s]
Total missed pixels: 882234
===== RATIO: 0.3 =====
Number of triangles that are close to the object: tensor(1973417, device='cuda:0')
Number of gaussians that are close to the object: tensor(3110894, device='cuda:0')
Rendering progress: 150it [00:00, 3906.09it/s]
Total missed pixels: 882234
===== RATIO: 0.32 =====
Number of triangles that are close to the object: tensor(1973417, device='cuda:0')
Number of gaussians that are close to the object: tensor(3110894, device='cuda:0')
Rendering progress: 150it [00:00, 3984.51it/s]
Total missed pixels: 882234
===== RATIO: 0.34 =====
Number of triangles that are close to the object: tensor(1973417, device='cuda:0')
Number of gaussians that are close to the object: tensor(3110894, device='cuda:0')
Rendering progress: 150it [00:00, 4262.53it/s]
Total missed pixels: 882234
===== RATIO: 0.36 =====
Number of triangles that are close to the object: tensor(1973417, device='cuda:0')
Number of gaussians that are close to the object: tensor(3110894, device='cuda:0')
Rendering progress: 150it [00:00, 3440.12it/s]
Total missed pixels: 882234
===== RATIO: 0.38 =====
Number of triangles that are close to the object: tensor(1973417, device='cuda:0')
Number of gaussians that are close to the object: tensor(3110894, device='cuda:0')
Rendering progress: 150it [00:00, 3472.32it/s]
Total missed pixels: 882234
===== RATIO: 0.4 =====
Number of triangles that are close to the object: tensor(1973417, device='cuda:0')
Number of gaussians that are close to the object: tensor(3110894, device='cuda:0')
Rendering progress: 150it [00:00, 4445.94it/s]
Total missed pixels: 882234
===== RATIO: 0.42 =====
Number of triangles that are close to the object: tensor(1973417, device='cuda:0')
Number of gaussians that are close to the object: tensor(3110894, device='cuda:0')
Rendering progress: 150it [00:00, 4315.72it/s]
Total missed pixels: 882234
===== RATIO: 0.44 =====
Number of triangles that are close to the object: tensor(1973417, device='cuda:0')
Number of gaussians that are close to the object: tensor(3110894, device='cuda:0')
Rendering progress: 150it [00:00, 536.18it/s]
Total missed pixels: 882234
===== RATIO: 0.46 =====
Number of triangles that are close to the object: tensor(1973417, device='cuda:0')
Number of gaussians that are close to the object: tensor(3110894, device='cuda:0')
Rendering progress: 150it [00:00, 4145.03it/s]
Total missed pixels: 882234
===== RATIO: 0.48 =====
Number of triangles that are close to the object: tensor(1973417, device='cuda:0')
Number of gaussians that are close to the object: tensor(3110894, device='cuda:0')
Rendering progress: 150it [00:00, 3341.42it/s]
Total missed pixels: 882234
===== RATIO: 0.5 =====
Number of triangles that are close to the object: tensor(1973417, device='cuda:0')
Number of gaussians that are close to the object: tensor(3110894, device='cuda:0')
Rendering progress: 150it [00:00, 4272.69it/s]
Total missed pixels: 882234
===== RATIO: 0.55 =====
Number of triangles that are close to the object: tensor(1973417, device='cuda:0')
Number of gaussians that are close to the object: tensor(3110894, device='cuda:0')
Rendering progress: 150it [00:00, 4138.79it/s]
Total missed pixels: 882234
===== RATIO: 0.6 =====
Number of triangles that are close to the object: tensor(1973417, device='cuda:0')
Number of gaussians that are close to the object: tensor(3110894, device='cuda:0')
Rendering progress: 150it [00:00, 3270.41it/s]
Total missed pixels: 882234
===== RATIO: 0.65 =====
Number of triangles that are close to the object: tensor(1973417, device='cuda:0')
Number of gaussians that are close to the object: tensor(3110894, device='cuda:0')
Rendering progress: 150it [00:00, 4302.76it/s]
Total missed pixels: 882234
===== RATIO: 0.7 =====
Number of triangles that are close to the object: tensor(1973417, device='cuda:0')
Number of gaussians that are close to the object: tensor(3110894, device='cuda:0')
Rendering progress: 150it [00:00, 4199.34it/s]
Total missed pixels: 882234
===== RATIO: 0.75 =====
Number of triangles that are close to the object: tensor(1973417, device='cuda:0')
Number of gaussians that are close to the object: tensor(3110894, device='cuda:0')
Rendering progress: 150it [00:00, 4315.42it/s]
Total missed pixels: 882234
===== RATIO: 0.8 =====
Number of triangles that are close to the object: tensor(1973417, device='cuda:0')
Number of gaussians that are close to the object: tensor(3110894, device='cuda:0')
Rendering progress: 150it [00:00, 4136.78it/s]
Total missed pixels: 882234
===== RATIO: 0.85 =====
Number of triangles that are close to the object: tensor(1973417, device='cuda:0')
Number of gaussians that are close to the object: tensor(3110894, device='cuda:0')
Rendering progress: 150it [00:00, 3574.00it/s]
Total missed pixels: 882234
RATIO: 0.05 MISSED PIXELS: 882234
RATIO: 0.1 MISSED PIXELS: 882234
RATIO: 0.2 MISSED PIXELS: 882234
RATIO: 0.25 MISSED PIXELS: 882234
RATIO: 0.3 MISSED PIXELS: 882234
RATIO: 0.32 MISSED PIXELS: 882234
RATIO: 0.34 MISSED PIXELS: 882234
RATIO: 0.36 MISSED PIXELS: 882234
RATIO: 0.38 MISSED PIXELS: 882234
RATIO: 0.4 MISSED PIXELS: 882234
RATIO: 0.42 MISSED PIXELS: 882234
RATIO: 0.44 MISSED PIXELS: 882234
RATIO: 0.46 MISSED PIXELS: 882234
RATIO: 0.48 MISSED PIXELS: 882234
RATIO: 0.5 MISSED PIXELS: 882234
RATIO: 0.55 MISSED PIXELS: 882234
RATIO: 0.6 MISSED PIXELS: 882234
RATIO: 0.65 MISSED PIXELS: 882234
RATIO: 0.7 MISSED PIXELS: 882234
RATIO: 0.75 MISSED PIXELS: 882234
RATIO: 0.8 MISSED PIXELS: 882234
RATIO: 0.85 MISSED PIXELS: 882234
===== BEST RATIO: 0.05 =====
====== Number of triangles from TRIANGLES_VIEW_COUNTER: 3110894 ======
Error executing code:
from utils.general_utils import safe_state
from scene_representation import SceneRepresentation
from edit_utils import *
from gpt.code_gen import setup_LMP
from opt import get_opts
import numpy as np
hparams = get_opts()
#safe_state(hparams.quiet)
scene = SceneRepresentation(hparams)
scene.render_from_3DGS()
table_obj = detect_object(scene, 'table')
basketball = retrieve_asset(scene, 'basketball')
for i in range(5):
pos = sample_point_above_object(scene, table_obj)
basketball_copy = make_copy(basketball)
basketball_copy = translate_object(basketball_copy, pos)
basketball_copy = allow_physics(basketball_copy)
insert_object(scene, basketball_copy)
scene.render_scene(skip_render_3DGS=False)
Traceback (most recent call last):
File "/workspace/autovfx/edit_scene.py", line 35, in <module>
run_scene_editing(hparams)
File "/workspace/autovfx/edit_scene.py", line 24, in run_scene_editing
edit_lmp(args.edit_text)
File "/workspace/autovfx/gpt/LMP.py", line 174, in __call__
exec_safe(to_exec, gvars, lvars)
File "/workspace/autovfx/gpt/LMP.py", line 236, in exec_safe
raise e
File "/workspace/autovfx/gpt/LMP.py", line 233, in exec_safe
exec(code_str, custom_gvars, lvars)
File "<string>", line 11, in <module>
File "/workspace/autovfx/edit_utils.py", line 137, in detect_object
obj_mesh_path = extract_object_from_scene(scene_representation, object_name, obj_id)
File "/workspace/autovfx/extract/extract_object.py", line 274, in extract_object_from_scene
scene_mesh_remaining.export(os.path.join(scene_mesh_save_dir, "removal_mesh.obj"))
AttributeError: 'numpy.ndarray' object has no attribute 'export'
Thank you so much for your excellent work. I have a few questions. First, when I run bash train_3dgs.sh garden using the link you provided below, it says that cameras.json is missing. https://drive.google.com/drive/folders/1eRdSAqDloGXk04JK60v3io6GHWdomy2N
Of course, I was able to find cameras.json in this link: https://drive.google.com/drive/folders/1KE8LSA_r-3f2LVlTLJ5k4SHENvbwdAfN, but I’m not sure during which process cameras.json is generated.
Additionally, I want to conduct experiments on a road dataset. Could you share the result files you’ve prepared for the road dataset so I can directly test them?
I can proceed with experiments using this link: https://drive.google.com/drive/folders/1tT1YSppo2lC4Let_vIIKcsr6OWlSFmqX, but the following processes take additional time, which causes delays.
I would greatly appreciate it if you could share the link. Thank you very much.