search

NVlabs / FoundationPose

[CVPR 2024 Highlight] FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects
https://nvlabs.github.io/FoundationPose/
Other
1.01k stars 112 forks source link

running the demo -- File "/data/FoundationPose/estimater.py", line 120, in make_rotation_grid rot_grid = mycpp.cluster_poses(30, 99999, rot_grid, self.symmetry_tfs.data.cpu().numpy()) AttributeError: 'NoneType' object has no attribute 'cluster_poses' #8

Closed monajalal closed 1 month ago

monajalal commented 2 months ago

Hi Bowen,

Great piece of research work!

Could you please share your insights on this bug with me? I followed all of your instructions for conda env

(foundationpose) mona@ada:/data/FoundationPose$ python run_demo.py --debug 2
Warp 1.0.2 initialized:
   CUDA Toolkit 11.5, Driver 12.2
   Devices:
     "cpu"      : "x86_64"
     "cuda:0"   : "NVIDIA RTX 6000 Ada Generation" (47 GiB, sm_89, mempool enabled)
   Kernel cache:
     /home/mona/.cache/warp/1.0.2
[__init__()] self.cfg: 
 lr: 0.0001
c_in: 6
zfar: 'Infinity'
debug: null
n_view: 1
run_id: 3wy8qqex
use_BN: true
exp_name: 2024-01-11-20-02-45
n_epochs: 62
save_dir: /home/bowenw/debug/2024-01-11-20-02-45/
use_mask: false
loss_type: pairwise_valid
optimizer: adam
batch_size: 64
crop_ratio: 1.1
enable_amp: true
use_normal: false
max_num_key: null
warmup_step: -1
input_resize:
- 160
- 160
max_step_val: 1000
vis_interval: 1000
weight_decay: 0
normalize_xyz: true
resume_run_id: null
clip_grad_norm: 'Infinity'
lr_epoch_decay: 500
render_backend: nvdiffrast
train_num_pair: 5
lr_decay_epochs:
- 50
n_epochs_warmup: 1
make_pair_online: false
gradient_max_norm: 'Infinity'
max_step_per_epoch: 10000
n_rendering_workers: 1
save_epoch_interval: 100
n_dataloader_workers: 100
split_objects_across_gpus: true
ckpt_dir: /data/FoundationPose/learning/training/../../weights/2024-01-11-20-02-45/model_best.pth

[__init__()] self.h5_file:None
[__init__()] Using pretrained model from /data/FoundationPose/learning/training/../../weights/2024-01-11-20-02-45/model_best.pth
[__init__()] init done
[__init__()] welcome
[__init__()] self.cfg: 
 lr: 0.0001
c_in: 6
zfar: .inf
debug: null
w_rot: 0.1
n_view: 1
run_id: null
use_BN: true
rot_rep: axis_angle
ckpt_dir: /data/FoundationPose/learning/training/../../weights/2023-10-28-18-33-37/model_best.pth
exp_name: 2023-10-28-18-33-37
save_dir: /tmp/2023-10-28-18-33-37/
loss_type: l2
optimizer: adam
trans_rep: tracknet
batch_size: 64
crop_ratio: 1.2
use_normal: false
BN_momentum: 0.1
max_num_key: null
warmup_step: -1
input_resize:
- 160
- 160
max_step_val: 1000
normal_uint8: false
vis_interval: 1000
weight_decay: 0
n_max_objects: null
normalize_xyz: true
clip_grad_norm: 'Infinity'
rot_normalizer: 0.3490658503988659
trans_normalizer:
- 0.019999999552965164
- 0.019999999552965164
- 0.05000000074505806
max_step_per_epoch: 25000
val_epoch_interval: 10
n_dataloader_workers: 60
enable_amp: true
use_mask: false

[__init__()] self.h5_file:
[__init__()] Using pretrained model from /data/FoundationPose/learning/training/../../weights/2023-10-28-18-33-37/model_best.pth
[__init__()] init done
[reset_object()] self.diameter:0.19646325799497472, vox_size:0.009823162899748735
[reset_object()] self.pts:torch.Size([607, 3])
[reset_object()] reset done
[make_rotation_grid()] cam_in_obs:(42, 4, 4)
[make_rotation_grid()] rot_grid:(252, 4, 4)
Traceback (most recent call last):
  File "run_demo.py", line 41, in <module>
    est = FoundationPose(model_pts=mesh.vertices, model_normals=mesh.vertex_normals, mesh=mesh, scorer=scorer, refiner=refiner, debug_dir=debug_dir, debug=debug, glctx=glctx)
  File "/data/FoundationPose/estimater.py", line 27, in __init__
    self.make_rotation_grid(min_n_views=40, inplane_step=60)
  File "/data/FoundationPose/estimater.py", line 120, in make_rotation_grid
    rot_grid = mycpp.cluster_poses(30, 99999, rot_grid, self.symmetry_tfs.data.cpu().numpy())
AttributeError: 'NoneType' object has no attribute 'cluster_poses'

I also have:

$ ls /data/YCB_Video/
total 20K
drwxr-xr-x 23 mona mona 4.0K Sep 25  2023 ref_views_16
drwxr-xr-x 23 mona mona 4.0K Nov  8 15:58 ref_views_4
drwxr-xr-x 23 mona mona 4.0K Nov  8 21:13 ref_views_8
drwx------ 27 mona mona 4.0K Mar 28 15:21 ..
drwxrwxr-x  5 mona mona 4.0K Mar 28 15:27 .

and

(foundationpose) mona@ada:/data/FoundationPose$ ls
total 172K
drwxrwxr-x  2 mona mona 4.0K Mar 28 13:52 assets
drwxrwxr-x  5 mona mona 4.0K Mar 28 13:52 learning
drwxrwxr-x  8 mona mona 4.0K Mar 28 13:52 .git
drwxrwxr-x  2 mona mona 4.0K Mar 28 13:52 docker
drwxrwxr-x  5 mona mona 4.0K Mar 28 13:59 mycpp
drwxrwxr-x  9 mona mona 4.0K Mar 28 15:03 nvdiffrast
drwxrwxr-x  2 mona mona 4.0K Mar 28 15:06 __pycache__
drwxrwxr-x  3 mona mona 4.0K Mar 28 15:09 demo_data
drwxrwxr-x  4 mona mona 4.0K Mar 28 15:14 weights
drwx------ 27 mona mona 4.0K Mar 28 15:21 ..
drwxrwxr-x  3 mona mona 4.0K Mar 28 15:22 bundlesdf
drwxrwxr-x  4 mona mona 4.0K Mar 28 15:23 debug
drwxrwxr-x 13 mona mona 4.0K Mar 28 15:24 .
-rw-rw-r--  1 mona mona  35K Mar 28 13:52 Utils.py
-rw-rw-r--  1 mona mona 4.3K Mar 28 13:52 LICENSE
-rw-rw-r--  1 mona mona 2.1K Mar 28 13:52 .gitignore
-rw-rw-r--  1 mona mona 5.1K Mar 28 13:52 run_ycb_video.py
-rw-rw-r--  1 mona mona 3.2K Mar 28 13:52 run_demo.py
-rw-rw-r--  1 mona mona 8.0K Mar 28 13:52 readme.md
-rw-rw-r--  1 mona mona 2.5K Mar 28 13:52 offscreen_renderer.py
-rw-rw-r--  1 mona mona  11K Mar 28 13:52 estimater.py
-rw-rw-r--  1 mona mona  21K Mar 28 13:52 datareader.py
-rw-rw-r--  1 mona mona  212 Mar 28 13:52 build_all.sh
-rw-rw-r--  1 mona mona 4.9K Mar 28 15:23 run_linemod.py
(foundationpose) mona@ada:/data/FoundationPose$ ls weights
total 228M
drwxrwxr-x  2 mona mona 4.0K Mar 28 15:12 2024-01-11-20-02-45
drwxrwxr-x  2 mona mona 4.0K Mar 28 15:14 2023-10-28-18-33-37
drwxrwxr-x  4 mona mona 4.0K Mar 28 15:14 .
drwxrwxr-x 13 mona mona 4.0K Mar 28 15:24 ..
-rw-rw-r--  1 mona mona 168M Mar 28 15:10 2024-01-11-20-02-45-20240328T190945Z-001.zip
-rw-rw-r--  1 mona mona  61M Mar 28 15:13 2023-10-28-18-33-37-20240328T191312Z-001.zip
(foundationpose) mona@ada:/data/FoundationPose$ ls demo_data/
total 346M
drwxrwxrwx  6 mona mona 4.0K Dec  6 01:41 mustard0
drwxrwxr-x  3 mona mona 4.0K Mar 28 15:09 .
drwxrwxr-x 13 mona mona 4.0K Mar 28 15:24 ..
-rw-rw-r--  1 mona mona 346M Mar 28 15:08 mustard0.zip

(base) mona@ada:~$ uname -a
Linux ada 6.5.0-25-generic #25~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Tue Feb 20 16:09:15 UTC 2 x86_64 x86_64 x86_64 GNU/Linux
(base) mona@ada:~$ lsb_release -a
LSB Version:    core-11.1.0ubuntu4-noarch:security-11.1.0ubuntu4-noarch
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.3 LTS
Release:    22.04
Codename:   jammy

and

(base) mona@ada:~$ nvidia-smi
Thu Mar 28 15:28:50 2024       
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.104.12             Driver Version: 535.104.12   CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA RTX 6000 Ada Gene...    On  | 00000000:52:00.0  On |                  Off |
| 51%   77C    P2             218W / 300W |   5246MiB / 49140MiB |     73%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+

+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|    0   N/A  N/A      4427      G   /usr/lib/xorg/Xorg                          866MiB |
|    0   N/A  N/A      4598      G   /usr/bin/gnome-shell                         54MiB |
|    0   N/A  N/A      5119      G   ...AAAAAAAACAAAAAAAAAA= --shared-files       72MiB |
|    0   N/A  N/A      5274      G   ...ures=SpareRendererForSitePerProcess       68MiB |
|    0   N/A  N/A    124228      G   ...2208806,17303107387989153126,262144       52MiB |
|    0   N/A  N/A   1357737      G   ...sion,SpareRendererForSitePerProcess      315MiB |
|    0   N/A  N/A   1603088      G   ...irefox/3941/usr/lib/firefox/firefox      248MiB |
|    0   N/A  N/A   1949758    C+G   ...e/ov/pkg/isaac_sim-2023.1.1/kit/kit     3422MiB |
|    0   N/A  N/A   1951053      G   meshlab                                       8MiB |
+---------------------------------------------------------------------------------------+
wenbowen123 commented 2 months ago

did you finish the step of build.sh?

Sgt-Hashtag commented 1 month ago

I am also facing the same issue with this

---- This was the Output from running the build_all.sh---- Consolidate compiler generated dependencies of target mycpp [100%] Built target mycpp build_all.sh: line 4: cd: /kaolin: No such file or directory Obtaining file:///home/adas/Desktop/temp/6DPose/FoundationPose/bundlesdf/mycuda Preparing metadata (setup.py) ... done Installing collected packages: common Running setup.py develop for common Successfully installed common-0.0.0

kaolin is built in the docker I am guessing but why is it called in the build_all.sh

wenbowen123 commented 1 month ago

@Sgt-Hashtag Kaolin needs to compile based each machine's GPU. The docker image's one is built with a 3090, not necessarily work for you, so it needs rebuild.

Sgt-Hashtag commented 1 month ago

@Sgt-Hashtag Kaolin needs to compile based each machine's GPU. The docker image's one is built with a 3090, not necessarily work for you, so it needs rebuild.

Okay...since there is no kaolin directory in this repository then should we clone the kaolin repo as well and add here?

wenbowen123 commented 1 month ago

@Sgt-Hashtag are you using docker? If so, the kaolin is already in the root folder. Build.sh will cd there and rebuild

guillermodecelisrodriguez commented 1 month ago

hello i am facing the same error: AttributeError: 'NoneType' object has no attribute 'cluster_poses'

I am using docker and once i start the container i run build_all.sh obtaining this output:

Installing collected packages: kaolin Running setup.py develop for kaolin Successfully installed kaolin-0.15.0 Obtaining file:///home/bowen/bundlesdf/mycuda Preparing metadata (setup.py) ... done Installing collected packages: common Running setup.py develop for common Successfully installed common-0.0.0

Seems okey to me but when running "python run_demo.py" i get the same error:

_[init()] self.h5_file: [init()] Using pretrained model from /home/bowen/learning/training/../../weights/2023-10-28-18-33-37/model_best.pth [init()] init done [reset_object()] self.diameter:0.19646325799497472, vox_size:0.009823162899748735 [reset_object()] self.pts:torch.Size([607, 3]) [reset_object()] reset done [make_rotation_grid()] cam_in_obs:(42, 4, 4) [make_rotation_grid()] rot_grid:(252, 4, 4) Traceback (most recent call last): File "run_demo.py", line 41, in est = FoundationPose(model_pts=mesh.vertices, model_normals=mesh.vertex_normals, mesh=mesh, scorer=scorer, refiner=refiner, debug_dir=debug_dir, debug=debug, glctx=glctx) File "/home/bowen/estimater.py", line 27, in init self.make_rotation_grid(min_n_views=40, inplane_step=60) File "/home/bowen/estimater.py", line 120, in make_rotation_grid rot_grid = mycpp.cluster_poses(30, 99999, rot_grid, self.symmetrytfs.data.cpu().numpy()) AttributeError: 'NoneType' object has no attribute 'cluster_poses'

I believe the problem may be related with the build of mycpp as i get the next error when executing build_all.sh:

CMake Error: The source "/home/bowen/mycpp/CMakeLists.txt" does not match the source "/home/Desktop/FoundationPose/mycpp/CMakeLists.txt" used to generate cache. Re-run cmake with a different source directory.

wenbowen123 commented 1 month ago

/home/bowen/mycpp/CMakeLists.txt did you move the mycpp folder outside of this repo?

You have to finish the build, otherwise the mycpp will be None.

Sgt-Hashtag commented 1 month ago

@Sgt-Hashtag are you using docker? If so, the kaolin is already in the root folder. Build.sh will cd there and rebuild

Thanks it works now...but during the inference got OutOfMemoryError ..will have to figure that out i guess

aghaphd commented 1 month ago

I have same error:

(my) root@0d5221b7d18f:/workspace# python run_demo.py Warp 1.0.2 initialized: CUDA Toolkit 11.5, Driver 11.4 Devices: "cpu" : "x86_64" "cuda:0" : "NVIDIA GeForce RTX 3070 Ti Laptop GPU" (8 GiB, sm_86, mempool enabled) Kernel cache: /root/.cache/warp/1.0.2 [init()] self.cfg: lr: 0.0001 c_in: 6 zfar: 'Infinity' debug: null n_view: 1 run_id: 3wy8qqex use_BN: true exp_name: 2024-01-11-20-02-45 n_epochs: 62 save_dir: /home/ali/debug/2024-01-11-20-02-45/ use_mask: false loss_type: pairwise_valid optimizer: adam batch_size: 64 crop_ratio: 1.1 enable_amp: true use_normal: false max_num_key: null warmup_step: -1 input_resize:

[init()] self.h5_file:None [init()] Using pretrained model from /workspace/learning/training/../../weights/2024-01-11-20-02-45/model_best.pth [init()] init done [init()] welcome [init()] self.cfg: lr: 0.0001 c_in: 6 zfar: .inf debug: null w_rot: 0.1 n_view: 1 run_id: null use_BN: true rot_rep: axis_angle ckpt_dir: /workspace/learning/training/../../weights/2023-10-28-18-33-37/model_best.pth exp_name: 2023-10-28-18-33-37 save_dir: /tmp/2023-10-28-18-33-37/ loss_type: l2 optimizer: adam trans_rep: tracknet batch_size: 64 crop_ratio: 1.2 use_normal: false BN_momentum: 0.1 max_num_key: null warmup_step: -1 input_resize:

[init()] self.h5_file: [init()] Using pretrained model from /workspace/learning/training/../../weights/2023-10-28-18-33-37/model_best.pth [init()] init done [reset_object()] self.diameter:0.19646325799497472, vox_size:0.009823162899748735 [reset_object()] self.pts:torch.Size([607, 3]) [reset_object()] reset done [make_rotation_grid()] cam_in_obs:(42, 4, 4) [make_rotation_grid()] rot_grid:(252, 4, 4) Traceback (most recent call last): File "run_demo.py", line 41, in est = FoundationPose(model_pts=mesh.vertices, model_normals=mesh.vertex_normals, mesh=mesh, scorer=scorer, refiner=refiner, debug_dir=debug_dir, debug=debug, glctx=glctx) File "/workspace/estimater.py", line 27, in init self.make_rotation_grid(min_n_views=40, inplane_step=60) File "/workspace/estimater.py", line 120, in make_rotation_grid rot_grid = mycpp.cluster_poses(30, 99999, rot_grid, self.symmetry_tfs.data.cpu().numpy()) AttributeError: 'NoneType' object has no attribute 'cluster_poses' (my) root@0d5221b7d18f:/workspace#

can you help me where is the problem?

aghaphd commented 1 month ago

Hello,

I have tried to run that, the buiid_all.sh run smoothly but later the same error of demo appear. I have no idea where is the issue?

Thanks.

On Thu, 25 Apr 2024 at 16:55, Jeffrey @.***> wrote:

In Docker, execute the bash script build_all.sh before running the Python script run_demo.py. It works for me.

— Reply to this email directly, view it on GitHub https://github.com/NVlabs/FoundationPose/issues/8#issuecomment-2076695195, or unsubscribe https://github.com/notifications/unsubscribe-auth/AU3QN46TJMZMAJU5SRHJ7QLY7DAGRAVCNFSM6AAAAABFNLSFYSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANZWGY4TKMJZGU . You are receiving this because you commented.Message ID: @.***>