search

shanice-l / gdrnpp_bop2022

PyTorch Implementation of GDRNPP, winner (most of the awards) of the BOP Challenge 2022 at ECCV'22
Apache License 2.0
240 stars 51 forks source link

Missing file of `lm_full` in directory `ref` #6

Closed yinguoxiangyi closed 2 years ago

yinguoxiangyi commented 2 years ago

The missing file is used in https://github.com/shanice-l/gdrnpp_bop2022/blob/c2cd81ad5749d5355685666920d58d2ce7c8c8dd/det/yolox/data/datasets/lm_syn_imgn.py#L321

akshay-bapat-magna commented 2 years ago

This is possibly a typo. The ref/ directory contains lmo_full.py so it is likely referring to that module.

yinguoxiangyi commented 2 years ago

But there are many ref.lm_full in the file lm_syn_imgn.py, are you sure it's a typo?
I will replace all the lm_full with lmo_full and have a try.

Another question is where is the config file in the following commands? Could you please provide some sample commands?

./det/yolox/tools/test_yolox.sh <config_path> <gpu_ids> <ckpt_path> (other args)
./core/gdrn_modeling/test_gdrn.sh <config_path> <gpu_ids> <ckpt_path> (other args)
./core/gdrn_modeling/test_gdrn_depth_refine.sh <config_path> <gpu_ids> <ckpt_path> (other args)
akshay-bapat-magna commented 2 years ago

I am also trying to figure out the config file argument myself. I am also trying to use this repository for the first time, so even the lmo_full thing I mentioned is just my guess.

yinguoxiangyi commented 2 years ago

Uh strange, are you not the author of the repo?

akshay-bapat-magna commented 2 years ago

Nope! :)

funnymind commented 2 years ago

But there are many ref.lm_full in the file lm_syn_imgn.py, are you sure it's a typo? I will replace all the lm_full with lmo_full and have a try.

Another question is where is the config file in the following commands? Could you please provide some sample commands?

./det/yolox/tools/test_yolox.sh <config_path> <gpu_ids> <ckpt_path> (other args)
./core/gdrn_modeling/test_gdrn.sh <config_path> <gpu_ids> <ckpt_path> (other args)
./core/gdrn_modeling/test_gdrn_depth_refine.sh <config_path> <gpu_ids> <ckpt_path> (other args)

Hi I am also looking into this solution. For lm_full i try to use from this repo: https://github.com/THU-DA-6D-Pose-Group/GDR-Net/tree/main/ref For parameters i use like this: ./core/gdrn_modeling/test_gdrn.sh ./configs/gdrn/tless/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_tless.py 0 ./output/tless/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_tless/model_final_wo_optim.pth

Still under study, not sure how accurate this is.

yinguoxiangyi commented 2 years ago

@funnymind thanks for your hint, the location of lm_full you provide is probably true, I'll have a try.

And do you get the final test result successfully with your config? Is the evaluation result the same as the paper says?

yinguoxiangyi commented 2 years ago

Also, I think we need more help from the author @wangg12

wangg12 commented 2 years ago

Hi, I guess it should be ok by changing lm_full to lmo_full since lm and lmo share the same id:name mapping. The issue might be caused by removing ref/lm_full.py but forgetting to change some necessary codes. @shanice-l

yinguoxiangyi commented 2 years ago

lm_full exist in many files, not only lm_syn_imgn.py. The modification of code may be so much

funnymind commented 2 years ago

if there is no difference in the content, then it may be easier to just copy lmo_full file to lm_full ?

shanice-l commented 2 years ago

Sry for the missing files. I've uploaded it in the nearest commit.

shanice-l commented 2 years ago

But there are many ref.lm_full in the file lm_syn_imgn.py, are you sure it's a typo? I will replace all the lm_full with lmo_full and have a try.

Another question is where is the config file in the following commands? Could you please provide some sample commands?

./det/yolox/tools/test_yolox.sh <config_path> <gpu_ids> <ckpt_path> (other args)
./core/gdrn_modeling/test_gdrn.sh <config_path> <gpu_ids> <ckpt_path> (other args)
./core/gdrn_modeling/test_gdrn_depth_refine.sh <config_path> <gpu_ids> <ckpt_path> (other args)

I've also updated the readme. Now you can find some examples of using these commands.

yinguoxiangyi commented 2 years ago

Thanks very much, but another error occurred as following. 

root@96d962c42873:/workspace/gdrnpp_bop2022# ./det/yolox/tools/test_yolox.sh configs/yolox/bop_pbr/yolox_x_640_augCozyAAEhsv_ranger_30_epochs_lmo_pbr_lmo_bop_test.py 0 output/yolox/bop_pbr/yolox_x_640_augCozyAAEhsv_ranger_30_epochs_lmo_pbr_lmo_bop_test/model_final.pth
++ dirname ./det/yolox/tools/test_yolox.sh
+ this_dir=./det/yolox/tools
+ CFG=configs/yolox/bop_pbr/yolox_x_640_augCozyAAEhsv_ranger_30_epochs_lmo_pbr_lmo_bop_test.py
+ CUDA_VISIBLE_DEVICES=0
+ IFS=,
+ read -ra GPUS
+ NGPU=1
+ echo 'use gpu ids: 0 num gpus: 1'
use gpu ids: 0 num gpus: 1
+ CKPT=output/yolox/bop_pbr/yolox_x_640_augCozyAAEhsv_ranger_30_epochs_lmo_pbr_lmo_bop_test/model_final.pth
+ '[' '!' -f output/yolox/bop_pbr/yolox_x_640_augCozyAAEhsv_ranger_30_epochs_lmo_pbr_lmo_bop_test/model_final.pth ']'
+ NCCL_DEBUG=INFO
+ OMP_NUM_THREADS=1
+ MKL_NUM_THREADS=1
+ PYTHONPATH=./det/yolox/tools/../..:
+ CUDA_VISIBLE_DEVICES=0
+ python ./det/yolox/tools/main_yolox.py --config-file configs/yolox/bop_pbr/yolox_x_640_augCozyAAEhsv_ranger_30_epochs_lmo_pbr_lmo_bop_test.py --num-gpus 1 --eval-only train.init_checkpoint=output/yolox/bop_pbr/yolox_x_640_augCozyAAEhsv_ranger_30_epochs_lmo_pbr_lmo_bop_test/model_final.pth
You requested to import horovod which is missing or not supported for your OS.
Traceback (most recent call last):
  File "./det/yolox/tools/main_yolox.py", line 31, in <module>
    from det.yolox.data.datasets.dataset_factory import register_datasets_in_cfg
  File "/workspace/gdrnpp_bop2022/det/yolox/tools/../../../det/yolox/data/datasets/dataset_factory.py", line 5, in <module>
    from . import (
  File "/workspace/gdrnpp_bop2022/det/yolox/tools/../../../det/yolox/data/datasets/lm_dataset_d2.py", line 694, in <module>
    with open(cur_ann_file, "r") as f:
FileNotFoundError: [Errno 2] No such file or directory: '/workspace/gdrnpp_bop2022/datasets/BOP_DATASETS/lm/image_set/ape_train.txt'

Does image_set/ape_train.txt exist in LM train dataset. I use the following command to download the dataset, but can't find the file.

export SRC=https://bop.felk.cvut.cz/media/data/bop_datasets
wget $SRC/lm_base.zip         # Base archive with dataset info, camera parameters, etc.
wget $SRC/lm_models.zip       # 3D object models.
wget $SRC/lm_test_all.zip     # All test images ("_bop19" for a subset used in the BOP Challenge 2019/2020).
wget $SRC/lm_train_pbr.zip    # PBR training images (rendered with BlenderProc4BOP).

unzip lm_base.zip             # Contains folder "lm".
unzip lm_models.zip -d lm     # Unpacks to "lm".
unzip lm_test_all.zip -d lm   # Unpacks to "lm".
unzip lm_train_pbr.zip -d lm  # Unpacks to "lm".
shanice-l commented 2 years ago

Actually, the files for LM dataset are useless. I've removed them in the nearest commit.

yinguoxiangyi commented 2 years ago

Also, another error occurred. It is likely that hb_bdp is also missing in ref directory.

root@96d962c42873:/workspace/gdrnpp_bop2022# ./det/yolox/tools/test_yolox.sh configs/yolox/bop_pbr/yolox_x_640_augCozyAAEhsv_ranger_30_epochs_lmo_pbr_lmo_bop_test.py 0 output/yolox/bop_pbr/yolox_x_640_augCozyAAEhsv_ranger_30_epochs_lmo_pbr_lmo_bop_test/model_final.pth
++ dirname ./det/yolox/tools/test_yolox.sh
+ this_dir=./det/yolox/tools
+ CFG=configs/yolox/bop_pbr/yolox_x_640_augCozyAAEhsv_ranger_30_epochs_lmo_pbr_lmo_bop_test.py
+ CUDA_VISIBLE_DEVICES=0
+ IFS=,
+ read -ra GPUS
+ NGPU=1
+ echo 'use gpu ids: 0 num gpus: 1'
use gpu ids: 0 num gpus: 1
+ CKPT=output/yolox/bop_pbr/yolox_x_640_augCozyAAEhsv_ranger_30_epochs_lmo_pbr_lmo_bop_test/model_final.pth
+ '[' '!' -f output/yolox/bop_pbr/yolox_x_640_augCozyAAEhsv_ranger_30_epochs_lmo_pbr_lmo_bop_test/model_final.pth ']'
+ NCCL_DEBUG=INFO
+ OMP_NUM_THREADS=1
+ MKL_NUM_THREADS=1
+ PYTHONPATH=./det/yolox/tools/../..:
+ CUDA_VISIBLE_DEVICES=0
+ python ./det/yolox/tools/main_yolox.py --config-file configs/yolox/bop_pbr/yolox_x_640_augCozyAAEhsv_ranger_30_epochs_lmo_pbr_lmo_bop_test.py --num-gpus 1 --eval-only train.init_checkpoint=output/yolox/bop_pbr/yolox_x_640_augCozyAAEhsv_ranger_30_epochs_lmo_pbr_lmo_bop_test/model_final.pth
You requested to import horovod which is missing or not supported for your OS.
Traceback (most recent call last):
  File "./det/yolox/tools/main_yolox.py", line 31, in <module>
    from det.yolox.data.datasets.dataset_factory import register_datasets_in_cfg
  File "/workspace/gdrnpp_bop2022/det/yolox/tools/../../../det/yolox/data/datasets/dataset_factory.py", line 5, in <module>
    from . import (
  File "/workspace/gdrnpp_bop2022/det/yolox/tools/../../../det/yolox/data/datasets/hb_bench_driller_phone_d2.py", line 475, in <module>
    for obj in ref.hb_bdp.objects:
AttributeError: module 'ref' has no attribute 'hb_bdp'
shanice-l commented 2 years ago

I've deleted hb_bench_driller_phone_d2.py also in the latest commit.

yinguoxiangyi commented 2 years ago

The file lm_dataset_d2.py is added again, should it have been removed?

shanice-l commented 2 years ago

lm_dataset_d2.py is necessary since it's provide test set for lmo.

yinguoxiangyi commented 2 years ago

But this file caused following error.

root@96d962c42873:/workspace/gdrnpp_bop2022# ./det/yolox/tools/test_yolox.sh configs/yolox/bop_pbr/yolox_x_640_augCozyAAEhsv_ranger_30_epochs_lmo_pbr_lmo_bop_test.py 0 output/yolox/bop_pbr/yolox_x_640_augCozyAAEhsv_ranger_30_epochs_lmo_pbr_lmo_bop_test/model_final.pth
++ dirname ./det/yolox/tools/test_yolox.sh
+ this_dir=./det/yolox/tools
+ CFG=configs/yolox/bop_pbr/yolox_x_640_augCozyAAEhsv_ranger_30_epochs_lmo_pbr_lmo_bop_test.py
+ CUDA_VISIBLE_DEVICES=0
+ IFS=,
+ read -ra GPUS
+ NGPU=1
+ echo 'use gpu ids: 0 num gpus: 1'
use gpu ids: 0 num gpus: 1
+ CKPT=output/yolox/bop_pbr/yolox_x_640_augCozyAAEhsv_ranger_30_epochs_lmo_pbr_lmo_bop_test/model_final.pth
+ '[' '!' -f output/yolox/bop_pbr/yolox_x_640_augCozyAAEhsv_ranger_30_epochs_lmo_pbr_lmo_bop_test/model_final.pth ']'
+ NCCL_DEBUG=INFO
+ OMP_NUM_THREADS=1
+ MKL_NUM_THREADS=1
+ PYTHONPATH=./det/yolox/tools/../..:
+ CUDA_VISIBLE_DEVICES=0
+ python ./det/yolox/tools/main_yolox.py --config-file configs/yolox/bop_pbr/yolox_x_640_augCozyAAEhsv_ranger_30_epochs_lmo_pbr_lmo_bop_test.py --num-gpus 1 --eval-only train.init_checkpoint=output/yolox/bop_pbr/yolox_x_640_augCozyAAEhsv_ranger_30_epochs_lmo_pbr_lmo_bop_test/model_final.pth
You requested to import horovod which is missing or not supported for your OS.
Traceback (most recent call last):
  File "./det/yolox/tools/main_yolox.py", line 31, in <module>
    from det.yolox.data.datasets.dataset_factory import register_datasets_in_cfg
  File "/workspace/gdrnpp_bop2022/det/yolox/tools/../../../det/yolox/data/datasets/dataset_factory.py", line 5, in <module>
    from . import (
  File "/workspace/gdrnpp_bop2022/det/yolox/tools/../../../det/yolox/data/datasets/lm_dataset_d2.py", line 694, in <module>
    with open(cur_ann_file, "r") as f:
FileNotFoundError: [Errno 2] No such file or directory: '/workspace/gdrnpp_bop2022/datasets/BOP_DATASETS/lm/image_set/ape_train.txt'
shanice-l commented 2 years ago

Oops. I'll rewrite and test the code immediately. Plz wait for a minute.

yinguoxiangyi commented 2 years ago

Thanks.

shanice-l commented 2 years ago

I think there is no further reliance on LM. Sry for the trouble.

yinguoxiangyi commented 2 years ago

The output of the command ./det/yolox/tools/test_yolox.sh configs/yolox/bop_pbr/yolox_x_640_augCozyAAEhsv_ranger_30_epochs_lmo_pbr_lmo_bop_test.py 0 output/yolox/bop_pbr/yolox_x_640_augCozyAAEhsv_ranger_30_epochs_lmo_pbr_lmo_bop_test/model_final.pth is below.
From your side, does the data meet the expectation?

[1102_065133 det.yolox.evaluators.yolox_coco_evaluator@383]: Evaluating predictions with unofficial COCO API...
Loading and preparing results...
DONE (t=0.23s)
creating index...
index created!
[1102_065133 d2.evaluation.fast_eval_api@33]: Evaluate annotation type *bbox*
[1102_065133 d2.evaluation.fast_eval_api@95]: COCOeval_opt.evaluate() finished in 0.24 seconds.
[1102_065133 d2.evaluation.fast_eval_api@103]: Accumulating evaluation results...
[1102_065133 d2.evaluation.fast_eval_api@121]: COCOeval_opt.accumulate() finished in 0.06 seconds.
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.635
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.905
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.731
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.279
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.697
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.733
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.676
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.688
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.696
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.295
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.754
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.774
[1102_065133 det.yolox.evaluators.yolox_coco_evaluator@471]: Evaluation results for bbox: 
|   AP   |  AP50  |  AP75  |  APs   |  APm   |  APl   |
|:------:|:------:|:------:|:------:|:------:|:------:|
| 63.536 | 90.482 | 73.092 | 27.865 | 69.710 | 73.336 |
[1102_065133 det.yolox.evaluators.yolox_coco_evaluator@503]: Per-category bbox AP: 
| category   | AP     | category    | AP     | category   | AP     |
|:-----------|:-------|:------------|:-------|:-----------|:-------|
| ape        | 57.452 | can         | 80.020 | cat        | 54.236 |
| driller    | 76.059 | duck        | 69.249 | eggbox     | 39.370 |
| glue       | 55.871 | holepuncher | 76.031 |            |        |
[1102_065133 det.yolox.evaluators.yolox_coco_evaluator@533]: Per-category bbox AR: 
| category   | AR     | category   | AR     | category    | AR     | category   | AR     | category   | AR     |
|:-----------|:-------|:-----------|:-------|:------------|:-------|:-----------|:-------|:-----------|:-------|
| ape        | 64.225 | can        | 83.467 | cat         | 59.898 | driller    | 80.200 | duck       | 73.085 |
| eggbox     | 55.233 | glue       | 61.039 | holepuncher | 79.450 |            |        |            |        |
[1102_065133 det.yolox.engine.yolox_trainer@639]: Evaluation results for lmo_bop_test in csv format:
[1102_065133 d2.evaluation.testing@24]: copypaste: Task: bbox
[1102_065133 d2.evaluation.testing@25]: copypaste: AP,AP50,AP75,APs,APm,APl
[1102_065133 d2.evaluation.testing@26]: copypaste: 63.5360,90.4822,73.0919,27.8649,69.7097,73.3361
shanice-l commented 2 years ago

The evaluation results of Det are inaccurate. You can upload the csv files to the BOP system to get the accurate score.

yinguoxiangyi commented 2 years ago

Another error occurred when I ran the command ./core/gdrn_modeling/test_gdrn.sh configs/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo.py 0 output/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo/model_final_wo_optim.pth

Traceback (most recent call last):

> File "./core/gdrn_modeling/main_gdrn.py", line 233, in <module>
    main(args)
    │    └ Namespace(config_file='configs/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo.py', dist...
    └ <function main at 0x7fa2275fe710>

  File "./core/gdrn_modeling/main_gdrn.py", line 205, in main
    ).run(args, cfg)
          │     └ Config (path: configs/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo.py): {'OUTPUT_ROOT...
          └ Namespace(config_file='configs/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo.py', dist...

  File "/opt/conda/lib/python3.7/site-packages/pytorch_lightning/lite/lite.py", line 402, in _run_impl
    return run_method(*args, **kwargs)
           │           │       └ {}
           │           └ (Namespace(config_file='configs/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo.py', dis...
           └ functools.partial(<bound method LightningLite._run_with_strategy_setup of <__main__.Lite object at 0x7fa22826e590>>, <bound m...
  File "/opt/conda/lib/python3.7/site-packages/pytorch_lightning/lite/lite.py", line 409, in _run_with_strategy_setup
    return run_method(*args, **kwargs)
           │           │       └ {}
           │           └ (Namespace(config_file='configs/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo.py', dis...
           └ <bound method Lite.run of <__main__.Lite object at 0x7fa22826e590>>

  File "./core/gdrn_modeling/main_gdrn.py", line 183, in run
    return self.do_test(cfg, model)
           │    │       │    └ _LiteModule(
           │    │       │        (_forward_module): GDRN_DoubleMask(
           │    │       │          (backbone): FeatureListNet(
           │    │       │            (stem_0): Conv2d(3, 128, kernel_size...
           │    │       └ Config (path: configs/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo.py): {'OUTPUT_ROOT...
           │    └ <function GDRN_Lite.do_test at 0x7fa2279f4950>
           └ <__main__.Lite object at 0x7fa22826e590>

  File "/workspace/gdrnpp_bop2022/core/gdrn_modeling/../../core/gdrn_modeling/engine/engine.py", line 157, in do_test
    data_loader = build_gdrn_test_loader(cfg, dataset_name, train_objs=evaluator.train_objs)
                  │                      │    │                        │         └ ['ape', 'can', 'cat', 'driller', 'duck', 'eggbox', 'glue', 'holepuncher']
                  │                      │    │                        └ <core.gdrn_modeling.engine.gdrn_evaluator.GDRN_Evaluator object at 0x7fa2d7229a50>
                  │                      │    └ 'lmo_bop_test'
                  │                      └ Config (path: configs/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo.py): {'OUTPUT_ROOT...
                  └ <function build_gdrn_test_loader at 0x7fa227a31050>

  File "/workspace/gdrnpp_bop2022/core/gdrn_modeling/../../core/gdrn_modeling/datasets/data_loader.py", line 934, in build_gdrn_test_loader
    selected_scenes=cfg.DATASETS.get("EVAL_SCENE_IDS", None),
                    └ Config (path: configs/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo.py): {'OUTPUT_ROOT...

  File "/workspace/gdrnpp_bop2022/core/gdrn_modeling/../../core/utils/dataset_utils.py", line 167, in load_detections_into_dataset
    detections = mmcv.load(det_file)
                 │    │    └ 'datasets/BOP_DATASETS/lmo/test/test_bboxes/yolov4x_amodal_lmo_pbr_bop.json'
                 │    └ <function load at 0x7fa301228200>
                 └ <module 'mmcv' from '/opt/conda/lib/python3.7/site-packages/mmcv/__init__.py'>

  File "/opt/conda/lib/python3.7/site-packages/mmcv/fileio/io.py", line 64, in load
    with StringIO(file_client.get_text(file)) as f:
         │        │           │        └ 'datasets/BOP_DATASETS/lmo/test/test_bboxes/yolov4x_amodal_lmo_pbr_bop.json'
         │        │           └ <function FileClient.get_text at 0x7fa2f995ac20>
         │        └ <mmcv.fileio.file_client.FileClient object at 0x7fa227c95590>
         └ <class '_io.StringIO'>
  File "/opt/conda/lib/python3.7/site-packages/mmcv/fileio/file_client.py", line 1027, in get_text
    return self.client.get_text(filepath, encoding)
           │    │      │        │         └ 'utf-8'
           │    │      │        └ 'datasets/BOP_DATASETS/lmo/test/test_bboxes/yolov4x_amodal_lmo_pbr_bop.json'
           │    │      └ <function HardDiskBackend.get_text at 0x7fa2f997eef0>
           │    └ <mmcv.fileio.file_client.HardDiskBackend object at 0x7fa2275e20d0>
           └ <mmcv.fileio.file_client.FileClient object at 0x7fa227c95590>
  File "/opt/conda/lib/python3.7/site-packages/mmcv/fileio/file_client.py", line 552, in get_text
    with open(filepath, encoding=encoding) as f:
              │                  └ 'utf-8'
              └ 'datasets/BOP_DATASETS/lmo/test/test_bboxes/yolov4x_amodal_lmo_pbr_bop.json'

FileNotFoundError: [Errno 2] No such file or directory: 'datasets/BOP_DATASETS/lmo/test/test_bboxes/yolov4x_amodal_lmo_pbr_bop.json'
root@4bc8d44c51fb:/workspace/gdrnpp_bop2022# ls datasets/BOP_DATASETS/lmo/test/test_bboxes/
yolox_x_640_lmo_pbr_lmo_bop_test.json
root@4bc8d44c51fb:/workspace/gdrnpp_bop2022#

Maybe the filename of test_bboxes has changed.

shanice-l commented 2 years ago

Another error occurred when I ran the command ./core/gdrn_modeling/test_gdrn.sh configs/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo.py 0 output/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo/model_final_wo_optim.pth

Traceback (most recent call last):

> File "./core/gdrn_modeling/main_gdrn.py", line 233, in <module>
    main(args)
    │    └ Namespace(config_file='configs/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo.py', dist...
    └ <function main at 0x7fa2275fe710>

  File "./core/gdrn_modeling/main_gdrn.py", line 205, in main
    ).run(args, cfg)
          │     └ Config (path: configs/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo.py): {'OUTPUT_ROOT...
          └ Namespace(config_file='configs/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo.py', dist...

  File "/opt/conda/lib/python3.7/site-packages/pytorch_lightning/lite/lite.py", line 402, in _run_impl
    return run_method(*args, **kwargs)
           │           │       └ {}
           │           └ (Namespace(config_file='configs/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo.py', dis...
           └ functools.partial(<bound method LightningLite._run_with_strategy_setup of <__main__.Lite object at 0x7fa22826e590>>, <bound m...
  File "/opt/conda/lib/python3.7/site-packages/pytorch_lightning/lite/lite.py", line 409, in _run_with_strategy_setup
    return run_method(*args, **kwargs)
           │           │       └ {}
           │           └ (Namespace(config_file='configs/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo.py', dis...
           └ <bound method Lite.run of <__main__.Lite object at 0x7fa22826e590>>

  File "./core/gdrn_modeling/main_gdrn.py", line 183, in run
    return self.do_test(cfg, model)
           │    │       │    └ _LiteModule(
           │    │       │        (_forward_module): GDRN_DoubleMask(
           │    │       │          (backbone): FeatureListNet(
           │    │       │            (stem_0): Conv2d(3, 128, kernel_size...
           │    │       └ Config (path: configs/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo.py): {'OUTPUT_ROOT...
           │    └ <function GDRN_Lite.do_test at 0x7fa2279f4950>
           └ <__main__.Lite object at 0x7fa22826e590>

  File "/workspace/gdrnpp_bop2022/core/gdrn_modeling/../../core/gdrn_modeling/engine/engine.py", line 157, in do_test
    data_loader = build_gdrn_test_loader(cfg, dataset_name, train_objs=evaluator.train_objs)
                  │                      │    │                        │         └ ['ape', 'can', 'cat', 'driller', 'duck', 'eggbox', 'glue', 'holepuncher']
                  │                      │    │                        └ <core.gdrn_modeling.engine.gdrn_evaluator.GDRN_Evaluator object at 0x7fa2d7229a50>
                  │                      │    └ 'lmo_bop_test'
                  │                      └ Config (path: configs/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo.py): {'OUTPUT_ROOT...
                  └ <function build_gdrn_test_loader at 0x7fa227a31050>

  File "/workspace/gdrnpp_bop2022/core/gdrn_modeling/../../core/gdrn_modeling/datasets/data_loader.py", line 934, in build_gdrn_test_loader
    selected_scenes=cfg.DATASETS.get("EVAL_SCENE_IDS", None),
                    └ Config (path: configs/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo.py): {'OUTPUT_ROOT...

  File "/workspace/gdrnpp_bop2022/core/gdrn_modeling/../../core/utils/dataset_utils.py", line 167, in load_detections_into_dataset
    detections = mmcv.load(det_file)
                 │    │    └ 'datasets/BOP_DATASETS/lmo/test/test_bboxes/yolov4x_amodal_lmo_pbr_bop.json'
                 │    └ <function load at 0x7fa301228200>
                 └ <module 'mmcv' from '/opt/conda/lib/python3.7/site-packages/mmcv/__init__.py'>

  File "/opt/conda/lib/python3.7/site-packages/mmcv/fileio/io.py", line 64, in load
    with StringIO(file_client.get_text(file)) as f:
         │        │           │        └ 'datasets/BOP_DATASETS/lmo/test/test_bboxes/yolov4x_amodal_lmo_pbr_bop.json'
         │        │           └ <function FileClient.get_text at 0x7fa2f995ac20>
         │        └ <mmcv.fileio.file_client.FileClient object at 0x7fa227c95590>
         └ <class '_io.StringIO'>
  File "/opt/conda/lib/python3.7/site-packages/mmcv/fileio/file_client.py", line 1027, in get_text
    return self.client.get_text(filepath, encoding)
           │    │      │        │         └ 'utf-8'
           │    │      │        └ 'datasets/BOP_DATASETS/lmo/test/test_bboxes/yolov4x_amodal_lmo_pbr_bop.json'
           │    │      └ <function HardDiskBackend.get_text at 0x7fa2f997eef0>
           │    └ <mmcv.fileio.file_client.HardDiskBackend object at 0x7fa2275e20d0>
           └ <mmcv.fileio.file_client.FileClient object at 0x7fa227c95590>
  File "/opt/conda/lib/python3.7/site-packages/mmcv/fileio/file_client.py", line 552, in get_text
    with open(filepath, encoding=encoding) as f:
              │                  └ 'utf-8'
              └ 'datasets/BOP_DATASETS/lmo/test/test_bboxes/yolov4x_amodal_lmo_pbr_bop.json'

FileNotFoundError: [Errno 2] No such file or directory: 'datasets/BOP_DATASETS/lmo/test/test_bboxes/yolov4x_amodal_lmo_pbr_bop.json'
root@4bc8d44c51fb:/workspace/gdrnpp_bop2022# ls datasets/BOP_DATASETS/lmo/test/test_bboxes/
yolox_x_640_lmo_pbr_lmo_bop_test.json
root@4bc8d44c51fb:/workspace/gdrnpp_bop2022#

Maybe the filename of test_bboxes has changed.

Thanks for figuring out that!! The bug has been fixed.

wangg12 commented 2 years ago

The evaluation results of Det are inaccurate. You can upload the csv files to the BOP system to get the accurate score.

After update, This line https://github.com/shanice-l/gdrnpp_bop2022/blob/7b728c86f6c2f63078880d09e6f7fe2feb71577f/det/yolox/data/datasets/lmo_bop_test.py#L202 is not using amodal box.

yinguoxiangyi commented 2 years ago

The evaluation results of Det are inaccurate. You can upload the csv files to the BOP system to get the accurate score.

After update, This line

https://github.com/shanice-l/gdrnpp_bop2022/blob/7b728c86f6c2f63078880d09e6f7fe2feb71577f/det/yolox/data/datasets/lmo_bop_test.py#L202

is not using amodal box.

I am not very clear about why you say the Det test result is inaccurate. Is the inaccuracy caused by no using of amodal box?

yinguoxiangyi commented 2 years ago

How about below pose estimation result? Is this accurate? The following result is the output of ./core/gdrn_modeling/test_gdrn.sh configs/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo.py 0 output/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo/model_final_wo_optim.pth

20221103_033737|core.gdrn_modeling.engine.test_utils@453: 
objects      mspd_5:50  mssd_0.050:0.500  vsd_0.050:0.500  ad_0.020  ad_0.050  ad_0.100  reS_2  reS_5  reS_10  teS_2  teS_5  teS_10
ape          91.71      71.94             51.53            1.14      13.14     48.00     12.57  63.43  88.00   81.14  95.43  96.57
can          93.77      83.77             69.20            2.51      38.19     83.42     40.70  79.90  95.98   82.91  95.48  99.50
cat          91.75      66.55             51.52            1.75      16.96     48.54     14.62  45.03  78.36   59.65  86.55  93.57
driller      91.25      87.75             73.51            6.50      53.00     86.00     50.50  83.50  97.50   77.50  96.50  99.00
duck         90.89      53.22             49.27            0.00      1.67      8.89      9.44   29.44  71.67   52.78  95.56  98.89
eggbox       63.56      30.06             18.03            1.11      8.33      34.44     0.56   18.89  65.00   10.00  43.33  69.44
glue         85.93      68.50             49.74            10.71     54.29     76.43     10.00  35.71  74.29   55.71  85.00  95.00
holepuncher  89.75      66.15             47.08            0.50      5.50      27.00     26.50  64.50  94.00   43.00  95.50  98.00
Avg(8)       87.49      66.44             51.76            3.03      23.88     51.59     20.61  52.55  83.10   57.84  86.67  93.75
20221103_033737|core.gdrn_modeling.engine.test_utils@453: 
objects           ape    can    cat    driller  duck   eggbox  glue   holepuncher  Avg(8)
mspd_5:50         91.71  93.77  91.75  91.25    90.89  63.56   85.93  89.75        87.49
mssd_0.050:0.500  71.94  83.77  66.55  87.75    53.22  30.06   68.50  66.15        66.44
vsd_0.050:0.500   51.53  69.20  51.52  73.51    49.27  18.03   49.74  47.08        51.76
ad_0.020          1.14   2.51   1.75   6.50     0.00   1.11    10.71  0.50         3.03
ad_0.050          13.14  38.19  16.96  53.00    1.67   8.33    54.29  5.50         23.88
ad_0.100          48.00  83.42  48.54  86.00    8.89   34.44   76.43  27.00        51.59
reS_2             12.57  40.70  14.62  50.50    9.44   0.56    10.00  26.50        20.61
reS_5             63.43  79.90  45.03  83.50    29.44  18.89   35.71  64.50        52.55
reS_10            88.00  95.98  78.36  97.50    71.67  65.00   74.29  94.00        83.10
teS_2             81.14  82.91  59.65  77.50    52.78  10.00   55.71  43.00        57.84
teS_5             95.43  95.48  86.55  96.50    95.56  43.33   85.00  95.50        86.67
teS_10            96.57  99.50  93.57  99.00    98.89  69.44   95.00  98.00        93.75
100%|##########| 1/1 [00:00<00:00, 19.24it/s]
20221103_033737|core.gdrn_modeling.engine.test_utils@456: output/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo/inference_model_final_wo_optim/lmo_bop_test
20221103_033737|core.gdrn_modeling.engine.test_utils@80: eval time: 483.81608146600774s
yinguoxiangyi commented 2 years ago

When to do pose refinement, is the args of test_gdrn_depth_refine.sh the same as test_gdrn.sh?

yinguoxiangyi commented 2 years ago

If I use this command ./core/gdrn_modeling/test_gdrn_depth_refine.sh configs/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo.py 0 output/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo/model_final_wo_optim.pth which args are same as test_gdrn.sh, error occurred as following.

  File "/workspace/gdrnpp_bop2022/core/gdrn_modeling/../../core/gdrn_modeling/engine/gdrn_evaluator.py", line 81, in __init__
    texture_paths=self.data_ref.texture_paths if cfg.TEST.DEBUG else None,
                  │    │        │                └ Config (path: configs/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo.py): {'OUTPUT_ROOT...
                  │    │        └ None
                  │    └ <module 'ref.lmo_full' from '/workspace/gdrnpp_bop2022/core/gdrn_modeling/../../ref/lmo_full.py'>
                  └ <core.gdrn_modeling.engine.gdrn_evaluator.GDRN_Evaluator object at 0x7fa7238bb410>

  File "/opt/conda/lib/python3.7/site-packages/mmcv/utils/config.py", line 50, in __getattr__
    raise ex
          └ AttributeError("'ConfigDict' object has no attribute 'DEBUG'")

AttributeError: 'ConfigDict' object has no attribute 'DEBUG'

Seems there is not TEST.DEBUG in configs/gdrn/lmo_pbr/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_lmo.py

shanice-l commented 2 years ago

If you have new questions that are not related to this issue, plz start a new one.

shanice-l commented 2 years ago

This issue has been solved. If you don't have any further problems, it will be closed in 3 days.

yinguoxiangyi commented 2 years ago

Okay, thank you very much , I'll move the last 2 comments I submitted to a new issue. I think it is better to explain the inaccuracy problem in this issue because the context is related.

wangg12 commented 2 years ago

The evaluation results of Det are inaccurate. You can upload the csv files to the BOP system to get the accurate score.

After update, This line

https://github.com/shanice-l/gdrnpp_bop2022/blob/7b728c86f6c2f63078880d09e6f7fe2feb71577f/det/yolox/data/datasets/lmo_bop_test.py#L202

is not using amodal box.

I am not very clear about why you say the Det test result is inaccurate. Is the inaccuracy caused by no using of amodal box?

This should be resolved in the latest commit.