search

thodan / bop_toolkit

A Python toolkit of the BOP benchmark for 6D object pose estimation.
http://bop.felk.cvut.cz
MIT License
376 stars 135 forks source link

How to create datasets/BOP_DATASETS/mona/yolox_mona_test_bop.json for custom synthetic data named mona made with BlenderProc in BOP Format? #114

Closed monajalal closed 4 months ago

monajalal commented 4 months ago

I was wondering if I could get help from you regarding this inquiry. Please let me know if you may have any followup questions:

My synthetic dataset made with BlenderProc following lm and BOP format, has a test folder and each of the folders inside the test folder looks like below:

gdrnpp_bop2022/datasets/BOP_DATASETS/mona/test/000000$ ls
total 8.5M
drwxrwxr-x 23 mona mona 4.0K Feb  6 08:47 ..
drwxrwxr-x  2 mona mona  36K Feb  6 08:48 depth
drwxrwxr-x  2 mona mona  40K Feb  6 08:49 mask
drwxrwxr-x  2 mona mona  36K Feb  6 08:49 mask_visib
drwxrwxr-x  2 mona mona  36K Feb  6 10:13 rgb
drwxrwxr-x  6 mona mona 4.0K Feb  7 10:20 .
-rw-rw-r--  1 mona mona 8.3M Feb  6 08:43 scene_gt_coco.json
-rw-rw-r--  1 mona mona 6.5K Feb  6 08:51 scene_gt_info.json
-rw-rw-r--  1 mona mona  15K Feb  6 08:51 scene_camera.json
-rw-rw-r--  1 mona mona  11K Feb  6 08:52 scene_gt.json

gdrnpp_bop2022/datasets/BOP_DATASETS/mona/test$ ls
total 92K
drwxrwxr-x  6 mona mona 4.0K Feb  6 08:43 000001
drwxrwxr-x  6 mona mona 4.0K Feb  6 08:43 000002
drwxrwxr-x  6 mona mona 4.0K Feb  6 08:43 000003
drwxrwxr-x  6 mona mona 4.0K Feb  6 08:43 000004
drwxrwxr-x  6 mona mona 4.0K Feb  6 08:43 000005
drwxrwxr-x  6 mona mona 4.0K Feb  6 08:43 000006
drwxrwxr-x  6 mona mona 4.0K Feb  6 08:43 000007
drwxrwxr-x  6 mona mona 4.0K Feb  6 08:43 000008
drwxrwxr-x  6 mona mona 4.0K Feb  6 08:43 000009
drwxrwxr-x  6 mona mona 4.0K Feb  6 08:43 000010
drwxrwxr-x  6 mona mona 4.0K Feb  6 08:43 000011
drwxrwxr-x  6 mona mona 4.0K Feb  6 08:43 000012
drwxrwxr-x  6 mona mona 4.0K Feb  6 08:43 000013
drwxrwxr-x  6 mona mona 4.0K Feb  6 08:43 000014
drwxrwxr-x  6 mona mona 4.0K Feb  6 08:43 000015
drwxrwxr-x  6 mona mona 4.0K Feb  6 08:43 000016
drwxrwxr-x  6 mona mona 4.0K Feb  6 08:43 000017
drwxrwxr-x  6 mona mona 4.0K Feb  6 08:43 000018
drwxrwxr-x  6 mona mona 4.0K Feb  6 08:43 000019
drwxrwxr-x  6 mona mona 4.0K Feb  6 08:43 000020
drwxrwxr-x 23 mona mona 4.0K Feb  6 08:47 .
drwxrwxr-x  6 mona mona 4.0K Feb  6 09:41 ..

Here, let's assume my object is named mona. What script should I use to create the yolox_mona_test_bop.json file that would combine all the scene_gt_coco.json files inside each subfolder and converts it to the needed format?

When I perform training using my synthetic data, I get this error:

./core/gdrn_modeling/train_gdrn.sh configs/gdrn/mona/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_mona.py 0

20240207_090508|core.utils.my_writer@198: eta: 0:00:04  epoch: 1 iter: 999/1005[99.4%] time: 0.8545 lr: 0.0007992 max_mem: 25739M  total_loss: 21.54 (36.28)  loss_coor_x: 0.2556 (0.2636)  loss_coor_y: 0.264 (0.2709)  loss_coor_z: 0.1435 (0.2086)  loss_mask: 0.1077 (0.1372)  loss_mask_full: 0.09742 (0.1298)  loss_region: 19.12 (20.23)  loss_PM_R: 0.9215 (0.9231)  loss_centroid: 0.06136 (0.1017)  loss_z: 0.4276 (14.02)
epoch_str:  epoch: 1
20240207_090513|core.utils.my_writer@198: eta: 0:00:00  epoch: 1 iter: 1004/1005[99.9%] time: 0.8545 lr: 2.4927e-08 max_mem: 25739M  total_loss: 21.54 (36.21)  loss_coor_x: 0.2558 (0.2636)  loss_coor_y: 0.264 (0.2708)  loss_coor_z: 0.1435 (0.2082)  loss_mask: 0.1091 (0.1371)  loss_mask_full: 0.09805 (0.1297)  loss_region: 19.12 (20.23)  loss_PM_R: 0.9212 (0.9231)  loss_centroid: 0.05485 (0.1014)  loss_z: 0.4365 (13.95)
20240207_090513|fvcore.common.checkpoint@124: Saving checkpoint to output/gdrn/mona/classAware_mona/model_final.pth
20240207_090517|core.gdrn_modeling.datasets.mona@99: load cached dataset dicts from /home/mona/gdrnpp_bop2022/.cache/dataset_dicts_mona_test_d6a3796e790d48944a3628d1133d1155.pkl
20240207_090517|core.utils.dataset_utils@166: Loading detections for mona_test from: datasets/BOP_DATASETS/mona/yolox_mona_test_bop.json
20240207_090517|ERR|__main__@233: An error has been caught in function '<module>', process 'MainProcess' (803470), thread 'MainThread' (139772596868928):
Traceback (most recent call last):

> File "/home/mona/gdrnpp_bop2022/./core/gdrn_modeling/main_gdrn.py", line 233, in <module>
    main(args)
    │    └ Namespace(config_file='configs/gdrn/mona/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_mona.py', re...
    └ <function main at 0x7f1dbfc888b0>

  File "/home/mona/gdrnpp_bop2022/./core/gdrn_modeling/main_gdrn.py", line 199, in main
    Lite(
    └ <class '__main__.Lite'>

  File "/home/mona/anaconda3/envs/gdrnpp/lib/python3.9/site-packages/pytorch_lightning/lite/lite.py", line 408, in _run_impl
    return run_method(*args, **kwargs)
           │           │       └ {}
           │           └ (Namespace(config_file='configs/gdrn/mona/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_mona.py', r...
           └ functools.partial(<bound method LightningLite._run_with_strategy_setup of <__main__.Lite object at 0x7f1f57bc3730>>, <bound m...
  File "/home/mona/anaconda3/envs/gdrnpp/lib/python3.9/site-packages/pytorch_lightning/lite/lite.py", line 413, in _run_with_strategy_setup
    return run_method(*args, **kwargs)
           │           │       └ {}
           │           └ (Namespace(config_file='configs/gdrn/mona/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_mona.py', r...
           └ <bound method Lite.run of <__main__.Lite object at 0x7f1f57bc3730>>

  File "/home/mona/gdrnpp_bop2022/./core/gdrn_modeling/main_gdrn.py", line 189, in run
    return self.do_test(cfg, model)
           │    │       │    └ _LiteModule(
           │    │       │        (_module): GDRN_DoubleMask(
           │    │       │          (backbone): FeatureListNet(
           │    │       │            (stem_0): Conv2d(3, 128, kernel_size=(4, 4),...
           │    │       └ Config (path: configs/gdrn/mona/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_mona.py): {'OUTPUT_RO...
           │    └ <function GDRN_Lite.do_test at 0x7f1dc752fe50>
           └ <__main__.Lite object at 0x7f1f57bc3730>

  File "/home/mona/gdrnpp_bop2022/core/gdrn_modeling/../../core/gdrn_modeling/engine/engine.py", line 157, in do_test
    data_loader = build_gdrn_test_loader(cfg, dataset_name, train_objs=evaluator.train_objs)
                  │                      │    │                        │         └ ['mona']
                  │                      │    │                        └ <core.gdrn_modeling.engine.gdrn_evaluator.GDRN_Evaluator object at 0x7f1e1d221d00>
                  │                      │    └ 'mona_test'
                  │                      └ Config (path: configs/gdrn/mona/convnext_a6_AugCosyAAEGray_BG05_mlL1_DMask_amodalClipBox_classAware_mona.py): {'OUTPUT_RO...
                  └ <function build_gdrn_test_loader at 0x7f1dc756e280>

  File "/home/mona/gdrnpp_bop2022/core/gdrn_modeling/../../core/gdrn_modeling/datasets/data_loader.py", line 926, in build_gdrn_test_loader
    dataset_dicts = load_detections_into_dataset(
                    └ <function load_detections_into_dataset at 0x7f1dc755c160>

  File "/home/mona/gdrnpp_bop2022/core/gdrn_modeling/../../core/utils/dataset_utils.py", line 167, in load_detections_into_dataset
    detections = mmcv.load(det_file)
                 │    │    └ 'datasets/BOP_DATASETS/mona/yolox_mona_test_bop.json'
                 │    └ <function load at 0x7f1e38c5c5e0>
                 └ <module 'mmcv' from '/home/mona/anaconda3/envs/gdrnpp/lib/python3.9/site-packages/mmcv/__init__.py'>

  File "/home/mona/anaconda3/envs/gdrnpp/lib/python3.9/site-packages/mmcv/fileio/io.py", line 64, in load
    with StringIO(file_client.get_text(file)) as f:
         │        │           │        └ 'datasets/BOP_DATASETS/mona/yolox_mona_test_bop.json'
         │        │           └ <function FileClient.get_text at 0x7f1e38c4b5e0>
         │        └ <mmcv.fileio.file_client.FileClient object at 0x7f1dbfc39100>
         └ <class '_io.StringIO'>
  File "/home/mona/anaconda3/envs/gdrnpp/lib/python3.9/site-packages/mmcv/fileio/file_client.py", line 1030, in get_text
    return self.client.get_text(filepath, encoding)
           │    │      │        │         └ 'utf-8'
           │    │      │        └ 'datasets/BOP_DATASETS/mona/yolox_mona_test_bop.json'
           │    │      └ <function HardDiskBackend.get_text at 0x7f1e38c468b0>
           │    └ <mmcv.fileio.file_client.HardDiskBackend object at 0x7f1dbfc393a0>
           └ <mmcv.fileio.file_client.FileClient object at 0x7f1dbfc39100>
  File "/home/mona/anaconda3/envs/gdrnpp/lib/python3.9/site-packages/mmcv/fileio/file_client.py", line 555, in get_text
    with open(filepath, encoding=encoding) as f:
              │                  └ 'utf-8'
              └ 'datasets/BOP_DATASETS/mona/yolox_mona_test_bop.json'

FileNotFoundError: [Errno 2] No such file or directory: 'datasets/BOP_DATASETS/mona/yolox_mona_test_bop.json'

When I run the following:

(gdrnpp) mona@ada:~/gdrnpp_bop2022$ python core/gdrn_modeling/tools/tudl/convert_det_to_our_format.py
/home/mona/anaconda3/envs/gdrnpp/lib/python3.9/site-packages/mmcv/__init__.py:20: UserWarning: On January 1, 2023, MMCV will release v2.0.0, in which it will remove components related to the training process and add a data transformation module. In addition, it will rename the package names mmcv to mmcv-lite and mmcv-full to mmcv. See https://github.com/open-mmlab/mmcv/blob/master/docs/en/compatibility.md for more details.
  warnings.warn(
Traceback (most recent call last):
  File "/home/mona/gdrnpp_bop2022/core/gdrn_modeling/tools/tudl/convert_det_to_our_format.py", line 11, in <module>
    ds = mmcv.load(path)
  File "/home/mona/anaconda3/envs/gdrnpp/lib/python3.9/site-packages/mmcv/fileio/io.py", line 64, in load
    with StringIO(file_client.get_text(file)) as f:
  File "/home/mona/anaconda3/envs/gdrnpp/lib/python3.9/site-packages/mmcv/fileio/file_client.py", line 1030, in get_text
    return self.client.get_text(filepath, encoding)
  File "/home/mona/anaconda3/envs/gdrnpp/lib/python3.9/site-packages/mmcv/fileio/file_client.py", line 555, in get_text
    with open(filepath, encoding=encoding) as f:
FileNotFoundError: [Errno 2] No such file or directory: './output/yolox/bop_pbr/yolox_x_640_augCozyAAEhsv_ranger_30_epochs_mona_bop_test/inference/mona_test/coco_instances_results_bop.json'
(gdrnpp) mona@ada:~/gdrnpp_bop2022$ rg coco_instances_results_bop.json
det/yolox/evaluators/yolox_coco_evaluator.py
372:            file_path = os.path.join(self._output_dir, "coco_instances_results_bop.json")

core/gdrn_modeling/tools/tudl/convert_det_to_our_format.py
9:path = "./output/yolox/bop_pbr/yolox_x_640_augCozyAAEhsv_ranger_30_epochs_mona_bop_test/inference/mona_test/coco_instances_results_bop.json"

I am not sure how det/yolox/evaluators/yolox_coco_evaluator.py should be leveraged for this purpose.

https://github.com/shanice-l/gdrnpp_bop2022/issues/113

This is how the dataset looks like:

gdrnpp_bop2022/datasets/BOP_DATASETS/mona$ ls
total 68K
drwxrwxr-x  4 mona mona 4.0K Feb  2 15:16 ..
drwxrwxr-x 52 mona mona 4.0K Feb  6 08:42 train
drwxrwxr-x 23 mona mona 4.0K Feb  6 08:47 test
drwxrwxr-x  6 mona mona 4.0K Feb  6 09:41 .
drwxrwxr-x  2 mona mona 4.0K Feb  6 10:54 models
drwxrwxr-x  2 mona mona 4.0K Feb  6 13:46 models_eval
-rw-rw-r--  1 mona mona  44K Feb  6 09:57 test_targets_bop19.json

an example of a folder from train folder:

gdrnpp_bop2022/datasets/BOP_DATASETS/mona$ ls train/000000/
total 9.3M
drwxrwxr-x  6 mona mona 4.0K Feb  6 08:42 .
drwxrwxr-x 52 mona mona 4.0K Feb  6 08:42 ..
drwxrwxr-x  2 mona mona  36K Feb  6 09:20 depth
drwxrwxr-x  2 mona mona  40K Feb  6 09:20 mask
drwxrwxr-x  2 mona mona  36K Feb  6 09:21 mask_visib
drwxrwxr-x  2 mona mona  64K Feb  6 10:19 rgb
-rw-rw-r--  1 mona mona 8.3M Feb  6 08:42 scene_gt_coco.json
-rw-rw-r--  1 mona mona 395K Feb  6 09:21 scene_camera.json
-rw-rw-r--  1 mona mona 180K Feb  6 09:22 scene_gt_info.json
-rw-rw-r--  1 mona mona 285K Feb  6 09:22 scene_gt.json