search

facebookresearch / CutLER

Code release for "Cut and Learn for Unsupervised Object Detection and Instance Segmentation" and "VideoCutLER: Surprisingly Simple Unsupervised Video Instance Segmentation"
Other
913 stars 90 forks source link

copy_and_paste code error #59

Closed so45jj45 closed 6 months ago

so45jj45 commented 6 months ago

Thank you for your appreciation of our research.

When I read your paper, I understood that it uses the copy-and-paste technique on images to create synthetic videos. It appears that this code operates within the copy_and_paste(self, labeled_data, unlabeled_data) function of the train_loop.py.

The part keep = iou_matrix.max(1)[0] < 0.0 seems to always return false, leading me to believe that the current code does not copy instances from other images but rather copies the existing target frame for training.

When changing it to keep = iou_matrix.max(1)[0] < 0.5, it results in an error. I would appreciate any assistance in resolving this issue.

[02/16 08:27:16 mask2former_video.data_video.build]: Using training sampler TrainingSampler [02/16 08:27:16 d2.data.common]: Serializing the dataset using: <class 'detectron2.data.common._TorchSerializedList'> [02/16 08:27:16 d2.data.common]: Serializing 100 elements to byte tensors and concatenating them all ... [02/16 08:27:16 d2.data.common]: Serialized dataset takes 0.69 MiB [02/16 08:27:16 d2.data.build]: Making batched data loader with batch_size=2 INFO: use AMPTrainer. copy_paste hyper-params: True 1.0 False [02/16 08:27:36 d2.checkpoint.detection_checkpoint]: [DetectionCheckpointer] Loading from /workspace/CutLER/videocutler/pretrain/cutler_m2f_rn50.pth ... [02/16 08:27:36 fvcore.common.checkpoint]: [Checkpointer] Loading from /workspace/CutLER/videocutler/pretrain/cutler_m2f_rn50.pth ... [02/16 08:27:37 d2.engine.train_loop]: Starting training from iteration 0 ERROR [02/16 08:28:22 d2.engine.train_loop]: Exception during training: Traceback (most recent call last): File "/workspace/CutLER/detectron2/detectron2/engine/train_loop.py", line 155, in train self.run_step() File "/workspace/CutLER/CutLER/videocutler/mask2former_video/engine/defaults.py", line 508, in run_step self._trainer.run_step() File "/workspace/CutLER/CutLER/videocutler/mask2former_video/engine/train_loop.py", line 374, in run_step data = self.copy_and_paste(copy.deepcopy(data[::-1]), data) File "/workspace/CutLER/CutLER/videocutler/mask2former_video/engine/train_loop.py", line 212, in copy_and_paste copied_instances.gt_masks = copied_masks File "/workspace/CutLER/detectron2/detectron2/structures/instances.py", line 62, in setattr self.set(name, val) File "/workspace/CutLER/detectron2/detectron2/structures/instances.py", line 78, in set assert ( AssertionError: Adding a field of length 2 to a Instances of length 1 [02/16 08:28:22 d2.engine.hooks]: Total training time: 0:00:44 (0:00:00 on hooks) [02/16 08:28:22 d2.utils.events]: iter: 0 lr: N/A max_mem: 171M Traceback (most recent call last): File "/opt/conda/lib/python3.8/runpy.py", line 194, in _run_module_as_main return _run_code(code, main_globals, None, File "/opt/conda/lib/python3.8/runpy.py", line 87, in _run_code exec(code, run_globals) File "/root/.vscode-server/extensions/ms-python.python-2024.0.1/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/main.py", line 39, in cli.main() File "/root/.vscode-server/extensions/ms-python.python-2024.0.1/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 430, in main run() File "/root/.vscode-server/extensions/ms-python.python-2024.0.1/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 284, in run_file runpy.run_path(target, run_name="main") File "/root/.vscode-server/extensions/ms-python.python-2024.0.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 321, in run_path return _run_module_code(code, init_globals, run_name, File "/root/.vscode-server/extensions/ms-python.python-2024.0.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 135, in _run_module_code _run_code(code, mod_globals, init_globals, File "/root/.vscode-server/extensions/ms-python.python-2024.0.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 124, in _run_code exec(code, run_globals) File "/workspace/CutLER/CutLER/videocutler/train_net_video.py", line 314, in launch( File "/workspace/CutLER/detectron2/detectron2/engine/launch.py", line 84, in launch main_func(*args) File "/workspace/CutLER/CutLER/videocutler/train_net_video.py", line 305, in main return trainer.train() File "/workspace/CutLER/CutLER/videocutler/mask2former_video/engine/defaults.py", line 498, in train super().train(self.start_iter, self.max_iter) File "/workspace/CutLER/detectron2/detectron2/engine/train_loop.py", line 155, in train self.run_step() File "/workspace/CutLER/CutLER/videocutler/mask2former_video/engine/defaults.py", line 508, in run_step self._trainer.run_step() File "/workspace/CutLER/CutLER/videocutler/mask2former_video/engine/train_loop.py", line 374, in run_step data = self.copy_and_paste(copy.deepcopy(data[::-1]), data) File "/workspace/CutLER/CutLER/videocutler/mask2former_video/engine/train_loop.py", line 212, in copy_and_paste copied_instances.gt_masks = copied_masks File "/workspace/CutLER/detectron2/detectron2/structures/instances.py", line 62, in setattr self.set(name, val) File "/workspace/CutLER/detectron2/detectron2/structures/instances.py", line 78, in set assert ( AssertionError: Adding a field of length 2 to a Instances of length 1

frank-xwang commented 6 months ago

Hi, as I've left Meta and no longer have the permissions to make direct changes to this repository, I've implemented the bug fix in my local fork available at: https://github.com/frank-xwang/CutLER. Please let me know if you encounter any additional issues. Thank you!

update: got the push permission. Have updated the codes. :-)