KeyError

[YCA-eng]

wonderful work! When I ran the chat.py file, a problem occurred, but I couldn't find where the problem was. Can you provide some help?

My version is as follows： torch=1.13.1+cu117 mmcv=2.1.0 mmdet=3.3.0 mmengine=0.10.2

Traceback (most recent call last): File "/public2/home//project/3D/Lenna/chat.py", line 143, in main(sys.argv[1:]) File "/public2/home//project/3D/Lenna/chat.py", line 99, in main dino_input = load_as_mmdet(image_path, caption=g_dino_caption) File "/public2/home//project/3D/Lenna/utils/create_test_annfile_mmdet.py", line 44, in load_as_mmdet mmdet_pipeline = Compose(mmdet_pipline_cfg) File "/public2/home//anaconda3/envs/lenna/lib/python3.10/site-packages/mmengine-0.10.2-py3.10.egg/mmengine/dataset/base_dataset.py", line 38, in init transform = TRANSFORMS.build(transform) File "/public2/home//anaconda3/envs/lenna/lib/python3.10/site-packages/mmengine-0.10.2-py3.10.egg/mmengine/registry/registry.py", line 570, in build return self.build_func(cfg, args, kwargs, registry=self) File "/public2/home/***/anaconda3/envs/lenna/lib/python3.10/site-packages/mmengine-0.10.2-py3.10.egg/mmengine/registry/build_functions.py", line 100, in build_from_cfg raise KeyError( KeyError: 'FixScaleResize is not in the mmengine::transform registry. Please check whether the value of FixScaleResize is correct or it was registered as expected. More details can be found at https://mmengine.readthedocs.io/en/latest/advanced_tutorials/config.html#import-the-custom-module'

[weifei7]

Thanks for your feedback! You can simply change the key "FixScaleResize" to "Resize" or pull our latest commit. This change has little impact on model's performance. FYI, here is the package version we used: conda.txt

[YCA-eng]

Thanks for your reply, the above problem has been solved, but another problem arises：(Same question as above)

Traceback (most recent call last): File "/public2/home//project/3D/Lenna/chat.py", line 145, in main(sys.argv[1:]) File "/public2/home//project/3D/Lenna/chat.py", line 101, in main dino_input = load_as_mmdet(image_path, caption=g_dino_caption) File "/public2/home//project/3D/Lenna/utils/create_test_annfile_mmdet.py", line 44, in load_as_mmdet mmdet_pipeline = Compose(mmdet_pipline_cfg) File "/public2/home//anaconda3/envs/lenna/lib/python3.10/site-packages/mmengine/dataset/base_dataset.py", line 38, in init transform = TRANSFORMS.build(transform) File "/public2/home//anaconda3/envs/lenna/lib/python3.10/site-packages/mmengine/registry/registry.py", line 570, in build return self.build_func(cfg, args, kwargs, registry=self) File "/public2/home/***/anaconda3/envs/lenna/lib/python3.10/site-packages/mmengine/registry/build_functions.py", line 100, in build_from_cfg raise KeyError( KeyError: 'PackDetInputs is not in the mmengine::transform registry. Please check whether the value of PackDetInputs is correct or it was registered as expected. More details can be found at https://mmengine.readthedocs.io/en/latest/advanced_tutorials/config.html#import-the-custom-module'

I tried to configure the same environment as you. When I installed mmdet=3.1.0, the following error occurred:

(lenna) yuchunan@d6f5edefdcce:~/project/3D/Lenna$ python chat.py Traceback (most recent call last): File "/public2/home//project/3D/Lenna/chat.py", line 9, in from model.Lenna import LennaForCausalLM File "/public2/home//project/3D/Lenna/model/Lenna.py", line 7, in from .grounding_dino.build_gdino_mmdet import build_gdino File "/public2/home//project/3D/Lenna/model/grounding_dino/build_gdino_mmdet.py", line 1, in from .grounding_dino import GroundingDINO File "/public2/home//project/3D/Lenna/model/grounding_dino/grounding_dino.py", line 12, in from mmdet.models.layers.transformer.grounding_dino_layers import (GroundingDinoTransformerDecoder, GroundingDinoTransformerEncoder) ModuleNotFoundError: No module named 'mmdet.models.layers.transformer.grounding_dino_layers'

This problem may be caused by an low version.

[YCA-eng]

This is how I solved it:

def load_as_mmdet(file_name, caption=None):
    from mmdet.utils.setup_env import register_all_modules 
    register_all_modules(True)
    assert caption != None
    text = tuple([caption])
    mmdet_pipline_cfg = [
        dict(type='LoadImageFromFile', backend_args=None),
        dict(type='Resize', scale=(800, 1333), keep_ratio=True),
        dict(type='LoadAnnotations', with_bbox=True),
        dict(
            type='PackDetInputs',
            meta_keys=('img_id', 'img_path', 'ori_shape', 'img_shape',
                    'scale_factor', 'text', 'custom_entities'))
    ]
    mmdet_pipeline = Compose(mmdet_pipline_cfg)
    ann_file = create_annfile(file_name)
    g_dino_data_info = get_data_info(ann_file['images'][0], ann_file['annotations'], text)
    g_dino_data = mmdet_pipeline(g_dino_data_info)
    return g_dino_data

https://github.com/open-mmlab/mmdetection/issues/10900#issuecomment-1806997515

[weifei7]

@YCA-eng Sorry for not replying in time, may I ask if you can successfully run through the inference code now?

[Zekun525]

我是这样解决的：

def load_as_mmdet(file_name, caption=None):
    from mmdet.utils.setup_env import register_all_modules 
    register_all_modules(True)
    assert caption != None
    text = tuple([caption])
    mmdet_pipline_cfg = [
        dict(type='LoadImageFromFile', backend_args=None),
        dict(type='Resize', scale=(800, 1333), keep_ratio=True),
        dict(type='LoadAnnotations', with_bbox=True),
        dict(
            type='PackDetInputs',
            meta_keys=('img_id', 'img_path', 'ori_shape', 'img_shape',
                    'scale_factor', 'text', 'custom_entities'))
    ]
    mmdet_pipeline = Compose(mmdet_pipline_cfg)
    ann_file = create_annfile(file_name)
    g_dino_data_info = get_data_info(ann_file['images'][0], ann_file['annotations'], text)
    g_dino_data = mmdet_pipeline(g_dino_data_info)
    return g_dino_data

open-mmlab/mmdetection#10900 （评论）

这样修改后出现这样的错误： File "/home/user/anaconda3/envs/lenna/lib/python3.10/site-packages/mmdet/models/layers/transformer/grounding_dino_layers.py", line 238, in forward layer_id].self_attn_cfg.num_heads AttributeError: 'dict' object has no attribute 'num_heads'