search

microsoft / scene_graph_benchmark

image scene graph generation benchmark
MIT License
382 stars 86 forks source link

When I extract image features with VinVL, the AssertionError occur #24

Open PengPeixi opened 3 years ago

PengPeixi commented 3 years ago

hello! When I extract image features with VinVL by the command :

python tools/test_sg_net.py --config-file sgg_configs/vgattr/vinvl_x152c4.yaml TEST.IMS_PER_BATCH 2 MODEL.WEIGHT models/vinvl/vinvl_vg_x152c4.pth MODEL.ROI_HEADS.NMS_FILTER 1 MODEL.ROI_HEADS.SCORE_THRESH 0.2 DATA_DIR "../maskrcnn-benchmark-1/datasets1" TEST.IGNORE_BOX_REGRESSION True MODEL.ATTRIBUTE_ON True TEST.OUTPUT_FEATURE True

I get the traceback:

Traceback (most recent call last): File "tools/test_sg_net.py", line 129, in main() File "tools/test_sg_net.py", line 106, in main data_loaders_val = make_data_loader(cfg, is_train=False, is_distributed=distributed) File "/content/drive/MyDrive/VinVL/scene_graph_benchmark/scene_graph_benchmark/maskrcnn_benchmark/data/build.py", line 170, in make_data_loader datasets = build_dataset(cfg, transforms, DatasetCatalog, is_train or is_for_period) File "/content/drive/MyDrive/VinVL/scene_graph_benchmark/scene_graph_benchmark/maskrcnn_benchmark/data/build.py", line 45, in build_dataset cfg, dataset_name, factory_name, is_train File "/content/drive/MyDrive/VinVL/scene_graph_benchmark/scene_graph_benchmark/maskrcnn_benchmark/data/datasets/utils/config_args.py", line 7, in config_tsv_dataset_args assert op.isfile(full_yaml_file) AssertionError

I guess the reason is that the file flickr30k/tsv/flickr30k.yaml is not exist, so where can I find this yaml file?

I also try to delete the line : TEST: ("flickr30k/tsv/flickr30k.yaml",) in the file: vinvl_x152c4.yaml, but after run the code, there is nothing output.

hanxiaotian commented 3 years ago

could you please download the visual genome dataset and change TEST: ("flickr30k/tsv/flickr30k.yaml",) to TEST: ("("visualgenome/test_danfeiX_relation.yaml",), then start the command like this python tools/test_sg_net.py --config-file sgg_configs/vgattr/vinvl_x152c4.yaml TEST.IMS_PER_BATCH 2 MODEL.WEIGHT models/vinvl/vinvl_vg_x152c4.pth MODEL.ROI_HEADS.NMS_FILTER 1 MODEL.ROI_HEADS.SCORE_THRESH 0.2 DATA_DIR "path_to_the_datadir_visualgenome" TEST.IGNORE_BOX_REGRESSION True MODEL.ATTRIBUTE_ON True TEST.OUTPUT_FEATURE True

PengPeixi commented 3 years ago

could you please download the visual genome dataset and change TEST: ("flickr30k/tsv/flickr30k.yaml",) to TEST: ("("visualgenome/test_danfeiX_relation.yaml",), then start the command like this python tools/test_sg_net.py --config-file sgg_configs/vgattr/vinvl_x152c4.yaml TEST.IMS_PER_BATCH 2 MODEL.WEIGHT models/vinvl/vinvl_vg_x152c4.pth MODEL.ROI_HEADS.NMS_FILTER 1 MODEL.ROI_HEADS.SCORE_THRESH 0.2 DATA_DIR "path_to_the_datadir_visualgenome" TEST.IGNORE_BOX_REGRESSION True MODEL.ATTRIBUTE_ON True TEST.OUTPUT_FEATURE True

Thanks for your reply! I want to do the feature extraction on the images from the ImagetNet. Do you mean that I should put those images and the file 'test_danfeiX_relation.yaml' into the same directory whose path is "path_to_the_datadir_visualgenome"?

hanxiaotian commented 3 years ago

could you please download the visual genome dataset and change TEST: ("flickr30k/tsv/flickr30k.yaml",) to TEST: ("("visualgenome/test_danfeiX_relation.yaml",), then start the command like this python tools/test_sg_net.py --config-file sgg_configs/vgattr/vinvl_x152c4.yaml TEST.IMS_PER_BATCH 2 MODEL.WEIGHT models/vinvl/vinvl_vg_x152c4.pth MODEL.ROI_HEADS.NMS_FILTER 1 MODEL.ROI_HEADS.SCORE_THRESH 0.2 DATA_DIR "path_to_the_datadir_visualgenome" TEST.IGNORE_BOX_REGRESSION True MODEL.ATTRIBUTE_ON True TEST.OUTPUT_FEATURE True

Thanks for your reply! I want to do the feature extraction on the images from the ImagetNet. Do you mean that I should put those images and the file 'test_danfeiX_relation.yaml' into the same directory whose path is "path_to_the_datadir_visualgenome"?

yes, the image tsv file and ymal file should be under same directory.

gsrivas4 commented 3 years ago

@PengPeixi were you able to get the features for Imagenet images, Visual Genome images or any other image dataset? I am running code similar to your, but I am getting following error. Did you encounter similar errors?

@hanxiaotian I have tried generating features for Visual Genome dataset based on your instructions. This is the command that I am running:

python tools/test_sg_net.py --config-file sgg_configs/vgattr/vinvl_x152c4_edited.yaml TEST.IMS_PER_BATCH 2 MODEL.WEIGHT pretrained_model/vinvl_vg_x152c4.pth MODEL.ROI_HEADS.NMS_FILTER 1 MODEL.ROI_HEADS.SCORE_THRESH 0.2 DATA_DIR datasets TEST.IGNORE_BOX_REGRESSION True MODEL.ATTRIBUTE_ON True TEST.OUTPUT_FEATURE True

I have edited vinvl_x152c4_edited.yaml file to add feature to TEST.TSV_SAVE_SUBSET. I have also made following modification: DATASETS.TEST: "visualgenome/test_danfeiX_relation.yaml"

However, I am getting following error:

  File "tools/test_sg_net.py", line 197, in <module>
    main()
  File "tools/test_sg_net.py", line 193, in main
    run_test(cfg, model, args.distributed, model_name)
  File "tools/test_sg_net.py", line 72, in run_test
    save_predictions=cfg.TEST.SAVE_PREDICTIONS,
  File "/home/default/persistent_drive/image_captioning/vinvl/scene_graph_benchmark/maskrcnn_benchmark/engine/inference.py", line 298, in inference
    relation_on=cfg.MODEL.RELATION_ON,
  File "/home/default/persistent_drive/image_captioning/vinvl/scene_graph_benchmark/maskrcnn_benchmark/engine/inference.py", line 212, in convert_predictions_to_tsv
    tsv_writer(gen_rows(), os.path.join(output_folder, output_tsv_name))
  File "/home/default/persistent_drive/image_captioning/vinvl/scene_graph_benchmark/maskrcnn_benchmark/structures/tsv_file_ops.py", line 42, in tsv_writer
    for value in values:
  File "/home/default/persistent_drive/image_captioning/vinvl/scene_graph_benchmark/maskrcnn_benchmark/engine/inference.py", line 140, in gen_rows
    features = prediction.get_field('box_features').numpy()
  File "/home/default/persistent_drive/image_captioning/vinvl/scene_graph_benchmark/maskrcnn_benchmark/structures/bounding_box.py", line 43, in get_field
    return self.extra_fields[field]
KeyError: 'box_features'

Could you please suggest me what I am doing wrong.

gsrivas4 commented 2 years ago

@hanxiaotian @PengPeixi I could resolve the error. The issue was with my setup.

fengmq commented 2 years ago

@hanxiaotian @PengPeixi I could resolve the error. The issue was with my setup.

I want to extract image features with VinVL,where can I find "test_vgoi6_clipped.yaml"?

gsrivas4 commented 2 years ago

@fengmq I am using test_danfeiX_relation.yaml instead. AFAIR, I got this yaml file when I downloaded visual genome dataset features, etc. from here - https://github.com/microsoft/scene_graph_benchmark#perform-training. Qualitatively the captions look good.

Jennifer-6 commented 2 years ago

could you please download the visual genome dataset and change TEST: ("flickr30k/tsv/flickr30k.yaml",) to TEST: ("("visualgenome/test_danfeiX_relation.yaml",), then start the command like this python tools/test_sg_net.py --config-file sgg_configs/vgattr/vinvl_x152c4.yaml TEST.IMS_PER_BATCH 2 MODEL.WEIGHT models/vinvl/vinvl_vg_x152c4.pth MODEL.ROI_HEADS.NMS_FILTER 1 MODEL.ROI_HEADS.SCORE_THRESH 0.2 DATA_DIR "path_to_the_datadir_visualgenome" TEST.IGNORE_BOX_REGRESSION True MODEL.ATTRIBUTE_ON True TEST.OUTPUT_FEATURE True

Thanks for your reply! I want to do the feature extraction on the images from the ImagetNet. Do you mean that I should put those images and the file 'test_danfeiX_relation.yaml' into the same directory whose path is "path_to_the_datadir_visualgenome"?

yes, the image tsv file and ymal file should be under same directory.

hi ~ l want to extrace the nocaps feature,how can l get the yaml and tsv file?