error when load Osprey-724K/osprey_detail_description.json

[koda-11]

Hello, i am trying to reproduce osprey now. when i tried to train stage 3, there are some errors as below when load Osprey-724K/osprey_detail_description.json. """"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" Traceback (most recent call last): File "/home/user/LMM/Osprey/osprey/train/train_mem.py", line 13, in train() File "/home/user/LMM/Osprey/osprey/train/train.py", line 696, in train data_module = make_multitask_data_module(tokenizer=tokenizer, File "/home/user/LMM/Osprey/osprey/datasets/data_modules.py", line 58, in make_multitask_data_module train_dataset = build_osprey_dataset(dataset_config, File "/home/user/LMM/Osprey/osprey/datasets/data_modules.py", line 75, in build_osprey_dataset temp_dataset = build_osprey_dataset(cfg, tokenizer=tokenizer, data_args=data_args, *kwargs) File "/home/user/LMM/Osprey/osprey/datasets/data_modules.py", line 136, in build_osprey_dataset dataset = OspreyDetailedDescription( File "/home/user/LMM/Osprey/osprey/datasets/osprey_724k.py", line 234, in init super().init(tokenizer, data_args, ann_file, img_prefix) File "/home/user/LMM/Osprey/osprey/datasets/osprey_724k.py", line 53, in init super().init(tokenizer, data_args, ann_file, img_prefix) File "/home/user/LMM/Osprey/osprey/datasets/stage2_data.py", line 28, in init self.data_infos = self.load_annotations(ann_file) File "/home/user/LMM/Osprey/osprey/datasets/osprey_724k.py", line 258, in load_annotations answer = re.findall(r"<.>:\ (.*)", ann['description'][i])[0] IndexError: list index out of range """""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""

please check this. thanks

[xin-ran-w]

I met the same problem, the IndexError is caused by the region number not being equal to the num of the description. When I check the annotation, I found the description annotation of COCO_train2014_000000001688.jpg is wrong. By correct the format you can fix this error. Hope this will help you.

{'id': 1688, 'file_name': 'COCO_train2014_000000001688.jpg', 'description': '"description": ["<region1>: A black and brown dog is comfortably resting on a blue cushion, which is part of a chair. You can notice the dog\'s brown paws placed lightly on the cushion offering a warm touch to the scene.", "<region2>: In front of the dog and on the dining table, there are several decorative dog treats and cakes. One of them looks like a big white bone, while others have dots on them. There is also a dotted item in front of the treats on the left side. ", "<region3>: Among the other treats on the table, two small dog treats in the shape of bones stand out. They appear to have writing on them, suggesting they might be personalized or special in some way. ", "<region4>: The structure of a wooden chair is subtly visible in the background. Its brown color subtly contrasts with the blue cushion where the dog is seated, underlining the cozy and homely atmosphere of the scene."]', 'annotation': [{'bbox': [16.67, 362.5, 623.33, 173.33], 'segmentation': [[536.67, 375.83, 605.0, 400.83, 640.0, 457.5, 635.0, 509.17, 575.0, 522.5, 518.33, 505.83, 445.0, 514.17, 413.33, 514.17, 396.67, 535.83, 376.67, 505.83, 346.67, 487.5, 311.67, 475.83, 268.33, 475.83, 256.67, 462.5, 260.0, 425.83, 325.0, 437.5, 326.67, 437.5, 386.67, 432.5, 405.0, 454.17, 440.0, 459.17, 453.33, 437.5, 450.0, 402.5, 505.0, 389.17, 510.0, 365.83], [206.67, 362.5, 146.67, 372.5, 111.67, 390.83, 40.0, 422.5, 18.33, 449.17, 16.67, 490.83, 33.33, 502.5, 70.0, 487.5, 140.0, 482.5, 198.33, 472.5, 200.0, 434.17, 183.33, 417.5, 185.0, 400.83]]}, {'bbox': [33.39, 471.4, 291.73, 103.01], 'segmentation': [[35.75, 551.61, 35.75, 552.39, 40.47, 498.92, 67.99, 471.4, 287.38, 471.4, 325.12, 476.12, 307.82, 528.8, 310.97, 568.12, 296.03, 568.12, 277.16, 554.75, 274.8, 538.24, 259.86, 531.16, 234.69, 528.8, 209.53, 566.55, 192.23, 564.19, 192.23, 564.19, 178.86, 574.41, 104.16, 559.47, 93.15, 544.53, 70.35, 542.96, 64.06, 550.03, 33.39, 555.54]]}, {'bbox': [56.14, 528.15, 241.01, 99.54], 'segmentation': [[275.24, 555.07, 275.87, 538.79, 262.09, 529.4, 241.44, 528.15, 222.65, 543.8, 218.27, 555.07, 208.26, 575.73, 203.25, 598.26, 202.62, 617.67, 217.65, 625.81, 247.7, 623.93, 271.48, 613.91, 290.26, 599.52, 297.15, 577.6, 292.14, 567.59, 283.38, 558.82], [190.1, 626.43, 201.37, 617.67, 202.62, 604.52, 198.24, 601.39, 205.13, 593.88, 208.26, 579.48, 204.5, 570.72, 196.99, 565.08, 186.97, 565.71, 175.7, 576.35, 136.26, 563.21, 101.21, 560.08, 93.7, 547.56, 78.05, 539.42, 59.89, 551.94, 68.66, 570.72, 56.14, 581.36, 57.39, 593.88, 64.9, 603.27, 76.17, 603.9, 87.44, 604.52, 92.44, 596.39, 109.35, 602.65, 141.9, 609.53, 165.06, 613.29, 168.19, 623.3, 181.96, 627.69]]}, {'bbox': [171.03, 0.69, 415.18, 380.69], 'segmentation': [[180.69, 373.1, 180.69, 215.86, 171.03, 106.9, 183.45, 43.45, 223.45, 0.69, 537.93, 0.69, 566.9, 32.41, 586.21, 79.31, 575.17, 177.24, 568.28, 231.03, 562.76, 381.38, 520.0, 367.59, 524.14, 341.38, 531.03, 344.14, 510.34, 316.55, 480.0, 275.17, 480.0, 257.24, 493.79, 247.59, 506.21, 208.97, 495.17, 184.14, 481.38, 178.62, 459.31, 131.72, 420.69, 100.0, 386.21, 90.34, 362.76, 93.1, 342.07, 86.21, 296.55, 116.55, 263.45, 148.28, 244.14, 195.17, 256.55, 195.17, 234.48, 226.9, 242.76, 225.52, 242.76, 233.79, 213.79, 253.1, 202.76, 283.45, 188.97, 312.41, 187.59, 342.76, 197.24, 368.97, 183.45, 374.48]]}], 'height': 640, 'width': 640}

[LiWentomng]

Thanks. @xin-ran-w Sorry that we uploaded the incorrect version of this file before. @koda-11 We have updated the osprey_detail_description.json, please kindly re-download the updated one.

[xin-ran-w]

OK, thank you for the in-time reply😀, I will download this file.

[koda-11]

thank you for answer!