How to evaluate the model with custom images?

[Jeba-create]

First of all, Thank you so much for such an amazing work. I have two things to clarify

1. I have download some images from the google and put it on the same folder 'images' replacing the existing test folder with the new one, but still, on executing the program, it fetches the images from the original test data and not from the new one. Even if i give only the subset of images, it is fetching the whole. Could we test the model over new set of test images which is not present in the database, if so where should i do the changes?
2. I have tried to visualize the images or save the images with bounding box inserted setting save_image=True of the test.py script, but it is throwing an error "NotImplementedError". Could i visualize the test image with detected bounding boxes and labels?

[bbepoch]

I guess, you want to:

1) Do inference on one image, i.e., input an image file path, and output the result of the image. 2) Visualize the original image in dataset, or visualize the detection result.

For 1), If you study how the dataset works, it is not difficult to implement. But to make it easier, I am preparing a script to give an examle to inference on one given image in 2-3 days.

For 2), I suggest you to update your repo to the latest. The latest version also supports V-COCO and HOI-A datasets.

I hope it will help.

[Jeba-create]

Thank you so much for your reply

1. Actually my first query is Could i evaluate the model with some 100 new test images which is not present in either V-COCO and HOI-A datasets?. Because, I have tried out this with some new test images which is from some other datasets, but the inference is done only on the test images of hico dataset. (i have created a folder named test2015 and put some 100 images , but when i execute test.py script, the inference is done only for 9756 images which is present in the test2015 folder of hico dataset, even i have tried out by shifting those test2015 folder out of the working directory )

2. The second one is could i visualize the test image in which the predicted bounding box drawn and with the display of its confidence score. I think this is the portion that does this stuff, but its not working and throwing an error "NotImplementedError" , i have set save_image=True

3. I want to make inference on a single image. I think that script is under process.

[bbepoch]

1, You are supposed to read hico.py. Placing the images in the folder is not enough. https://github.com/bbepoch/HoiTransformer/blob/master/datasets/hico.py#L590

2, There was a bug for that version to visualize. You can print(image_id), and find it is an illegal file name. As you can see, it should be a file name. The bug has been fixed in the latest version.

[Jeba-create]

In line 590, annotation_file = './data/hico/hico_test_remake.odgt' is given, this file contains the information such as image id, subject id, interaction and bbox. But if we inputted the model with unseen images, which is not yet annotated, could this model works or even for the test images we need annotations (.odgt files)?. To compute evaluation metrics, we need groundtruth values, but in case if i want only the predicted scores and the detection areas(bbox) and not the quatitative values such as mAP, recall values as in finalreport.txt files, what should i do or could this be done?

Thanks in advance

[Jeba-create]

Sorry for asking this repeatedly, i have final suggestion. I am planning to use this HOI as one of the module in my project. I want to work with some real time examples. So, if i want to input some unseen image into the model, it is necessary to create the annotation file for that.

Thanks in advance

[bbepoch]

This may help. It gives an example to test on images.

https://github.com/bbepoch/HoiTransformer/blob/master/test_on_images.py

[Jeba-create]

Thank you so much for your great effort. Everything is working fine. I could find whatever I needed.

[leijue222]

This may help. It gives an example to test on images.

https://github.com/bbepoch/HoiTransformer/blob/master/test_on_images.py

Is there a pre-trained HOI Transformer model provided?

[Jeba-create]

Yes, the pre-trained model is provided. "/data/detr_coco/download_model.sh"

[leijue222]

Yes, the pre-trained model is provided. "/data/detr_coco/download_model.sh"

That‘s the DETR model. I mean the HOI model trained by main.py.

[bbepoch]

You can find them here,