about "generate_nsd_results.py" question

[ha1ha2hahaha]

I want to know the “generate_nsd_results.py” should train the model ? when I use your pretrained model，Thank to the #2 ，I can run the “generate_has_rels_results.py”. but ,I can't run the “generate_nsd_results.py”, I found this code required train the model. I compared the two codes “generate_has_rels_results.py” and “generate_nsd_results.py”，the “generate_nsd_results.py” don‘t have the model.eval() I should add this code in “generate_nsd_results.py” ？ thank you!

[muktilin]

model.eval () is on line 143. In addition, the model is not being trained, just extracting information from all the training sets. You need to replace the checkpoints/motif-precls-exmp with the trained motifs model, or download from https://drive.google.com/file/d/1X_hXO3O6Z_VJYyiJcuRHvCgZYxB2ZmxN/view

[ha1ha2hahaha]

Thank you very much！ and，I want to know the

rel_ids.extend([str(imageid) + '' + str(x) for x in range(results_dict_load.item()[image_id].get_field('pred_rel_rep').shape[0])])

on line 180 in the "generate_has_rels_results.py"，because, the model output don‘t have the “pre_rel_rep". I also found that "generate_has_rels_results.py" use the make_roi_relation_confidence_post_processor as the post_processor in the "inference.py", which don't add the "pre_rel_rep". ` rel_pair_idx = rel_pair_idx[sorting_idx] rel_class_prob = rel_class_prob[sorting_idx] rel_labels = rel_class[sorting_idx] scores = confidence[sorting_idx] boxlist.add_field('rel_pair_idxs', rel_pair_idx) # (#rel, 2) boxlist.add_field('pred_rel_scores', rel_class_prob) # (#rel, #rel_class) boxlist.add_field('pred_rel_labels', rel_labels) # (#rel, ) boxlist.add_field('ood_score', scores)

should have fields : rel_pair_idxs, pred_rel_class_prob, pred_rel_labels, pred_labels, pred_scores

Note

TODO Kaihua: add a new type of element, which can have different length with boxlist (similar to field, except that once

the boxlist has such an element, the slicing operation should be forbidden.)

it is not safe to add fields about relation into boxlist!

results.append(boxlist) return results `

I really want to know about this great work and look forward to your reply.

[ha1ha2hahaha]

Because if I add the "pre_rel_rep"，I found that in the generate_nsd_results.py on line 322 ,the len(original_rel_ids) is so longer than the len(original_rel_labels) . I would very much like your help！

[muktilin]

make_roi_relation_confidence_post_processor is used to obtain the confidence score c, thus, we use it to train MotifConfidencePredictor. But in generate_has_rels_results.py, we just want to extract information from original dataset, we use the MotifPredictor rather than MotifConfidencePredictor, the processor is make_roi_relation_post_processor. You just need to match the number of relationships in the dataset. I saved "pred_rel_rep" field when I wrote the code to generate the results_dict. You can replace it with get_field('pred_rel_labels') in generate_has_rels_results.py. Besides, the "pre_rel_rep" denotes the extracted features used for clustering. I just updated generate_has_rels_results.py, you can try the new version, it will now extract faster.