How to use RepMet as feature extractor?

[duynn912]

Hi @jshtok,

I am sorry for bothering you again.

I want to get embedding features which is the output of DML embedding module for pointed objects. Following your code, I see that after RepMet inferences an image, it will output 2000 proposals and 2000 embedding features corresponding to these 2000 proposals. My problem is where I can show RepMet the groundtruth area to get the embedding features instead of the areas of 2000 proposals in your code.

I hope to hear from you soon! Thank you very much! Best regards, Duynn

[jshtok]

Hi Duynn,

Please look at the function def get_model_output() in RepMet_routines.py. There you can see how the feature vectors (psp_final_embed_output) are returned.

Regards, Joseph

On Wed, Mar 18, 2020 at 11:54 AM duynn912 notifications@github.com wrote:

Hi @jshtok https://github.com/jshtok,

I am sorry for bothering you again.

I want to get embedding features which is the output of DML embedding module for pointed objects. Following your code, I see that after RepMet inferences an image, it will output 2000 proposals and 2000 embedding features corresponding to these 2000 proposals. My problem is where I can show RepMet the groundtruth area to get the embedding features instead of the areas of 2000 proposals in your code.

I hope to hear from you soon! Thank you very much! Best regards, Duynn

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/jshtok/RepMet/issues/28, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACOBU6TEBYMB4HLOLFWGO7LRICK5HANCNFSM4LONI4TQ .

[duynn912]

Hi @jshtok,

Thank you for your quick respond! However, where dose RepMet_routines.py locate in your repository? I can't locate this file.

[jshtok]

Hi, sorry for the confusion. In your code, you can see the usage example in the gen_reps() function, in few_shot_benchmark.py. The line

psp_embed = out_data[0]['psp_final_embed_output'].asnumpy() # rois_feats. [#rois, #dims]

is how to extract the features.

Regards, Joseph

On Thu, Mar 19, 2020 at 3:49 AM duynn912 notifications@github.com wrote:

Hi @jshtok https://github.com/jshtok,

Thank you for your quick respond! However, where dose RepMet_routines.py locate in your repository? I can't locate this file.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/jshtok/RepMet/issues/28#issuecomment-600944459, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACOBU6WO2UCYPHUX76DDQ43RIF227ANCNFSM4LONI4TQ .

[duynn912]

Hi @jshtok,

Yes, I start from this line psp_embed = out_data[0]['psp_final_embed_output'].asnumpy() . However, I see that this line output 2000 embedding features corresponding to 2000 object proposals (rois_embed = out_data[0]['rois_output'].asnumpy()), and the following codes behind this line will choose which embedding features overlapped with gt box over threshold=0.7. My problem is how I can provide the exact gt box to extract features from it instead of object proposals and threshold? I debug several times But I can't find way to do that!

[jshtok]

I see. In principle, you should feed the ROI pooling layer the GT boxes instead of the proposals produced by the RPN, but I have never attempted to to that. Please see the \fpn\symbols\resnet_v1_101_fpn_dcn_rcnn_oneshot_v3.py file where the graph of the model is defined, specifically the function get_symbol() in it. You can follow the data flow along layers and maybe figure out how to feed the GT boxes to ROI pooling layer.

Regards, Joseph

On Thu, Mar 19, 2020 at 12:05 PM duynn912 notifications@github.com wrote:

Hi @jshtok https://github.com/jshtok,

Yes, I start from this line psp_embed = out_data[0]['psp_final_embed_output'].asnumpy() . However, I see that this line output 2000 embedding features corresponding to 2000 object proposals (rois_embed = out_data[0]['rois_output'].asnumpy()), and the following codes behind this line will choose which embedding features overlapped with gt box over threshold=0.7. My problem is how I can provide the exact gt box to extract features from it instead of object proposals and threshold? I debug several times But I can't find way to do that!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/jshtok/RepMet/issues/28#issuecomment-601093222, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACOBU6V2C2NQETJLL2F7QHTRIHU63ANCNFSM4LONI4TQ .