unixcoder clone detection explainability

[yakirba]

Hi,

I am facing challenges in explaining how unixcoder/clone-detection/java model makes its predictions.

I would appreciate any guidance or resources on how to perform model explainability for this type of model. Specifically, I have tried techniques such SHAP and also tried to plot some mid layer outputs manually, but no significant intel appeared in any method.

example of encoder output from a pair of code snippets:

Thank you for your help in advance!

[guoday]

Sorry. I also don't know how to explain.

[yakirba]

ok thanks a lot! another Q regard clone detection: how come all clone detection implementations averages the last layer of transformers? don't you lose the contextual information doing so? have you tried adding another learned layer (dense/rnn/transformer/etc.)? thanks again