search

dongfang91 / Generate-and-Rank-ConNorm

Generate-and-Rank Framework with Semantic Type Regularization for Biomedical Concept Normalization
25 stars 4 forks source link

Question regarding ranking #5

Open Mahmedturk opened 4 years ago

Mahmedturk commented 4 years ago

Hi,

I just want o understand how does ranker assigns a rank to candidates? I mean do you treat is as a classification task and if its actually a candidate assign it 1 otherwise zero?Or is the probability is the rank here? What is your final layer in the ranker architecture? As in below "Lucene ranks the correct concept 4th in its list. The BERT ranker is able to compare “an abdominal wall hernia” to “Hernia of abdominal wall” and recognize that as a better match than the other options, re-assigning it to rank 1"

What is the rank here? In terms of IR i know there is a relevance score attached to each candidate but in terms of BERT how is the rank calculated?

dongfang91 commented 4 years ago

The final layer of the ranker is the softmax classification layer, please see below: image

During inference time, we select the candidate whose probability is the highest (produced by the softmax layer).

Mahmedturk commented 4 years ago

Thanks! I am working on a similar project and use huggingface transformers to do pair-wise classification. But I do not know how the softmax probablities can be printed? As I am new to this field. I use BERT for sequence classification, can you give me an idea how to print probabilities produced by softmax?