search

monologg / JointBERT

Pytorch implementation of JointBERT: "BERT for Joint Intent Classification and Slot Filling"
Apache License 2.0
662 stars 186 forks source link

Issue using shared softmax layer. #1

Closed alikhodadoost closed 4 years ago

alikhodadoost commented 4 years ago

Issue using shared softmax layer, the paper has mentioned using multiplication of slot filling softmax's https://github.com/monologg/JointBERT/blob/d6e06f2a8910b412e3c3ad84ee533615afdde160/model.py#L52

The original paper code: https://github.com/MahmoudWahdan/dialog-nlu/blob/7e34ebb3c370abe1044464b9a15c0b445a0a2384/models/joint_bert.py#L65

monologg commented 4 years ago

Hello:) Sry for the late reply.

I've checked the paper again, and I couldn't find that paper has mentioned about multiplication of softmax. It seems like you might confused with loss function and objective function.

I've saw the line 66 of the code, and this one look like using the hidden state corresponding to the first sub-token as input, not the multiplication of slot filling softmax.

As I see the github that you mentioned, it also used the same method for loss function (Cross Entropy) that I used. The only difference between mine and dialog-nlu github is that I use 1 * intent_loss + 1 * slot_loss and other uses 1 * intent_loss + 3 * slot_loss. Also the github you mentioned doesn't look like the original author's code.

My assumption might be wrong. If I'm wrong, just let me know:)

monologg commented 4 years ago

@alikhodadoost I'll close this issue. If you have any additional issue, let me know:)