SKTBrain / KoBERT

Korean BERT pre-trained cased (KoBERT)
Apache License 2.0
1.3k stars 368 forks source link

[BUG] BERTClassifier drop_out rate์—๋Ÿฌ #78

Closed Danden1 closed 2 years ago

Danden1 commented 2 years ago

๐Ÿ› Bug

๋„ค์ด๋ฒ„ ๋ฆฌ๋ทฐ์ฝ”๋“œ์—์„œ BERTClassifier์˜ dr_rate๋ฅผ None์ด๋‚˜ 0.0์œผ๋กœ ํ•˜๋ฉด ์˜ค๋ฅ˜๊ฐ€ ๋ฐœ์ƒํ•ฉ๋‹ˆ๋‹ค.

To Reproduce

image

๋˜๋Š”

image

๋กœ ์„ค์ •ํ•˜๋ฉด,

image

์ด์™€ ๊ฐ™์€ ์˜ค๋ฅ˜๊ฐ€ ๋œน๋‹ˆ๋‹ค.

if self.dr_rate:
        out = self.dropout(pooler)
else:
        out = pooler

return self.classifier(out)

๋กœ ๊ณ ์ณ์•ผ ํ•  ๊ฒƒ ๊ฐ™์Šต๋‹ˆ๋‹ค.

bage79 commented 2 years ago

@Danden1 ์˜ค๋ฅ˜๋กœ ์ธํ•ด, ๋ถˆํŽธ์„ ๋“œ๋ ค์„œ ์ฃ„์†กํ•ฉ๋‹ˆ๋‹ค.

๋ง์”€ํ•˜์‹  ๋‚ด์šฉ์„ ๋ฐ˜์˜ํ•˜์—ฌ ์ปค๋ฐ‹ํ•˜์˜€์Šต๋‹ˆ๋‹ค. https://github.com/SKTBrain/KoBERT/commit/e1f2f37055e7460d8427f6912579c0162cb69831

๊ฐ์‚ฌํ•ฉ๋‹ˆ๋‹ค. ๐Ÿ˜