search

openai / gpt-2

Code for the paper "Language Models are Unsupervised Multitask Learners"
https://openai.com/blog/better-language-models/
Other
22.57k stars 5.53k forks source link

A probable tweak for creation of word vectors ? #254

Open Prototype-001 opened 4 years ago

Prototype-001 commented 4 years ago

While creation of the word vectors do you account for the polarity of the sentence?

For example : 'You are good' 'You are not bad'

Here, good & bad have been used equally in the context of 'you' & 'are'. This might lead to them having rather similar word vectors. But since these words are antonyms of one another, they shouldn't be used interchangeably in the same context.