search

DevSinghSachan / ssl_text_classification

Semi Supervised Learning for Text-Classification
83 stars 15 forks source link

The principle and motivation, why the objective functions are design to related to word embeddings? #5

Open guotong1988 opened 5 years ago

guotong1988 commented 5 years ago

@DevSinghSachan Thank you！

sndnyang commented 4 years ago

... Hope my answer is what you want. Why do the objective functions compute gradients over the word embeddings?

Answer: Because it's adversarial training. You should know the concept of adversarial examples on CV/images. Then using this idea/concept to compute the adversarial examples/perturbations for text inputs. But if you compute a perturbation based on one-hot encoding, it's weird to have perturbations for [0 1 0 0] one-hot encoding. But perturbations on word embedding may make sense.