Evolution of word Embedding in NLP

[Praveenrajan27]

Title

Evolution of word Embedding in NLP

Description

Modeling natural language is a really complex problem because natural language doesn’t follow any mathematical construct. It is very abstract which makes it very hard to train language. In past there have been rule based solutions but it had limited performance in terms of automation Over time it became evident that if we want to solve NLP problems efficiently we have to treat it as a mathematical problem and from there started the journey of embeddings From representing text in a simple one hot encoded method to more advanced representation like vectors notation, we have come a long way. This talk takes a journey through how this representation known as the embedding have evolved over the years.

Duration

• [ ] 30 min
• [x] 45 min

Audience

Basic understanding of NLP would be good. This talk is for Beginner to Intermediate level

Outline

A detailed outline for your talk. The more detailed the better. (1000 words)

• Intro -

• Text analytics and how it is relevant today

• Challenges in working with NLP data - need to represent text as mathematical representation Why do need to represent text in mathematical format

• Early stage - representing text in mathematical space

  1. Early forms of representation using one hot encoding
  2. Measuring text relevancy using TF-IDF and how it improves representation quality
  3. Practical applications of word embedding leveraging TF-IDF

• Mid Stage - capturing relationship in words

  1. Intro of word2vec model Architectural overview - Skip gram and Continuous bag of words
  2. How word2vec started the legacy of pretrained models
  3. Limitations of word2vec model

• Current - Capturing contextual information in embeddings Intro to attention mechanism

  1. Overview of Attention based model - Bi,BERT
  2. Transfer Learning in NLP - ImageNet moment
  3. Architectural comparison between current state of the art NLP models - BERT, XLNet and GPT 1,2 Applications leveraging SOTA models

Additional notes

I am Data science Professional with VMware with 6+years of exp. I have completed my masters in software Engineering. My interests are in NLP and Language models

---

• [ ] Don't record this talk.

  Check this if you don't want your talk to be recorded.

[vinayak-mehta]

@Praveenrajan27 Are you available to present this talk on Nov 9?

[Praveenrajan27]

Yes I am!

[TrigonaMinima]

Hi @Praveenrajan27 will you be available for the talk on 29th Feb?

[Praveenrajan27]

Hi Shivam, Apologies just saw this message. May I do it for the next meetup?

On Wed, 19 Feb 2020, 12:33 Shivam Rana, notifications@github.com wrote:

Hi @Praveenrajan27 https://github.com/Praveenrajan27 will you be available for the talk on 29th Feb?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/pydatabangalore/talks/issues/24?email_source=notifications&email_token=AAYEMU74PYWC74EJ27DQTI3RDTKU5A5CNFSM4JDTQXPKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMGTDQY#issuecomment-588067267, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAYEMUYJJB22IUTZ5SXZJH3RDTKU5ANCNFSM4JDTQXPA .