2.6rc1

[C-K-Loan]

NLU 2.6

• Added 100+ new models from Spark NLP 2.6
  • New YAKE algorithm
  • New Multi-Class Classifier algorithm
• Improved outputs for Chunk level components
• Integrated removal of IOB prefixes of NER tags
• Integrated light pipeline which yields 10x speedup for predictions
• Easy and Copy pastable moel configs via pipe.print_info()
• N new Notebooks
• Recycling of Pandas indexes for predicting. No more ID columns, just pandas indexes.
• Up to 10x Speed improvement with light pipeline leverage from Spark NLP

Lots of new Demos (37 new Notebooks)

• Named Entity Recognition (NER) -NER pretrained on ONTO Notes -NER pretrained on CONLL

• Part of speech (POS)

  • POS pretrained on ANC dataset

• Classifiers

  • Unsupervised Keyword Extraction with YAKE
  • Toxic Text Classifier
  • Twitter Sentiment Classifier
  • Movie Review Sentiment Classifier
  • Sarcasm Classifier
  • 50 Class Questions Classifier
  • 20 Class Languages Classifier
  • Fake News Classifier
  • E2E Classifier
  • Cyberbullying Classifier
  • Spam Classifier

• Word and Sentence Embeddings

  • BERT Word Embeddings and T-SNE plotting
  • BERT Sentence Embeddings and T-SNE plotting
  • ALBERT Word Embeddings and T-SNE plotting
  • ELMO Word Embeddings and T-SNE plotting
  • XLNET Word Embeddings and T-SNE plotting
  • ELECTRA Word Embeddings and T-SNE plotting
  • COVIDBERT Word Embeddings and T-SNE plotting
  • BIOBERT Word Embeddings and T-SNE plotting
  • GLOVE Word Embeddings and T-SNE plotting
  • USE Sentence Embeddings and T-SNE plotting

• Depenency Parsing -Untyped Dependency Parsing -Typed Dependency Parsing

• Text Pre Processing and Cleaning

  • Tokenization
  • Stopwords removal
  • Stemming
  • Lemmatization
  • Normalizing
  • Spellchecking
  • Sentence Detecting

• Chunkers -N Gram -Entity Chunking

• Matchers -Date Matcher