[ML] Model Selection

[chorongi]

Paper reading on the following related work. Searching for usable tutorials & github repos for finetuning.

Text Generation Papers

http://rowanzellers.com/advice/ https://arxiv.org/abs/2004.03607

GPT-3 350GB

https://link.springer.com/content/pdf/10.1007/s11023-020-09548-1.pdf

BERT-16GB

https://arxiv.org/pdf/1903.10318.pdf

RoBERTa

https://arxiv.org/pdf/1907.11692.pdf

Distill BERT- 8 GB

https://arxiv.org/pdf/1910.01108.pdf https://ysu1989.github.io/courses/au20/cse5539/DistilBERT.pdf

Text summarization Papers

https://link.springer.com/content/pdf/10.1007/s41870-021-00853-1.pdf https://link.springer.com/content/pdf/10.1007/s11042-018-5749-3.pdf https://dl.acm.org/doi/abs/10.1145/3297156.3297232 https://dl.acm.org/doi/abs/10.1145/3297156.3297232 https://anubhav20057.medium.com/step-by-step-guide-abstractive-text-summarization-using-roberta-e93978234a90

Profanity Detection Papers

https://ieeexplore.ieee.org/abstract/document/9358624?casa_token=mDXveXEEfUcAAAAA:RlNm9VQL4NFmxDyGgu3QddpwuDxzyxz0hxqwEKhxuDJaISU4Me-Qh4KMzNThnTEeZbDeZg http://ceur-ws.org/Vol-2826/T2-23.pdf (HASOC)

BERT FineTune

https://skimai.com/tutorial-how-to-fine-tune-bert-for-summarization/ https://towardsdatascience.com/extractive-summarization-using-bert-966e912f4142 https://medium.com/analytics-vidhya/text-summarization-using-bert-gpt2-xlnet-5ee80608e961 https://github.com/dmmiller612/bert-extractive-summarizer https://arxiv.org/abs/1906.04165

[chorongi]

Successfully fine-tuned on sample Amazon Reviews dataset with RoBERTa model.