TromboneDavies / PolarOps

0 stars 0 forks source link

Get rid of duplicate comments in training data #61

Open divilian opened 3 years ago

akochans commented 3 years ago

https://www.interviewqs.com/ddi-code-snippets/drop-duplicate-rows-pandas https://stackoverflow.com/questions/12497402/python-pandas-remove-duplicates-by-columns-a-keeping-the-row-with-the-highest df = df.drop_duplicates(subset='column you're looking for duplicates in', keep="first")