bjascob / LemmInflect

A python module for English lemmatization and inflection.
MIT License
260 stars 25 forks source link

Pronouns support #15

Closed cfytrok closed 2 years ago

cfytrok commented 2 years ago

Please add pronouns support. lemminflect.getAllLemmas('his') returns {}

bjascob commented 2 years ago

I'd suggest adding logic to your code so that if lemminflect.getAllLemmas() returns '{}' to just use the original word. This method does a lookup and the raw NIH dateset which for some reason doesn't appear to contain any pronouns (except "I").

I will add the following (derived from... https://www.thefreedictionary.com/List-of-pronouns.htm)

        pronouns = ['all', 'another', 'any', 'anybody', 'anyone', 'anything', 'aught', 'both', 'each',
                    'everybody', 'everyone', 'everything', 'he', 'her', 'hers', 'herself', 'him',
                    'himself', 'his', 'idem', 'it', 'its', 'itself', 'many', 'me', 'my', 'myself',
                    'neither', 'none', 'our', 'ours', 'ourself', 'ourselves', 'she', 'some', 'somebody',
                    'someone', 'something', 'such', 'suchlike', 'that', 'thee', 'their', 'theirs',
                    'theirself', 'theirselves', 'them', 'themself', 'themselves', 'these', 'they',
                    'thine', 'this', 'those', 'thou', 'thy', 'thyself', 'us', 'we', 'what', 'whatever',
                    'whatnot', 'whether', 'which', 'whichever', 'whichsoever', 'who', 'whoever', 'whom',
                    'whomever', 'whomso', 'whomsoever', 'whose', 'whosever', 'whosesoever', 'whoso',
                    'whosoever', 'ye', 'yon', 'you', 'your', 'yours', 'yourself', 'yourselves']
bjascob commented 2 years ago

43f9358a960dcc89de5429437e690af6b3bf8896 adds the above list to lemma_overrides.csv and adds test cases for each in LemmatizerTests.py

Need to review and push update to pypi (version already updated to 0.2.3)

bjascob commented 2 years ago

Updates added in Release 0.2.3