Improve code quality based on report from @sourcery-ai (generate_features.py) - Githubissues

neomatrix369 / nlp_profiler

A simple NLP library allows profiling datasets with one or more text columns. When given a dataset and a column name containing text data, NLP Profiler will return either high-level insights or low-level/granular statistical information about the text in that column.

Other

243 stars 37 forks source link

Improve code quality based on report from @sourcery-ai (generate_features.py) #28

Open neomatrix369 opened 4 years ago

neomatrix369 commented 4 years ago

Describe the limitations

As per the report from @sourcery-ai in https://github.com/neomatrix369/nlp_profiler/pull/27#issuecomment-709986856 and https://github.com/neomatrix369/nlp_profiler/pull/34#issuecomment-710026749

Do the following:

reduce method length
make method(s), modules less complex (i.e. nlp_profiler/granular_features.py and other modules)
improve run-time working memory (maybe use memory-profiling)

Recommendation(s):

Extract out complex expressions
reduce working memory