search

neomatrix369 / nlp_profiler

A simple NLP library allows profiling datasets with one or more text columns. When given a dataset and a column name containing text data, NLP Profiler will return either high-level insights or low-level/granular statistical information about the text in that column.
Other
241 stars 37 forks source link

Apply docstrings to functions across the library source code #56

Open neomatrix369 opened 3 years ago

neomatrix369 commented 3 years ago

To be able to merge a pull request, there are a few checks:

Goal or purpose of the PR

Primarily using docly and applying it to all the source files of the whole library. A number of source files were missed due to errors generated during parsing by docly issue(s) to be raised with founders as part of the feedback program.

Changes implemented in the PR

Ran docly-gen on all the source files using the shell script apply-docstrings-to-functions.sh

Applied manual changes to a number of docstrings to make them more readable. And also captures issues with docly when trying to parse a handful of source files -- as a separate commit to make it easier to compare the before and after actions of using docly to generate docstrings.

docly-issues.txt contains a list of all the errors that occurred when tryingto parse the source file as a result no docstrings were generated for them.

To make docly a bit easier to use, there could an additional CLI arg i.e. --quiet and/or -y to allow defaulting to applying docstrings that it suggests. Also the --help could be a bit more clearer (maybe also provide an example or two).

codecov[bot] commented 3 years ago

Codecov Report

Merging #56 (f5a9867) into master (a610f0e) will not change coverage. The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff            @@
##            master       #56   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           22        22           
  Lines          379       379           
  Branches        54        54           
=========================================
  Hits           379       379
Impacted Files Coverage Δ
nlp_profiler/core.py 100.00% <ø> (ø)
nlp_profiler/generate_features/__init__.py 100.00% <ø> (ø)
...erate_features/parallelisation_methods/__init__.py 100.00% <ø> (ø)
nlp_profiler/granular_features/__init__.py 100.00% <ø> (ø)
nlp_profiler/granular_features/alphanumeric.py 100.00% <ø> (ø)
nlp_profiler/granular_features/chars_and_spaces.py 100.00% <ø> (ø)
nlp_profiler/granular_features/dates.py 100.00% <ø> (ø)
nlp_profiler/granular_features/duplicates.py 100.00% <ø> (ø)
nlp_profiler/granular_features/emojis.py 100.00% <ø> (ø)
nlp_profiler/granular_features/non_alphanumeric.py 100.00% <ø> (ø)
... and 8 more

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update a610f0e...7e0bea0. Read the comment docs.

neomatrix369 commented 3 years ago

@sourcery-ai in the PR I'm improving the docstrings of each of the function, I think your parser/analyser may want to consider it as an improvement and not consider it as an increase in method-length (so NOT a decrease in code quality as such) -- zero functionality has been changed by this process, and thus zero code has been changed, only comments have been added.

And in general, a narration on why there was code quality increase or decrease in the PR can also help -- if that can be ascertained via the diff code.

sourcery-ai[bot] commented 3 years ago

Sourcery Code Quality Report

❌  Merging this PR will decrease code quality in the affected files by 0.15%.

Quality metrics Before After Change
Complexity 1.49 ⭐ 1.49 ⭐ 0.00
Method Length 24.39 ⭐ 25.02 ⭐ 0.63 👎
Working memory 7.13 🙂 7.13 🙂 0.00
Quality 84.94% ⭐ 84.79% ⭐ -0.15% 👎
Other metrics Before After Change
Lines 440 772 332
Changed files Quality Before Quality After Quality Change
nlp_profiler/core.py 51.83% 🙂 51.72% 🙂 -0.11% 👎
nlp_profiler/generate_features/init.py 70.70% 🙂 70.54% 🙂 -0.16% 👎
nlp_profiler/generate_features/parallelisation_methods/init.py 87.10% ⭐ 86.96% ⭐ -0.14% 👎
nlp_profiler/granular_features/init.py 56.89% 🙂 56.74% 🙂 -0.15% 👎
nlp_profiler/granular_features/alphanumeric.py 95.01% ⭐ 94.84% ⭐ -0.17% 👎
nlp_profiler/granular_features/chars_and_spaces.py 93.68% ⭐ 93.51% ⭐ -0.17% 👎
nlp_profiler/granular_features/dates.py 88.24% ⭐ 88.08% ⭐ -0.16% 👎
nlp_profiler/granular_features/duplicates.py 82.43% ⭐ 82.27% ⭐ -0.16% 👎
nlp_profiler/granular_features/emojis.py 90.50% ⭐ 90.34% ⭐ -0.16% 👎
nlp_profiler/granular_features/non_alphanumeric.py 95.01% ⭐ 94.84% ⭐ -0.17% 👎
nlp_profiler/granular_features/noun_phase_count.py 82.51% ⭐ 82.51% ⭐ 0.00%
nlp_profiler/granular_features/numbers.py 95.01% ⭐ 94.84% ⭐ -0.17% 👎
nlp_profiler/granular_features/punctuations.py 92.34% ⭐ 92.17% ⭐ -0.17% 👎
nlp_profiler/granular_features/sentences.py 96.50% ⭐ 96.34% ⭐ -0.16% 👎
nlp_profiler/granular_features/stop_words.py 92.30% ⭐ 92.18% ⭐ -0.12% 👎
nlp_profiler/granular_features/words.py 95.01% ⭐ 94.84% ⭐ -0.17% 👎
nlp_profiler/high_level_features/init.py 67.79% 🙂 67.63% 🙂 -0.16% 👎
nlp_profiler/high_level_features/grammar_quality_check.py 87.45% ⭐ 87.30% ⭐ -0.15% 👎

Here are some functions in these files that still need a tune-up:

File Function Complexity Length Working Memory Quality Recommendation
nlp_profiler/core.py apply_text_profiling 4 ⭐ 134 😞 16 ⛔ 51.72% 🙂 Try splitting into smaller methods. Extract out complex expressions
nlp_profiler/granular_features/init.py apply_granular_features 0 ⭐ 75 🙂 33 ⛔ 56.74% 🙂 Extract out complex expressions
nlp_profiler/high_level_features/init.py apply_high_level_features 0 ⭐ 43 ⭐ 17 ⛔ 67.63% 🙂 Extract out complex expressions
nlp_profiler/generate_features/init.py generate_features 2 ⭐ 63 🙂 11 😞 70.54% 🙂 Extract out complex expressions
nlp_profiler/granular_features/noun_phase_count.py gather_nouns 2 ⭐ 59 ⭐ 10 😞 73.06% 🙂 Extract out complex expressions

Legend and Explanation

The emojis denote the absolute quality of the code:

The 👍 and 👎 indicate whether the quality has improved or gotten worse with this pull request.

Please see our documentation here for details on how these metrics are calculated.

We are actively working on this report - lots more documentation and extra metrics to come!

Let us know what you think of it by mentioning @sourcery-ai in a comment.

neomatrix369 commented 3 years ago

@shubhadeep let's discuss improvements and feedback on this PR that I used docly