Random input feature dropping curve, model selection visualization [issue #1024]

[charlesbmi]

Summary

This PR addresses https://github.com/DistrictDataLabs/yellowbrick/issues/1204, which requested a feature-dropping-curve (also known as a neuron dropping curve in neural decoding).

I have made the following changes:

1. Added a module to yellowbrick.model_selection called dropping_curve. It is largely based on yellowbrick.model_selection.validation_curve

Sample Code and Plot

from sklearn.naive_bayes import MultinomialNB
from sklearn.preprocessing import OneHotEncoder, LabelEncoder

from yellowbrick.datasets import load_game
from yellowbrick.model_selection.dropping_curve import dropping_curve

# Load Connect-4 game data
X, y = load_game()
X_enc = OneHotEncoder().fit_transform(X)
le = LabelEncoder()
y_enc = le.fit_transform(y)

dropping_curve(
    MultinomialNB(),
    X_enc,
    y_enc,
    feature_sizes=np.linspace(0.05, 1, 20),
)

TODOs and questions

Still to do:

• Unit tests

Questions for the @DistrictDataLabs/team-oz-maintainers:

• This code generates ~30 additional blank figures when I run it through a Python script. Is that expected?

CHECKLIST

• [ ] Is the commit message formatted correctly?
• [ ] Have you noted the new functionality/bugfix in the release notes of the next release?

• [x] Included a sample plot to visually illustrate your changes?
• [x] Do all of your functions and methods have docstrings?
• [x] Have you added/updated unit tests where appropriate?
• [x] Have you updated the baseline images if necessary?
• [x] Have you run the unit tests using pytest?
• [x] Is your code style correct (are you using PEP8, pyflakes)?
• [ ] Have you documented your new feature/functionality in the docs?

• [ ] Have you built the docs using make html?

[bbengfort]

@charlesincharge Happy New Year! And thank you for opening this PR into Yellowbrick! @lwgray @pdamodaran @Kautumn06 would any of you be interested in doing the code review for this?

[pdamodaran]

@bbengfort - apologies for the delay in responding. I can review this PR next weekend.

[bbengfort]

Thanks @pdamodaran!

[codecov[bot]]

Codecov Report

Merging #1206 (67566cc) into develop (6fb2e9b) will increase coverage by 0.02%. The diff coverage is 92.30%.

@@             Coverage Diff             @@
##           develop    #1206      +/-   ##
===========================================
+ Coverage    90.45%   90.48%   +0.02%     
===========================================
  Files           91       92       +1     
  Lines         5135     5200      +65     
===========================================
+ Hits          4645     4705      +60     
- Misses         490      495       +5     

Impacted Files	Coverage Δ
yellowbrick/model_selection/dropping_curve.py	92.18% <92.18%> (ø)
yellowbrick/model_selection/__init__.py	100.00% <100.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 6fb2e9b...67566cc. Read the comment docs.

[lwgray]

@charlesincharge what version of matplotlib did you use when developing this feature?

[charlesbmi]

@lwgray I used matplotlib 3.5.0

[lwgray]

@charlesincharge Thanks for contributing such a wonderful visualizer. I have spent quite a bit of time working on it because of conflicts with our recent upgrade to Sci-kit 1.0. To help us get this up and merged, would you mind doing the documentation? You can read more about it here https://www.scikit-yb.org/en/latest/contributing/developing_visualizers.html#documentation

Here is a recent PR that has docs added https://github.com/DistrictDataLabs/yellowbrick/pull/1189

[charlesbmi]

Awesome, thanks for all the help! Hope this is useful for everyone!

[lwgray]

@charlesincharge This is a wonderful visualizer. Do you think you can do the docs?

[charlesbmi]

Thanks! Yes, I'll write some docs!

[lwgray]

@charlesincharge Can you reference https://github.com/DistrictDataLabs/yellowbrick/issues/1235 in your new PR?