search

ddbourgin / numpy-ml

Machine learning, in numpy
https://numpy-ml.readthedocs.io/
GNU General Public License v3.0
15.26k stars 3.7k forks source link

Naive bayes #68

Closed sfsf9797 closed 3 years ago

sfsf9797 commented 3 years ago

Implement Gaussian naive Bayes Class and Basic documentation.

Refer to the formula of Gaussian naive Bayes from the notes

compare the performance with the gaussianNB from sci-kit learn.

ddbourgin commented 3 years ago

Amazing! At first glance this looks great, @sfsf9797 ! I'm pretty slammed with work right now, but am going to have a look at this this weekend.

ddbourgin commented 3 years ago

Hey @sfsf9797 - Thanks for the PR! I just had a more thorough look and committed a few changes. Brief summary:

  1. I moved this under the linear_models module rather than keeping it as a single hanging model.
  2. I think there might have been a few bugs in the original log likelihood calc. I've committed what I believe is the correct version, but please go through it and make sure you agree.
  3. I expanded the unit test you included -- comparing model accuracies is a good start (thanks!), but I think it actually masked some problems with the implementation. In particular, testing on multiple random cases revealed that there were mismatches in accuracy between sklearn and the current implementation, and comparing the actual class probabilities (rather than just the predictions) revealed a bug in the log posterior calculation.
  4. I expanded the documentation to provide a better overview of the model.

Please feel free to make adjustments or ask questions. Once we both agree on the implementation and are happy with the model performance, I'm happy to merge.

sfsf9797 commented 3 years ago

Hi, thank you so much for all the feedbacks, I will go through all these and get back to you this weekend.

sfsf9797 commented 3 years ago

sorry, I am kind of busy these few weeks, but I will get you back latest by another 2 weeks times.

sfsf9797 commented 3 years ago

Hi @ddbourgin thanks for correcting my implementation. I have learnt a lot from you. I am pretty satisfied with the model after a few round of checking. Lastly, Thank you a lot for the all the comments!

ddbourgin commented 3 years ago

Awesome, merged! Thanks for the PR @sfsf9797 :)