search

PyImageSearch / Deep-Learning-for-Computer-Vision-with-Python

2 stars 1 forks source link

chapter 9 - regularization.py code #2

Closed chopsuey5000 closed 3 years ago

chopsuey5000 commented 3 years ago

Issue

I'm running the regularization.py code with 10 iterations and I get a warning saying (ConvergenceWarning: Maximum number of iteration reached before convergence. Consider increasing max_iter to improve the fit.):

python regularization.py --dataset ../datasets/animals
[INFO] loading images...
[INFO] processed 500/3000
[INFO] processed 1000/3000
[INFO] processed 1500/3000
[INFO] processed 2000/3000
[INFO] processed 2500/3000
[INFO] processed 3000/3000
[INFO] training model with 'None' penalty
C:\Users\chops\anaconda3\envs\DL4CV_SB\lib\site-packages\sklearn\linear_model\_stochastic_gradient.py:577: ConvergenceWarning: Maximum number of iteration reached before convergence. Consider increasing max_iter to improve the fit.
  ConvergenceWarning)
[INFO] 'None' penalty accuracy: 49.33%
[INFO] training  model with 'l1' penalty
C:\Users\chops\anaconda3\envs\DL4CV_SB\lib\site-packages\sklearn\linear_model\_stochastic_gradient.py:577: ConvergenceWarning: Maximum number of iteration reached before convergence. Consider increasing max_iter to improve the fit.
  ConvergenceWarning)
[INFO] 'l1' penalty accuracy: 45.47%
[INFO] training model with 'l2' penalty
C:\Users\chops\anaconda3\envs\DL4CV_SB\lib\site-packages\sklearn\linear_model\_stochastic_gradient.py:577: ConvergenceWarning: Maximum number of iteration reached before convergence. Consider increasing max_iter to improve the fit.
  ConvergenceWarning)
[INFO] 'l2' penalty accuracy: 50.53%

Then I increased the number of iterations until 70 and the warning have disappeared but I cannot see any accuracy improvement using L1 or L2 regularizations as the textbook mentions.

>python regularization.py --dataset ../datasets/animals
[INFO] loading images...
[INFO] processed 500/3000
[INFO] processed 1000/3000
[INFO] processed 1500/3000
[INFO] processed 2000/3000
[INFO] processed 2500/3000
[INFO] processed 3000/3000
[INFO] training model with 'None' penalty
[INFO] 'None' penalty accuracy: 51.33%
[INFO] training model with 'l1' penalty
[INFO] 'l1' penalty accuracy: 47.47%
[INFO] training model with 'l2' penalty
[INFO] 'l2' penalty accuracy: 48.53%

Could please tell what is the issue?

Thanks in advance,

sayakpaul commented 3 years ago

Can you one of our VMs and see if this issue persists?

chopsuey5000 commented 3 years ago

Hi Sayak,

I've tried with the VM and the problem persists.

With 10 iterations the output is:

$ python regularization.py --dataset ../datasets/animals
[INFO] loading images...
[INFO] processed 500/3000
[INFO] processed 1000/3000
[INFO] processed 1500/3000
[INFO] processed 2000/3000
[INFO] processed 2500/3000
[INFO] processed 3000/3000
[INFO] tranining model with 'None' penalty
[INFO] 'None' penalty accuracy: 53.73%
[INFO] tranining model with 'l1' penalty
[INFO] 'l1' penalty accuracy: 53.33%
[INFO] tranining model with 'l2' penalty
[INFO] 'l2' penalty accuracy: 54.53%
(dl4cv) pyimagesearch@pyimagesearch-dl4cv:~/Documents/deep_learning/c9_optimization_methods$ python regularization.py --dataset ../datasets/animals
[INFO] loading images...
[INFO] processed 500/3000
[INFO] processed 1000/3000
[INFO] processed 1500/3000
[INFO] processed 2000/3000
[INFO] processed 2500/3000
[INFO] processed 3000/3000
[INFO] tranining model with 'None' penalty
/home/pyimagesearch/.virtualenvs/dl4cv/lib/python3.6/site-packages/sklearn/linear_model/stochastic_gradient.py:561: ConvergenceWarning: Maximum number of iteration reached before convergence. Consider increasing max_iter to improve the fit.
  ConvergenceWarning)
[INFO] 'None' penalty accuracy: 45.07%
[INFO] tranining model with 'l1' penalty
/home/pyimagesearch/.virtualenvs/dl4cv/lib/python3.6/site-packages/sklearn/linear_model/stochastic_gradient.py:561: ConvergenceWarning: Maximum number of iteration reached before convergence. Consider increasing max_iter to improve the fit.
  ConvergenceWarning)
[INFO] 'l1' penalty accuracy: 43.07%
[INFO] tranining model with 'l2' penalty
/home/pyimagesearch/.virtualenvs/dl4cv/lib/python3.6/site-packages/sklearn/linear_model/stochastic_gradient.py:561: ConvergenceWarning: Maximum number of iteration reached before convergence. Consider increasing max_iter to improve the fit.
  ConvergenceWarning)
[INFO] 'l2' penalty accuracy: 51.87%

And with 100 iterations the output is:

$ python regularization.py --dataset ../datasets/animals
[INFO] loading images...
[INFO] processed 500/3000
[INFO] processed 1000/3000
[INFO] processed 1500/3000
[INFO] processed 2000/3000
[INFO] processed 2500/3000
[INFO] processed 3000/3000
[INFO] tranining model with 'None' penalty
[INFO] 'None' penalty accuracy: 53.73%
[INFO] tranining model with 'l1' penalty
[INFO] 'l1' penalty accuracy: 53.33%
[INFO] tranining model with 'l2' penalty
[INFO] 'l2' penalty accuracy: 54.53%
sayakpaul commented 3 years ago

I see. What happens when you up it 1000?

chopsuey5000 commented 3 years ago

Same result as with 100

$ python regularization.py --dataset ../datasets/animals
[INFO] loading images...
[INFO] processed 500/3000
[INFO] processed 1000/3000
[INFO] processed 1500/3000
[INFO] processed 2000/3000
[INFO] processed 2500/3000
[INFO] processed 3000/3000
[INFO] tranining model with 'None' penalty
[INFO] 'None' penalty accuracy: 53.73%
[INFO] tranining model with 'l1' penalty
[INFO] 'l1' penalty accuracy: 53.33%
[INFO] tranining model with 'l2' penalty
[INFO] 'l2' penalty accuracy: 54.53%
sayakpaul commented 3 years ago

You meant 1000, right?

chopsuey5000 commented 3 years ago

Yes

sayakpaul commented 3 years ago

I took a look at the original results again:

image

They don't vary that much from what you have gotten. The increased performance might just have been due to the stochasticity that is induced in the algorithm under consideration.