Training can get stuck in loop with Validation Error = N/A

[sliedes]

I realize I must be doing something really stupid here without realizing it, but I cannot get even the simple example in README.rst to work (everything in the example/ directory works, so my setup is presumably not broken).

Here's my code:

from sknn.mlp import Classifier, Layer
from sklearn.datasets import make_classification

X, y = make_classification(n_features=2, n_redundant=0, n_informative=2,
                           random_state=0, n_clusters_per_class=1)
nn = Classifier(
    layers=[
        Layer("Maxout", units=10, pieces=2),
        Layer("Softmax")],
    learning_rate=0.001,
    verbose=True)
nn.fit(X, y)

This program gets stuck in a loop, printing this output:

$ python3 test.py
Initializing neural network with 2 layers, 2 inputs and 2 outputs.
  - Dense: Maxout     Units:  10  
  - Dense: Softmax    Units:  2   

Training on dataset of 100 samples with 400 total size.
  - Early termination after 50 stable iterations.

Epoch    Validation Error      Time
-----------------------------------
    1           N/A            0.0s
    2           N/A            0.0s
    3           N/A            0.0s
    4           N/A            0.0s
    5           N/A            0.0s
    6           N/A            0.0s
    7           N/A            0.0s
...

I realize there must be a user error here, but it would probably not hurt to throw an error instead of getting stuck in such a state.

[alexjc]

Try setting n_iter or n_stable to terminate in finite amount of time. Also try passing in a valid_size to display validation error.

[sliedes]

Ah, yes, that helps. Thank you! Perhaps it might be worth detecting n_iter=n_stable=valid_size=None if they always lead to an infinite loop?

[alexjc]

Absolutely it is, that's why I left the ticket open :-) Thanks!

[bridgesra]

@alexjc Thanks, for the comments, this is helpful. I have a question about these parameters, and another more general question. Can you help with these two questions?

http://stackoverflow.com/questions/32958847/what-exactly-does-valid-size-f-stable-n-stable-do-in-scikit-neuralnetwork-fitt

http://stackoverflow.com/questions/32957564/configuration-of-neural-network-for-regression-using-scikit-neuralnetwork

[alexjc]

@bridgesra Will you post a new ticket with those specific questions? I don't want to mix the discussion here.

[bridgesra]

Sure! very detailed question formulated here https://github.com/aigamedev/scikit-neuralnetwork/issues/117