Warning in NeuralNetTorch

[mastoffel]

When running param_search=True:

em = AutoEmulate()
em.setup(X, y, n_jobs=-2, model_subset=["NeuralNetTorch"], param_search=True)
best = em.compare()

We sometimes get an error:

ValueError: Input contains NaN.

I think this is because some of the search_space parameters don't work for some reason.

[bryanlimy]

Setting error_score="raise" to RandomizedSearchCV shows a more complete error log. It would appears the SGD can sometimes explode depending on the learning rate.

Somewhat related question on stackoverflow https://stackoverflow.com/questions/62674258/randomized-search-value-error-input-contains-nan-infinity-or-a-value-too-large.

We can remove SGD but it seems like it achieved the best results in some cases, especially with smaller datasets.

[mastoffel]

ok, good work catching this. That makes sense and seems ok to me. I wouldn't remove SGD for this.

I guess this makes this another logging issue related to #68 #126. We would want this to potentially be logged to a file, but not necessarily printed to the user except they explicitly want it.

[kallewesterling]

I guess this makes this another logging issue related to https://github.com/alan-turing-institute/autoemulate/issues/68 https://github.com/alan-turing-institute/autoemulate/issues/126. We would want this to potentially be logged to a file, but not necessarily printed to the user except they explicitly want it.

@mastoffel Don't forget to add a comment on #68 or #126 about this so we can pick it up later :)