search

automl / AutoFolio

Automated Algorithm Selection with Hyperparameter Optimization
https://www.automl.org
BSD 2-Clause "Simplified" License
36 stars 11 forks source link

RuntimeWarning: invalid value encountered in double_scalars #7

Closed cychong9228 closed 6 years ago

cychong9228 commented 6 years ago

Hi Marius,

I had encountered a problem with calculating the Normalized Score.

AutoFolio-master/autofolio/validation/validate.py:86: RuntimeWarning: invalid value encountered in double_scalars self.logger.info("Normalized Score: %.4f" %( ( par10 - oracle) / (sbs - oracle))) INFO:Stats:Normalized Score: nan

Initially I thought that my input data contains negative values or extremely small values causing the error. However, I noticed that the same issue also occurs when running the toy_example data.

mlindauer commented 6 years ago

Hi,

I'm in Stockholm for IJCAI right now and cannot look in detail into it. However, the warning sounds like that sbs == oracle s.t. AutoFolio divides by 0 and the string casting to float raises a warning, since nan is not a float. Could you please check in you logs whether this is the issue. You should see oracle and sbs scores some lines above this error.

Cheers, Marius

cychong9228 commented 6 years ago

Hi Marius,

Yup you are right. It was because sbs == oracle.

INFO:PairwiseClassifier:Fit PairwiseClassifier with <class 'autofolio.selector.classifiers.random_forest.RandomForest'> INFO:AutoFolio:Predict on Test INFO:Stats:Number of instances: 1 INFO:Stats:Average Solution Quality: 2.0000 INFO:Stats:Oracle: 2.0000 INFO:Stats:Single Best: 2.0000 /AutoFolio-master/autofolio/validation/validate.py:86: RuntimeWarning: invalid value encountered in double_scalars self.logger.info("Normalized Score: %.4f" %( ( par10 - oracle) / (sbs - oracle))) INFO:Stats:Normalized Score: nan

Regards, Chong

mlindauer commented 6 years ago

Great! So, we can close this issue?

Cheers, Marius