MLP regression constant values

[ghost]

Hello,

I'm trying to train a multi layer perceptron for non linear regression on a dataset but it keeps giving me this error :

Warning message:
In nominalTrainWorkflow(x = x, y = y, wts = weights, info = trainInfo,  :
  There were missing values in resampled performance measures.

I tried doing it again with a R dataset to see if my data was the problem but I keep getting the error and i have no idea why.

I already tried adding or removing tuneGrid, trControl, even linOut. It will always give me constant results for some reasons.

        library(datasets)
        library(MASS)
        library(caret)
        DP = caret::createDataPartition(Boston$medv, p=0.75, list = F)

        train = Boston[DP,]
        test = Boston[-DP,]
        colnames(train) = colnames(Boston)
        colnames(test) = colnames(Boston)

        mlp = caret::train(medv ~., data = Boston, method = "mlp", trControl = trainControl(method = "cv", number = 3),
                           tuneGrid = expand.grid(size = 1:3), linOut = T, metric = "RMSE")

        Yp = caret::predict.train(mlp, test[,1:13])

R version 3.4.2 (2017-09-28)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows >= 8 x64 (build 9200)

Matrix products: default

locale:
[1] LC_COLLATE=French_Belgium.1252  LC_CTYPE=French_Belgium.1252    LC_MONETARY=French_Belgium.1252
[4] LC_NUMERIC=C                    LC_TIME=French_Belgium.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] RSNNS_0.4-9     Rcpp_0.12.13    MASS_7.3-47     caret_6.0-77    ggplot2_2.2.1   lattice_0.20-35 mRMRe_2.0.7    
[8] igraph_1.1.2    survival_2.41-3

loaded via a namespace (and not attached):
 [1] tidyselect_0.2.3   purrr_0.2.4        reshape2_1.4.2     kernlab_0.9-25     splines_3.4.2      colorspace_1.3-2  
 [7] stats4_3.4.2       yaml_2.1.14        prodlim_1.6.1      rlang_0.1.4        ModelMetrics_1.1.0 foreign_0.8-69    
[13] glue_1.2.0         withr_2.1.0        bindrcpp_0.2       foreach_1.4.3      bindr_0.1          plyr_1.8.4        
[19] dimRed_0.1.0       lava_1.5.1         robustbase_0.92-8  stringr_1.2.0      timeDate_3042.101  munsell_0.4.3     
[25] gtable_0.2.0       recipes_0.1.1      codetools_0.2-15   psych_1.7.8        parallel_3.4.2     class_7.3-14      
[31] DEoptimR_1.0-8     broom_0.4.3        scales_0.5.0       ipred_0.9-6        CVST_0.2-1         mnormt_1.5-5      
[37] stringi_1.1.6      dplyr_0.7.4        RcppRoll_0.2.2     ddalpha_1.3.1      grid_3.4.2         tools_3.4.2       
[43] magrittr_1.5       lazyeval_0.2.1     tibble_1.3.4       tidyr_0.7.2        DRR_0.0.2          pkgconfig_2.0.1   
[49] Matrix_1.2-11      lubridate_1.7.1    gower_0.1.2        assertthat_0.2.0   iterators_1.0.8    R6_2.2.2          
[55] rpart_4.1-11       sfsmisc_1.1-1      nnet_7.3-12        nlme_3.1-131       compiler_3.4.2  

[topepo]

Is a warning (not an error) and commonly occurs if a model predicts the same value across all of the holdout samples so that there is no variance in the predictions. The consequence is that the R² cannot be computed. That looks like it is the case here.

[ghost]

Alright thanks. But initially i didn't want to build a model with no variance in the predictions (doesn't make sense looking at the data), this has happened to me quite a few times now and i don't understand how to get a working prediction model. Could you elaborate a little bit on why does the model gets to think the predictions are constant ?

[topepo]

Sorry for the delay.

The issue is that you should center and scale the predictors before the model. The preProc option can be used to do that:

> library(datasets)
> library(MASS)
> library(caret)
Loading required package: lattice
Loading required package: ggplot2
> DP = caret::createDataPartition(Boston$medv, p = 0.75, list = F)
> 
> train = Boston[DP, ]
> test = Boston[-DP, ]
> colnames(train) = colnames(Boston)
> colnames(test) = colnames(Boston)
> 
> set.seed(244)
> mlp = caret::train(
+   medv ~ .,
+   data = Boston,
+   method = "mlp",
+   trControl = trainControl(
+     method = "cv",
+     number = 3,
+     savePredictions = "final"
+   ),
+   preProc = c("center", "scale"),
+   tuneGrid = expand.grid(size = 1:3),
+   linOut = TRUE,
+   metric = "RMSE"
+ )
Warning message:
In nominalTrainWorkflow(x = x, y = y, wts = weights, info = trainInfo,  :
  There were missing values in resampled performance measures.
> 
> Yp = caret::predict.train(mlp, test[, 1:13])
> summary(Yp)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
  20.64   20.64   20.64   26.09   41.94   41.94