predict method for pruned networks

[fawda123]

Hi again,

I had a quick question about the predict method for pruned neural networks. I'm getting some weird results when I predict values with new data from a pruned mlp network. Here's some code to illustrate the problem.

A plot of the pruned network...

library(RSNNS)
# devtools::install_github('fawda123/NeuralNetTools')
library(NeuralNetTools)
x <- neuraldat[, c('X1', 'X2', 'X3')]
y <- neuraldat[, 'Y1']

# pruned model using code from RSSNS pruning demo
pruneFuncParams <- list(max_pr_error_increase = 10.0, pr_accepted_error = 1.0, 
  no_of_pr_retrain_cycles = 1000, min_error_to_stop = 0.01, init_matrix_value = 1e-6, 
  input_pruning = TRUE, hidden_pruning = TRUE)
mod <- mlp(x, y, size = 5, pruneFunc = "OptimalBrainSurgeon", 
  pruneFuncParams = pruneFuncParams)

# plot shows that only X3 has weighted connections
plotnet(mod)

Now, when I give the pruned model some new data with a constant value for X3, why would the predicted response change? I would expect changing values for X1 and X2 to have no effect on model predictions because the connection weights are zero, whereas changing values for X3 would produce a response for Y because the connection weights are non-zero. Am I thinking about this correctly?

# ranges
# apply(x, 2, range)

# no obs for prediction data
n <- 10

# holding X3 constant gets a response
newdat <- data.frame(X1 = seq(0, 1, length = n), X2 = seq(0, 1, length = n), X3 = 0.5)
predict(mod, newdata = newdat)

# holding all but X3 constant returns a single value for the response

newdat <- data.frame(X1 = 0.5, X2 = 0.5, X3 = seq(0, 1, length = n))
predict(mod, newdata = newdat)

Any idea why this is happening?

Thanks,

Marcus

[cbergmeir]

Hi, the problem was that the data was actually introduced in reverse order than the labels for the input units. I just committed a fix that the data is now introduced in reverse order, so can you please check if this resolves the problem?

[fawda123]

Hi, that definitely changed the output but I still don't think it's correct. Check out this example:

set.seed(123)
library(NeuralNetTools)
library(RSNNS)

x <- neuraldat[, c('X1', 'X2', 'X3')]
y <- neuraldat[, 'Y1']

# pruned model using code from RSSNS pruning demo
pruneFuncParams <- list(max_pr_error_increase = 10.0, pr_accepted_error = 1.0, 
 no_of_pr_retrain_cycles = 1000, min_error_to_stop = 0.01, init_matrix_value = 1e-6, 
 input_pruning = TRUE, hidden_pruning = TRUE)
mod <- mlp(x, y, size = 5, pruneFunc = "OptimalBrainSurgeon", 
 pruneFuncParams = pruneFuncParams)

plotnet(mod)

lekprofile(mod, x)

mod$snnsObject$getCompleteWeightMatrix()

The pruned model shows only X1 as being important, whereas the lekprofile results show only a response to changes in X3. The lekprofile method uses the predict function that I showed in the last example. The plot and the weights matrix also confirm the problem. Any thoughts? Thanks.

[cbergmeir]

I had another look into this. I think I found the problem now and hopefully could fix it. Can you please check?

[fawda123]

Yep, that fixed it, thanks! I'll look forward to the next CRAN release.

[cbergmeir]

The new version is on CRAN now