Closed mcvta closed 5 years ago
itdxer
commented
5 years ago Thank you for reporting this issue
I know that the matrix is invertible, is this a problem with the version of neupy?
It might be, would it be possible for you to upgrade to the latest neap version?
itdxer
commented
5 years ago I've just realized that you're using WeightElimination which was removed from the latest version. Could you also check if you get the same error without WeightElimination?
mcvta
commented
5 years ago Removing WeightElimination, with the same versions of neupy and tensorFlow I get this:
Traceback (most recent call last):
File "C:\Users\manue\Google Drive\Neural NET\workspace\Torrao\ALoxygenLevenbergMarquardt_Incremento.py", line 84, in LevenbergMarquardt object contains invalid properties: decay_rate
Upgrading neupy to version 0.8.2 I get this:
Traceback (most recent call last):
File "C:\Users\manue\Google Drive\Neural NET\workspace\Torrao\ALoxygenLevenbergMarquardt_Incremento.py", line 84, in
itdxer
commented
5 years ago decay_rate should be removed as well, since it's part of the WeightElimination.
For the version 0.8.2, can you try to rename connection= to network=?
mcvta
commented
5 years ago I have just replaced connection by network... The network is working fine! Can you tell me which addons work with LevenbergMarquardt? Thank you
itdxer
commented
5 years ago Great :+1:
I decided to remove addons starting from version 0.8 since it was a bit difficult to support them and I wanted to make code a bit more flexible for tensorflow (see http://neupy.com/docs/layers/with-tensorflow.html).
You can user regularisers like this (see http://neupy.com/pages/cheatsheet.html#regularizers)
from neupy import algorithms
from neupy.layers import *
optimizer = algorithms.Momentum(
Input(5) >> Relu(10) >> Sigmoid(1),
step=algorithms.l2(decay_rate=0.1)
)And learning rate updates could be used like this (see http://neupy.com/pages/cheatsheet.html#learning-rate-update-rules)
from neupy import algorithms
from neupy.layers import *
optimizer = algorithms.Momentum(
Input(5) >> Relu(10) >> Sigmoid(1),
step=algorithms.step_decay(
initial_value=0.1,
reduction_freq=100,
)
)I will try this approach. Thank you.
I´m using neupy 0.72 and tensorflow 1.12.0, python 3.6, all installed with anaconda. I´m getting this error message. I know that the matrix is invertible, is this a problem with the version of neupy?
Main information
[ALGORITHM] LevenbergMarquardt
[OPTION] verbose = True [OPTION] epoch_end_signal = None [OPTION] show_epoch = 1 [OPTION] shuffle_data = True [OPTION] train_end_signal = None [OPTION] addons = ['WeightElimination'] [OPTION] error = mse [OPTION] mu = 0.1 [OPTION] mu_update_factor = 2 [OPTION] decay_rate = 0.1 [OPTION] zero_weight = 1
[TENSORFLOW] Initializing Tensorflow variables and functions. [TENSORFLOW] Initialization finished successfully. It took 0.26 seconds
Start training
[TRAINING DATA] shapes: (6543, 19) [TEST DATA] shapes: (1096, 19) [TRAINING] Total epochs: 20
| Epoch | Train err | Valid err | Time |
| 1 | 1.7737 | nan | 2 sec | Traceback (most recent call last): File "C:\Users\manue\anaconda2\envs\py36\lib\site-packages\tensorflow\python\client\session.py", line 1334, in _do_call return fn(*args) File "C:\Users\manue\anaconda2\envs\py36\lib\site-packages\tensorflow\python\client\session.py", line 1319, in _run_fn options, feed_dict, fetch_list, target_list, run_metadata) File "C:\Users\manue\anaconda2\envs\py36\lib\site-packages\tensorflow\python\client\session.py", line 1407, in _call_tf_sessionrun run_metadata) tensorflow.python.framework.errors_impl.InvalidArgumentError: Input matrix is not invertible. [[{{node training-updates/MatrixSolve}} = MatrixSolve[T=DT_FLOAT, adjoint=false, _device="/job:localhost/replica:0/task:0/device:CPU:0"](training-updates/add, training-updates/MatMul_1)]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "C:\Users\manue\Google Drive\Neural NET\workspace\Torrao\ALoxygenLevenbergMarquardt_Incremento.py", line 112, in
cgnet.train(XTrainScaled, yTrainScaled, XTestScaled, yTestScaled, epochs=20)
File "C:\Users\manue\anaconda2\envs\py36\lib\site-packages\neupy\algorithms\constructor.py", line 479, in train
*args, *kwargs
File "C:\Users\manue\anaconda2\envs\py36\lib\site-packages\neupy\algorithms\base.py", line 408, in train
train_error = train_epoch(input_train, target_train)
File "C:\Users\manue\anaconda2\envs\py36\lib\site-packages\neupy\algorithms\constructor.py", line 496, in train_epoch
return self.methods.train_epoch(as_tuple(input_train, target_train))
File "C:\Users\manue\anaconda2\envs\py36\lib\site-packages\neupy\algorithms\constructor.py", line 178, in wrapper
feed_dict=feed_dict,
File "C:\Users\manue\anaconda2\envs\py36\lib\site-packages\tensorflow\python\client\session.py", line 929, in run
run_metadata_ptr)
File "C:\Users\manue\anaconda2\envs\py36\lib\site-packages\tensorflow\python\client\session.py", line 1152, in _run
feed_dict_tensor, options, run_metadata)
File "C:\Users\manue\anaconda2\envs\py36\lib\site-packages\tensorflow\python\client\session.py", line 1328, in _do_run
run_metadata)
File "C:\Users\manue\anaconda2\envs\py36\lib\site-packages\tensorflow\python\client\session.py", line 1348, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Input matrix is not invertible.
[[node training-updates/MatrixSolve (defined at C:\Users\manue\anaconda2\envs\py36\lib\site-packages\neupy\algorithms\gd\lev_marq.py:158) = MatrixSolve[T=DT_FLOAT, adjoint=false, _device="/job:localhost/replica:0/task:0/device:CPU:0"](training-updates/add, training-updates/MatMul_1)]]
Caused by op 'training-updates/MatrixSolve', defined at: File "C:\Users\manue\Google Drive\Neural NET\workspace\Torrao\ALoxygenLevenbergMarquardt_Incremento.py", line 85, in
addons=[algorithms.WeightElimination]
File "C:\Users\manue\anaconda2\envs\py36\lib\site-packages\neupy\algorithms\gd\base.py", line 99, in init
super(BaseGradientDescent, self).init(connection, options)
File "C:\Users\manue\anaconda2\envs\py36\lib\site-packages\neupy\algorithms\constructor.py", line 283, in init
super(ConstructibleNetwork, self).init(*args, *kwargs)
File "C:\Users\manue\anaconda2\envs\py36\lib\site-packages\neupy\algorithms\constructor.py", line 121, in init
self.init_methods()
File "C:\Users\manue\anaconda2\envs\py36\lib\site-packages\neupy\algorithms\constructor.py", line 329, in init_methods
training_updates = self.init_train_updates()
File "C:\Users\manue\anaconda2\envs\py36\lib\site-packages\neupy\algorithms\regularization\weight_elimination.py", line 71, in init_train_updates
original_updates = super(WeightElimination, self).init_train_updates()
File "C:\Users\manue\anaconda2\envs\py36\lib\site-packages\neupy\algorithms\gd\lev_marq.py", line 158, in init_train_updates
tf.matmul(J_T, tf.expand_dims(err_for_each_sample, 1))
File "C:\Users\manue\anaconda2\envs\py36\lib\site-packages\tensorflow\python\ops\gen_linalg_ops.py", line 1246, in matrix_solve
"MatrixSolve", matrix=matrix, rhs=rhs, adjoint=adjoint, name=name)
File "C:\Users\manue\anaconda2\envs\py36\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "C:\Users\manue\anaconda2\envs\py36\lib\site-packages\tensorflow\python\util\deprecation.py", line 488, in new_func
return func(args, kwargs)
File "C:\Users\manue\anaconda2\envs\py36\lib\site-packages\tensorflow\python\framework\ops.py", line 3274, in create_op
op_def=op_def)
File "C:\Users\manue\anaconda2\envs\py36\lib\site-packages\tensorflow\python\framework\ops.py", line 1770, in init
self._traceback = tf_stack.extract_stack()
InvalidArgumentError (see above for traceback): Input matrix is not invertible. [[node training-updates/MatrixSolve (defined at C:\Users\manue\anaconda2\envs\py36\lib\site-packages\neupy\algorithms\gd\lev_marq.py:158) = MatrixSolve[T=DT_FLOAT, adjoint=false, _device="/job:localhost/replica:0/task:0/device:CPU:0"](training-updates/add, training-updates/MatMul_1)]]