mlech26l
commented
1 year ago Interesting use case. You need to be careful when changing the sparsity mask because TensorFlow probably requires a recompilation in order for this to have an effect.
One option would be to create a new LTC network with the desired wiring and then copy all the weights but not the sparsity mask from the old to the new model.
Thank you for your nice work. I still work on LTC network and I would like to ask about model evaluation. I trained LTC network with 64 neurons and already had the best model weight. However, I would like to drop some connection of LTC network (such as randomly drop 20% wiring connection of LTC network). When training, I used fully-connected wiring. When testing model, I tried to change into random wiring instead of fully-connected wiring. However, I still got the same result when I tested random wiring and original fully-connected wiring with the same model. It looks like the model is fixed and it only used fully-connected wiring (because I used it training). I wonder if there is any way to drop some connection among LTC neurons while evaluating model.