iancovert
commented
5 years ago Hi Avinash,
You've noticed that the experiment files typically save a dictionary of results, with an entry called 'GC_est'. The way this 1D array is computed is: i) the relevant first layer weights are extracted from the neural network (all weights in MLP; weights that touch the input in LSTM), ii) those weights are reshaped if applicable (in MLP we reshape the weights to put all the lags for one input variable in the same column), iii) we take the norm of the weight matrix to quickly determine if there are columns where all weights are equal to zero.
If you follow a similar pattern in your experiments, you can determine the sparsity pattern by checking where zeros are located in the 'GC_est' entry in the results dictionary.
As for data preprocessing, results will definitely vary depending on the scaling of your inputs. It would probably be reasonable to standardize your input data (as people do in linear l1-regularized models) by scaling each feature to be in the range [-1, 1] or to have variance 1.
Let me know if there's anything else I can help clarify.
Ian
10avinash
Hi Ian, Thanks for your previous reply. I am trying to run my own dataset on your c-mlp and c-lstm architectures. I wanted to know if you have any general guidelines in preprocessing the data. Further, how does one extract the first layer weights once we train it on the parallel c-mlp. I have written a small script to check the output of the dictionary stored in Experiment/Results. Thanks again for taking the time.
Thanks, Avinash