Closed Davidott4 closed 7 years ago
The 1D convolution is correct for this application. 2D convolution would imply that you're trying to convolve over the one-hot encoding dimension, which you're not. The loss staying consistent probably has more to do with the fact that the network is fairly deep, so it could take a while to get any results. Could you give more information about training (how long, how you're presenting the dataset)?
A good sanity check might be to choose a positive and negative example from your dataset and feed them in over and over again to see if the network overfits on them.
Also, a "true" biLSTM would probably use use merge([l_lstm, r_lstm], mode='concat', concat_axis=-1)
Could you post the code you use to encode/OHE the DNA sequences? When I tried it on a similar problem, my model didn't learn much, and I have the same issue as you with 1D convolutions (no learning..). Thanks!
I'm trying to build a Convolutional Bi-directional LSTM to classify DNA sequences ala this paper: DanQ: a hybrid convolutional and recurrent deep neural network for quantifying the function of DNA sequences
The short version of it is to one-hot encode a DNA sequence:
'ATACG...' = [ [1,0,0,0], [0,0,0,1], [1,0,0,0], [0,1,0,0], [0,0,1,0], ...],
Then feed it to a convolutional-relu-maxpooling layer to find motifs, then into a bidirectional LSTM network to learn long-distance dependancies.
The original source code is here.
However, it uses an outdated version of Keras and includes a dependency on Seya, which is what I'd like to avoid doing. Here is my first attempt at building the model:
`
`
Unfortunately, the loss remained nearly constant during training, and the accuracy stayed constant as well. This leads me to believe that I have set the model up incorrectly, or that 1D convolution is the wrong kind of convolution to use for this input type So i attempted to make switch to 2D convolution:
`
`
Which gives me the following error when trying to feed the flattened layer into the LSTM:
Exception: Input 0 is incompatible with layer lstm_4: expected ndim=3, found ndim=2
Is this the correct setup for a convolutional bi-directional LSTM? The original code uses a Sequential model, but because this model branches, that doesn't seem like the right track. If so, then I likely need to upgrade to a 2D Convolution LSTM, in which case, how can I fix the input error?