Since our dataset is a bunch of time-series data, we can use an LSTM model to try to find patterns based on sequential events.
For this, we can use the class torch.nn.LSTM, from PyTorch, and add two fully connected linear layers at the end, that makes a linear regression from the outputs of the LSTM layers, to one single parameter, considering a regression problem.
class LSTM1(nn.Module):
def __init__(self, num_classes, input_size, hidden_size, num_layers, seq_length):
super(LSTM1, self).__init__()
self.num_classes = num_classes #number of classes
self.num_layers = num_layers #number of layers
self.input_size = input_size #input size
self.hidden_size = hidden_size #hidden state
self.seq_length = seq_length #sequence length
self.lstm = nn.LSTM(input_size=input_size, hidden_size=hidden_size,
num_layers=num_layers, batch_first=True) #lstm
init.kaiming_normal_(self.lstm.weight, a=0.1)
self.lstm.bias.data.zero_()
self.fc_1 = nn.Linear(hidden_size, 128) #fully connected 1
init.kaiming_normal_(self.fc_1.weight, a=0.1)
self.fc_1.bias.data.zero_()
self.fc = nn.Linear(128, num_classes) #fully connected last layer
init.kaiming_normal_(self.fc.weight, a=0.1)
self.fc.bias.data.zero_()
self.relu = nn.ReLU()
def forward(self,x):
h_0 = Variable(torch.zeros(self.num_layers, x.size(0), self.hidden_size)) #hidden state
c_0 = Variable(torch.zeros(self.num_layers, x.size(0), self.hidden_size)) #internal state
# Propagate input through LSTM
output, (hn, cn) = self.lstm(x, (h_0, c_0)) #lstm with input, hidden, and internal state
hn = hn.view(-1, self.hidden_size) #reshaping the data for Dense layer next
out = self.relu(hn)
out = self.fc_1(out) #first Dense
out = self.relu(out) #relu
out = self.fc(out) #Final Output
return out
Here we have defined the hidden state, and internal state first, initialized with zeros. First of all, we are going to pass the hidden state and internal state in LSTM, along with the input at the current timestamp t. This will return a new hidden state, current state, and output. We’ll reshape the output so that it can pass to a Dense Layer. Next, simply apply activations, and pass them to the dense layers, and return the output.
Since our dataset is a bunch of time-series data, we can use an LSTM model to try to find patterns based on sequential events. For this, we can use the class torch.nn.LSTM, from PyTorch, and add two fully connected linear layers at the end, that makes a linear regression from the outputs of the LSTM layers, to one single parameter, considering a regression problem.
Here we have defined the hidden state, and internal state first, initialized with zeros. First of all, we are going to pass the hidden state and internal state in LSTM, along with the input at the current timestamp t. This will return a new hidden state, current state, and output. We’ll reshape the output so that it can pass to a Dense Layer. Next, simply apply activations, and pass them to the dense layers, and return the output.
Adapted from https://cnvrg.io/pytorch-lstm/