I believe using go_backwards=True does not make a ConvLSTM bidirectional. It just makes the input reversed. Instead, there should be a Bidirectional layer wrapping around the ConvLSTM to do the job. This way there would be two separate outputs for forward and backward cells. I'm not sure if those should be concatenated or averaged based on the model proposed in the paper. For the first option we have to make the number output channels half, so that the dimensions stay constant.
I believe using go_backwards=True does not make a ConvLSTM bidirectional. It just makes the input reversed. Instead, there should be a Bidirectional layer wrapping around the ConvLSTM to do the job. This way there would be two separate outputs for forward and backward cells. I'm not sure if those should be concatenated or averaged based on the model proposed in the paper. For the first option we have to make the number output channels half, so that the dimensions stay constant.