Nice to meet you. Thanks for the great work and sharing.
I have been using your SCINet and have found a few bugs and inefficient implementations.
I have been using it with local fixes, but I thought I would contribute to the community. If you would like to make use of it, please feel free to do so.
There are two areas that need to be fixed.
The first is to fix the use of in-place operation in the Tensor calculation.
Pytorch does not allow in-place operations such as +=, but they were being used in several places.
This operation is only used when the model has certain arguments and may have been missed during testing.
The second is the implementation of SCINet_Tree.zip_up_the_pants.
This implementation uses a for loop for the Tensor operation, which does not take advantage of the GPU's computational efficiency.
Therefore, I modified the implementation to use only matrix operations without changing the calculation result.
In my experiments on my local GPU, this modification resulted in a speedup of about 30~40 times.
Nice to meet you. Thanks for the great work and sharing. I have been using your SCINet and have found a few bugs and inefficient implementations. I have been using it with local fixes, but I thought I would contribute to the community. If you would like to make use of it, please feel free to do so. There are two areas that need to be fixed. The first is to fix the use of in-place operation in the Tensor calculation. Pytorch does not allow in-place operations such as +=, but they were being used in several places. This operation is only used when the model has certain arguments and may have been missed during testing. The second is the implementation of SCINet_Tree.zip_up_the_pants. This implementation uses a for loop for the Tensor operation, which does not take advantage of the GPU's computational efficiency. Therefore, I modified the implementation to use only matrix operations without changing the calculation result. In my experiments on my local GPU, this modification resulted in a speedup of about 30~40 times.