Applying times_transpose operation to vectors sequence output from Recurrence layer

[anat0li]

I’d like to have a sequence of vectors output from bidirectional recurrent layer to be unpacked to a tensor. Then I’d like to apply cntk.times_transpose operation to that tensor, as in the following example:

import numpy as np
import cntk as C
a = np.arange(6, dtype=np.float32).reshape(3,2)
print(a, '\n\n', C.times_transpose(a, a).eval())

output:

[[0. 1.]
 [2. 3.]
 [4. 5.]]

 [[ 1.  3.  5.]
 [ 3. 13. 23.]
 [ 5. 23. 41.]]

The result would be a matrix with the elements equal to pairwise dot products of all vectors from input sequence. In some cases such a matrix could be interpreted a probability matrix; and softmax could be applied to each matrix row to make a final prediction.

I can see how this model can be implemented in Tensorflow, but it is not clear for me, how it can be done in cntk. I have tried some sequence unpacking operations, such as C.sequence.unpack or C.layers.PastValueWindow, but I’m getting runtime errors with some not too helpful error messages, such as “RuntimeError: TensorOp: Tensor operations are currently not supported for sparse matrices.”. Could anyone point me in the right direction, please? Thanks in advance.

[anat0li]

Closing this issue because this functionality is available in TF. Sorry if the description was not clear enough.