add torch.cosine_similarity

vieting commented 2 years ago

So far, I added a DotProductAlongDim module to do the dot product. The rest is done inside torch.nn.functional.cosine_similarity using existing functions. However, there is a problem with the DotLayer:

    line: assert not set(b_reduce_axes).intersection(b_var_axes)
    locals:
      set = <builtin> <class 'set'>
      b_reduce_axes = <local> [2]
      intersection = <not found>
      b_var_axes = <local> [2]

for the layer

*** root/'DotProductAlongDim' layer dict: {'class': 'dot', 'red1': 'F', 'red2': 'F', 'from': ['data', 'add']}

with inputs

layer root/'data' output: Data{'data', [B,T|'time:data'[B],F|F'feature:data'(7)]}
layer root/'add' output: Data{'add_output', [B,T|'time:data'[B],F|F'feature:data'(7)]}

albertz commented 2 years ago

Btw, you need to rebase this such that we get the new behavior version (#93).

vieting commented 2 years ago

Apparently the output still does not match.

albertz commented 2 years ago

Apparently the output still does not match.

This is strange. Can you check at what point it gets wrong? You could also calculate it by hand for some simple values.

albertz commented 2 years ago

The PR can be marked as ready, right? (When all tests are passing now.)

rwth-i6 / pytorch-to-returnn

add torch.cosine_similarity #92