search

SciSharp / TensorFlow.NET

.NET Standard bindings for Google's TensorFlow for developing, training and deploying Machine Learning models in C# and F#.
https://scisharp.github.io/tensorflow-net-docs
Apache License 2.0
3.26k stars 524 forks source link

Missing LayerNormalization & MultiHeadAttention Keras Layers #797

Open lqdev opened 3 years ago

lqdev commented 3 years ago

@Oceania2018 can you assign this issue to me. I'll try and take a stab at it.

I was trying to see whether implementing the Keras Transformer recommendation sample is feasible in .NET. However, I believe two of the layers, LayerNormalization and MultiHeadAttention are missing from the current implementation.

GeorgeS2019 commented 3 years ago

@lqdev In your approach to MultiHeadAttention, perhaps you could share your feedback to this thread of discussion.

Perhaps we may find more people interested on this topics for .NET

lindadamama commented 3 years ago

missing layerNormalization However, Torchsharp's APIS are complete https://github.com/tensorflow/tensorflow/tree/v2.7.0/tensorflow/python/keras/layers/normalization https://github.com/dotnet/TorchSharp/tree/main/src/TorchSharp/NN/Normalization