Could Seq2SeqSharp contribute to Tensorflow.NET NLU features in win-win way?

[GeorgeS2019]

Suggestions

• Currently Tensorlfow.NET lacks key Keras layers relevant for NLU (e.g. seq2seq and transformer). For examples:
  • Normalization layers
    • LayerNormalization
  • Attention layers
    • MultiHeadAttention
    • Attention

LayerNormalization, MultiHeadAttention and Attention layers in C# (for example) have been implemented in Seq2SeqSharp

The questions:

• Could these Seq2SeqSharp codes useful to speed up the implementation of NLU Keras layers in Tensorflow.NET?
• Is there a need for Seq2SeqSharp to adopt Keras layer naming?
  • This could help codes implemented in Tensorflow.NET to be relevant for Seq2SeqSharp and vice versa.
  • This also make more developers aware of the transformer framework in Seq2SeqSharp.
  • Hopefully more .NET developers could join and speed up transformer NLU development for the .NET community.

[zhongkaifu]

Thanks @GeorgeS2019. That's a great idea. However, Tensorflow.NET uses Google's Tensorflow as backend, but Seq2SeqSharp has its owned backend, such as computing graph, CUDA kernel, memory management and others. So, it could be problem to mix them up. One approach maybe work is to migrate Seq2SeqSharp's attention layers and normalization layers (forward pass only) to Tensorflow.NET, but it requires the operations between Seq2SeqSharp and Tensorflow.NET are aligned.