Open WilliamTambellini opened 1 year ago
@WilliamTambellini,
Isn't reciprocal square root supported by eltwise pow algorithm with alpha =1 and beta = -1/2?
Will adding sin/cos algorithms to eltwise primitive be enough to cover embeddings case?
Thanks
+1 for adding cos/sin elementwise! I would also greatly appreciate it!
@WilliamTambellini @lucasalavapena Thank you for the request, regarding sin/cos: ML frameworks have their own implementation and C++ applications have access to the standard math library with cos/sin functions, so what is the benefit of having an additional implementation under oneDNN API?
@WilliamTambellini I see that positional encoding does not depend on the actual data, is it pre-computed prior to inference/training and remains constant during the process?
@igorsafo I am still relatively new to oneDNN, so hopefully what I am saying makes sense :smile: .
I was more thinking of having it as an op for dnnl::graph, so you can get some benefits you normally can get with the graph extension (memory layout and use of graph compiler?).
Aren't some of the existing element wise functions part of the standard math library too?
Hi Here are some unary ops currently not implemented in 1dnn (or I missed them):
https://en.wikipedia.org/wiki/File:Gjl-t(x).svg
Best