Very cool! :)

[dynamicwebpaige]

Love this project, as well as your Seq2Seq implementation and RL library!

Have you considered integrating with Swift for TensorFlow? 😄

[palle-k]

Swift for TensorFlow includes its own Tensor library, which you can read about in the TensorFlow documentation, so if you want to work with TensorFlow, this is probably the better way to go. This library provides an alternative implementation of automatic differentiation, similar to how PyTorch and TensorFlow work differently.

[RahulBhalley]

Basically it means both can't be integrated together because of the basic structure they use i.e. their own Tensor internal implementation, right? S4TF have different design principle and ur library have different. Am I correct?

[palle-k]

S4TF has two parts:

• The first is the computation of derivatives. S4TF uses the @differentiable attribute, which declares a function as differentiable. If the function is atomic, additional parameters for the attribute describe, how to compute the derivative.
• The second component is a Tensor library, which provides a set of operations that have been declared as differentiable. The tensor library provides operations, such as broadcasting, reductions and other neural network operators and features such as GPU and TPU support.

DL4S follows a very similar approach. It provides a Tensor library with a similar set of operations (though TensorFlow supports a much wider selection of operators). It also provides a way of computing derivatives of operations. S4TF and DL4S use a similar approach for computing derivatives. Each "atomic" operation specifies a way to compute its derivative given its inputs and the result of the operation. The derivative of a sequence of operations is computed by packpropagating through the compute graph of those operations.

So, both features that S4TF provides, are also implemented by DL4S. Using S4TF's method of differentiation would solve an issue that is already solved in DL4S. Also, replacing DL4S' method of differentiation with @differentiable will not magically add support for TPUs and other hardware accelerators.

It would be possible to add an additional S4TF engine to DL4S, similar to how a CPU engine is provided already and a GPU engine is in the works. This would then use S4TF's Tensor implementation. But I don't see a point in doing that. It would make more sense to use S4TF directly then.

[RahulBhalley]

It would be possible to add an additional S4TF engine to DL4S, similar to how a CPU engine is provided already and a GPU engine is in the works. This would then use S4TF's Tensor implementation. But I don't see a point in doing that. It would make more sense to use S4TF directly then.

Yup. That's right.