One of the bottleneck Etaler have is that encoding is done in the frontend. Which is easy to implement. You encode the SDR into an array and call Tensor() to copy whatever data is in that array into a real Tensor. But this is very slow. On CPU, there is at least one extra array copy and memory allocation. One GPU, the current code path copies the entire array over PCIe to the GPU.
So the obvious solution is to move the responsibility of encoding data to the backend. But could we do so in an elegant way? And we need to maintain the high expandability.
One of the bottleneck Etaler have is that encoding is done in the frontend. Which is easy to implement. You encode the SDR into an array and call
Tensor()to copy whatever data is in that array into a real Tensor. But this is very slow. On CPU, there is at least one extra array copy and memory allocation. One GPU, the current code path copies the entire array over PCIe to the GPU.So the obvious solution is to move the responsibility of encoding data to the backend. But could we do so in an elegant way? And we need to maintain the high expandability.