High Level Features - Githubissues

getkeops / keops

KErnel OPerationS, on CPUs and GPUs, with autodiff and without memory overflows

https://www.kernel-operations.io

MIT License

1.04k stars 64 forks source link

High Level Features #186

Open hypnopump opened 3 years ago

hypnopump commented 3 years ago

This PR implements some high level features such as:

[x] Interface to call torch.nn.Linear() the kernel way.
[x] Reproducible conda environment for pykeops in Linux-gpu
[x] Naive implementation for several activation functions
[x] Simple MH Attention Layer (probably to move as an example?)

Projected roadmap:

[ ] Provide interface for simple MLPs
[ ] Optimized activation functions (to do once the engine is rewritten)

Proposals:

[ ] Element-wise maximum and minimum operator for multiple tensors : max([0,1], [1,0]) = [1,1]