Update docs and example to include positional embeddings

dingo-actual / infini-transformer

PyTorch implementation of Infini-Transformer from "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention" (https://arxiv.org/abs/2404.07143)

MIT License

280 stars 23 forks source link

Update docs and example to include positional embeddings #19

Closed dingo-actual closed 6 months ago

dingo-actual commented 6 months ago

After changing the way positional embeddings are incorporated into the transformer classes, the docs and example need to be updated.

dingo-actual commented 6 months ago

Implemented.