Could you provide the pretrained weights for testing?

corl-team / rebased

Official implementation of the paper "Linear Transformers with Learnable Kernel Functions are Better In-Context Models"

Apache License 2.0

151 stars 4 forks source link

Could you provide the pretrained weights for testing? #2

Open kabachuha opened 6 months ago

kabachuha commented 6 months ago

Hello!

The concept is awesome, and it would be nice to integrate it into the huggingface/transformers library. However, to ensure that everything works correctly and matches the paper results, we need to have pretrained weights. It would be very neat if you could provide at least the tiniest weights for testing :)

elephantmipt commented 6 months ago

Hi, thank you for your interest! You can find pre-trained weights and simple generation code here. We are planning to merge it in this repo soon. Note that there is currently no caching, so text generation may take a long time. Please let me know if you need help.

kabachuha commented 6 months ago

Thanks! ❤️