search

LLukas22 / llm-rs-python

Unofficial python bindings for the rust llm library. 🐍❤️🦀
MIT License
71 stars 4 forks source link

use pydantic.v1 in langchain.py #30

Closed andri-jpg closed 12 months ago

andri-jpg commented 12 months ago

fix issue #29

LLukas22 commented 12 months ago

Thanks 👍. If the CI passes I'll merge it and create a bugfix release.

andri-jpg commented 12 months ago

However, when I attempted to install using pip install . from the source without making any code changes, it also failed and returned errors from Cargo.

LLukas22 commented 12 months ago

Yeah the tokenizers create had some breaking changes, meaning this PR is needed.

I'll sync the latest main branch of rustformers when i get back home. That means i'll also have to implement custom RoPE scaling and probably Falcon. 🤔

LLukas22 commented 12 months ago

Alright this seams to work, keep in mind that langchain doesn't officially support pydantic 2 meaning there still could be some issues. I'm merging it for now.

andri-jpg commented 12 months ago

Alright this seams to work, keep in mind that langchain doesn't officially support pydantic 2 meaning there still could be some issues. I'm merging it for now.

Alright, I'll try reinstalling from the source again. I'll create another GitHub issue if any other problems come up.

LLukas22 commented 12 months ago

Yeah let me know if everything works, then i'll create a new release 👍

andri-jpg commented 12 months ago

image Nice, everything works well. Both llm_rs.langchain and Langchain are running smoothly without any issues. By the way, I've noticed a slight performance improvement here – the generation time has become slightly faster (from 1.4s to 0.8s), model = gpt2.

LLukas22 commented 12 months ago

Yeah there is less copying going on, as i changed all repeat operations into broadcastable mul/add operations, which means less time should be spend copying data around.