Open david101-hunter opened 3 months ago
Hi @david101-hunter, the error that you see happens because the FAISS vector store object cannot be serialized using Pickle. This change might have happened recently or could arise from the new embedding class you pass to the object.
ZenML uses the concept of materializers to save (serialize) and load (deserialize) objects from artifact stores. This is important because steps in a pipeline might need inputs from other steps which might be running in different environments. ZenML takes the output of a step, saves it using the logic from the compatible materializer and then loads it in whatever step needs it.
By default, if you don't specify a materializer for an output of the step, ZenML chooses to use pickle to serialize the object. This is not recommended for production use cases (see Is Pickle safe?). In this case, pickle can't serialize a part of the object which means you should create a custom materializer. In other words, you have to tell ZenML how to save and load an object of type FAISS. This way, whenever a FAISS object is encountered, ZenML will use your logic to interact with that object.
You can read this section here on FAISS docs on how to best serialize their objects. https://python.langchain.com/v0.1/docs/integrations/vectorstores/faiss/#saving-and-loading
from langchain_community.embeddings.huggingface import HuggingFaceEmbeddings
pkl = db.serialize_to_bytes() # serializes the faiss
embeddings = HuggingFaceEmbeddings(model_name="all-MiniLM-L6-v2")
db = FAISS.deserialize_from_bytes(
embeddings=embeddings, serialized=pkl
) # Load the index
You can read the ZenML docs on how to create a custom materializer using the logic above. https://docs.zenml.io/user-guide/advanced-guide/data-management/handle-custom-data-types#developing-a-custom-materializer
You can create this materializer for this project yourself and if it works, feel free to contribute it to this project and to the ZenML OSS repo. Let me know if anything is unclear.
Thanks for your explanation. I'm starting to try with the information that you provide.
@david101-hunter what version of langchain are you using?
@strickvl I use version langchain: 0.1.20
This project, e.g., has the version pinned to langchain==0.0.325
. I think subsequent versions will raise this error. They introduced an async mode which I think is causing the issue. This project (as currently coded) is implemented for the exact specs listed in the requirements.txt
file. To use a later version of langchain
you'll hit this materialization issue + would have to update the code somehow to make that work.
I used langchain==0.0.325
by pip install -r requirement
. However, I have updated these conflict version right after
Maybe @wjayesh knows more about the original set of requirements, but I just know that higher versions of langchain>0.0.325
are going to hit this materialization issue.
@wjayesh can you provide full requirements?
I'm able to get the requirements file to resolve as follows:
# assuming a fresh / blank environment
# cd into the llm-agents project
pip install -U pip uv
uv pip install -r requirements.txt
Everything gets resolved correctly and without any errors, using the 0.0.325 version of langchain.
Thanks. What can I do after I run this pipeline?. Maybe that's a stupid question, but I'm curious about it
@david101-hunter After you have run the pipeline, you would have an agent object that has a tool which can answer questions about ZenML. You can now
In short, what happens after the pipeline is run, is you get an agent that you can release to whatever system needs it. In our case, it could act like a support rep but you can design it in any other way.
Thanks
"Deploy this agent at some endpoint. By that, I mean, you can load the agent object into some FastAPI server and have it serve requests over a path on the server."
Do ZenML official document provide tutorial about it? or you can provide some key words?
You can use this page here to understand how to consume artifacts from ZenML https://docs.zenml.io/user-guide/starter-guide/manage-artifacts#consuming-artifacts-produced-by-other-pipelines
once you have the agent object, you can just create a web server using a framework of your choice and create a function at some path that uses this object and runs the input query on it.
@wjayesh I don't quite understand. I loaded agent object like.
How FastAPI will work with this information?
@wjayesh I need your help
.ask()
or any other functions.@wjayesh now, I still have not used this agent. I tried method: ask(), ... It's not wokring
I am sorry. the function should be run
. You can also find this in the notebook that is part of this project. Go to the Testing the agent
section.
agent.run({"input": question,"chat_history": []})
@wjayesh , OllamaEndpointNotFoundError: Ollama call failed with status code 404. Maybe your model is not found and you should
pull the model with ollama pull llama2
.
This ís locally model. I don't set authenticaiton. why it raise 404. I use docker to build gemma:2b from ollama
@wjayesh Can you say something? Please.
I'm follow this tutorials: creating-a-production-ready-llm-powered-agent-with-zenml
I have changed
Here is full logs
How can I fix?