dagshub[bot]
commented
3 months ago Open david101-hunter opened 3 months ago
dagshub[bot]
commented
3 months ago 
wjayesh
commented
3 months ago Hi @david101-hunter, the error that you see happens because the FAISS vector store object cannot be serialized using Pickle. This change might have happened recently or could arise from the new embedding class you pass to the object.
ZenML uses the concept of materializers to save (serialize) and load (deserialize) objects from artifact stores. This is important because steps in a pipeline might need inputs from other steps which might be running in different environments. ZenML takes the output of a step, saves it using the logic from the compatible materializer and then loads it in whatever step needs it.
By default, if you don't specify a materializer for an output of the step, ZenML chooses to use pickle to serialize the object. This is not recommended for production use cases (see Is Pickle safe?). In this case, pickle can't serialize a part of the object which means you should create a custom materializer. In other words, you have to tell ZenML how to save and load an object of type FAISS. This way, whenever a FAISS object is encountered, ZenML will use your logic to interact with that object.
You can read this section here on FAISS docs on how to best serialize their objects. https://python.langchain.com/v0.1/docs/integrations/vectorstores/faiss/#saving-and-loading
from langchain_community.embeddings.huggingface import HuggingFaceEmbeddings
pkl = db.serialize_to_bytes() # serializes the faiss
embeddings = HuggingFaceEmbeddings(model_name="all-MiniLM-L6-v2")
db = FAISS.deserialize_from_bytes(
embeddings=embeddings, serialized=pkl
) # Load the indexYou can read the ZenML docs on how to create a custom materializer using the logic above. https://docs.zenml.io/user-guide/advanced-guide/data-management/handle-custom-data-types#developing-a-custom-materializer
You can create this materializer for this project yourself and if it works, feel free to contribute it to this project and to the ZenML OSS repo. Let me know if anything is unclear.
david101-hunter
commented
3 months ago Thanks for your explanation. I'm starting to try with the information that you provide.
strickvl
commented
3 months ago @david101-hunter what version of langchain are you using?
david101-hunter
commented
3 months ago @strickvl I use version langchain: 0.1.20
strickvl
commented
3 months ago This project, e.g., has the version pinned to langchain==0.0.325. I think subsequent versions will raise this error. They introduced an async mode which I think is causing the issue. This project (as currently coded) is implemented for the exact specs listed in the requirements.txt file. To use a later version of langchain you'll hit this materialization issue + would have to update the code somehow to make that work.
david101-hunter
commented
3 months ago I used langchain==0.0.325 by pip install -r requirement. However, I have updated these conflict version right after
strickvl
commented
3 months ago Maybe @wjayesh knows more about the original set of requirements, but I just know that higher versions of langchain>0.0.325 are going to hit this materialization issue.
david101-hunter
commented
3 months ago @wjayesh can you provide full requirements?
strickvl
commented
3 months ago I'm able to get the requirements file to resolve as follows:
# assuming a fresh / blank environment
# cd into the llm-agents project
pip install -U pip uv
uv pip install -r requirements.txtEverything gets resolved correctly and without any errors, using the 0.0.325 version of langchain.
david101-hunter
commented
3 months ago Thanks. What can I do after I run this pipeline?. Maybe that's a stupid question, but I'm curious about it
wjayesh
commented
3 months ago @david101-hunter After you have run the pipeline, you would have an agent object that has a tool which can answer questions about ZenML. You can now
In short, what happens after the pipeline is run, is you get an agent that you can release to whatever system needs it. In our case, it could act like a support rep but you can design it in any other way.
david101-hunter
commented
3 months ago Thanks
"Deploy this agent at some endpoint. By that, I mean, you can load the agent object into some FastAPI server and have it serve requests over a path on the server."
Do ZenML official document provide tutorial about it? or you can provide some key words?
wjayesh
commented
3 months ago You can use this page here to understand how to consume artifacts from ZenML https://docs.zenml.io/user-guide/starter-guide/manage-artifacts#consuming-artifacts-produced-by-other-pipelines
once you have the agent object, you can just create a web server using a framework of your choice and create a function at some path that uses this object and runs the input query on it.
david101-hunter
commented
3 months ago @wjayesh I don't quite understand. I loaded agent object like.
How FastAPI will work with this information?
david101-hunter
commented
3 months ago @wjayesh I need your help
wjayesh
commented
3 months ago .ask() or any other functions.david101-hunter
commented
3 months ago @wjayesh now, I still have not used this agent. I tried method: ask(), ... It's not wokring
wjayesh
commented
3 months ago I am sorry. the function should be run. You can also find this in the notebook that is part of this project. Go to the Testing the agent section.
agent.run({"input": question,"chat_history": []})@wjayesh , OllamaEndpointNotFoundError: Ollama call failed with status code 404. Maybe your model is not found and you should
pull the model with ollama pull llama2.
This ís locally model. I don't set authenticaiton. why it raise 404. I use docker to build gemma:2b from ollama
david101-hunter
commented
3 months ago @wjayesh Can you say something? Please.
I'm follow this tutorials: creating-a-production-ready-llm-powered-agent-with-zenml
I have changed
Here is full logs
How can I fix?