search

stanfordnlp / dspy

DSPy: The framework for programming—not prompting—foundation models
https://dspy-docs.vercel.app/
MIT License
13.97k stars 1.07k forks source link

Can't save Compiled KNNFewShot Examples #1089

Open Zoher15 opened 1 month ago

Zoher15 commented 1 month ago

Hi,

I tried the following but just got an empty json:

knn_fewshot = KNNFewShot(KNN, k=8, trainset=train)
compiled_knn_fewshot = knn_fewshot.compile(Hotpot, trainset=train)
compiled_knn_fewshot.save(f"Hotpot-knn-demos.json")

This would help in not having to repeat the knn search everytime?

okhat commented 1 month ago

Great note @Zoher15 , are you trying to avoid repeating the KNN indexing of the trainset every time?

This is actually pretty fundamental. It's not trivial to figure out how to save a KNNFewShot program.

Zoher15 commented 1 month ago

@okhat So my understanding is that predict/knn simply takes the inputs and finds the K centroids from training. If that is the case, then it is not really a teleprompter/optimizer? It is just a great way to sample examples and something that could actually work with other optimizer/teleprompters? I could see myself bootstrapping these specific centroids from a teacher model for instance. And in that case, yes it would be great to be able to save the centroid and avoid indexing every time.

arnavsinghvi11 commented 2 weeks ago

Hi @Zoher15 , would saving the state of the KNN model via self.KNN.trainset_vectors suffice here?