Better Langchain integration

[BlueKiji77]

For a few days now I have been trying to use these example to develop RAG solution with DSPy using models downloaded from Huggingface but the integration has GPT4 hardcoded.

I modified LangChainPredict to support a model wrapped by HuggingfacePipeline but I keep hitting blocks requiring to sift through the codebase and modify functions.

Do you have any advice/solution on how to definitevily integrate Langchain and DSPy using any model loaded from Huggingface?

[arnavsinghvi11]

Hi @BlueKiji77 you can use HFClientTGI to support HuggingFace models and configure this to be your lm through dspy.settings(lm = ...)

[BlueKiji77]

Hello arnavsinghvi11, I am aware of HFClientTGI but for my case I need to be able to use models locally loaded in memory. That is why I have been trying to get HuggingfacePipeline working here.

[arnavsinghvi11]

Hi @BlueKiji77 , you can try using HFModel but this is not recommended. Feel free to push a PR to support HuggingFacePipeline in DSPy!

[antonvs88]

Hi, I've also been trying to run the example code from the example, and I get stuck at this part

zeroshot_chain = (RunnablePassthrough.assign(context=retrieve) | LangChainPredict(prompt, llm) | StrOutputParser())

I get the following error AttributeError: 'FieldInfo' object has no attribute 'finalize'

It seems to point to dspy/predict/langchain.py line 80.

Has anyone had success in running that example notebook?

[lucifer2288]

mark, i run LLMs locally too, hitting the same issue

[97k]

Is it possible to use Hugging Face Inference API for LLMs similar to HuggingFaceEndpoint LLM class in langchain? 🤔

cc: @okhat @lucifer2288 @antonvs88 @arnavsinghvi11 @BlueKiji77

[jjovalle99]

Same error here!

This is my env:

[tool.poetry.dependencies]
python = "~3.11"
dspy-ai = "^2.4.0"
langchain = "^0.1.14"
langchain-openai = "^0.1.1"

[kyutcho]

I am getting the same error: AttributeError: 'FieldInfo' object has no attribute 'finalize'. Anyone solved this issue?

EDIT: using version 2.1.1 solved this issue

[jeanbapt]

Same issue trying to run this code (langchain/dspy) with ollama locally.

Also there are some dependencies on OAI GPT-4 in LangchainPredict ... that I would get rid of.

def _build_signature(self, template):
        gpt4T = dspy.OpenAI(model='gpt-4-1106-preview', max_tokens=4000, model_type='chat')

        with dspy.context(lm=gpt4T): parts = dspy.Predict(Template2Signature)(template=template)

[herbgruutz]

Trying to run the example in Google Colab I'm getting the same error

[Ritvik19]

I am getting the same error: AttributeError: 'FieldInfo' object has no attribute 'finalize'. Anyone solved this issue?

EDIT: using version 2.1.1 solved this issue

share you langchain dspy and openai version...

[AldemarHdez]

any updates solving this error?

[thegallier]

langchain '0.1.16' dspy-0.1.5

same issue

[AriMKatz]

This was sort of a proof of concept. I gather from twitter that both teams are working on a more extensive and robust integration.

[dentro-innovation]

Yes a better langchain integration would be highly appreciated!

If even the official example from the langchain docs doesn't work...

[pechaut78]

I am getting the same error: AttributeError: 'FieldInfo' object has no attribute 'finalize'. Anyone solved this issue?

EDIT: using version 2.1.1 solved this issue

Does not work for me with latest but there is an ongoig PR waiting for approval with a fix (apparently)

[dentro-innovation]

fyi with the latest main branch of today I was able to get https://github.com/stanfordnlp/dspy/blob/main/examples/tweets/compiling_langchain.ipynb to run!

Just activate your venv in your working folder, then cd somewhere else where you clone this dspy repo, cd into the dspy folder and install the lib with pip install -e . to install the main branch (the latest main isn't released as a pypi package yet).

Also beware that tweet_metric.py (https://github.com/stanfordnlp/dspy/blob/main/examples/tweets/tweet_metric.py) which they use in the notebook has some lines missing so that evaluate fails for me. Add this to tweet_metric.py at the top to make it work:

colbertv2_wiki17_abstracts = dspy.ColBERTv2(url='http://20.102.90.50:2017/wiki17_abstracts')
dspy.settings.configure(rm=colbertv2_wiki17_abstracts)

Otherwise, could run the example with the latest langchain versions 💪🏼

[kyutcho]

fyi with the latest main branch of today I was able to get https://github.com/stanfordnlp/dspy/blob/main/examples/tweets/compiling_langchain.ipynb to run!

Just activate your venv in your working folder, then cd somewhere else where you clone this dspy repo, cd into the dspy folder and install the lib with pip install -e . to install the main branch (the latest main isn't released as a pypi package yet).

Also beware that tweet_metric.py (https://github.com/stanfordnlp/dspy/blob/main/examples/tweets/tweet_metric.py) which they use in the notebook has some lines missing so that evaluate fails for me. Add this to tweet_metric.py at the top to make it work:

colbertv2_wiki17_abstracts = dspy.ColBERTv2(url='http://20.102.90.50:2017/wiki17_abstracts')
dspy.settings.configure(rm=colbertv2_wiki17_abstracts)

Otherwise, could run the example with the latest langchain versions 💪🏼

Can you share what langchain and dspy version you used for this?

[aishwaryap]

@kyutcho I didn't try that notebook exactly but a close variation of it with dspy-ai==2.4.13 and it works. However if you swap out BootstrapFewShotWithRandomSearch with another optimizer such as COPRO there are still errors. This is what I get with COPRO

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[12], line 4
      2 teleprompter = COPRO(prompt_model=llm, metric=metric)
      3 kwargs = dict(num_threads=8, display_progress=True, display_table=0)
----> 4 compiled_prompt_opt = teleprompter.compile(zeroshot_chain, trainset=trainset, eval_kwargs=kwargs)
      5 eval_score = evaluate(compiled_prompt_opt, devset=testset, **kwargs)

File [~/Documents/venvs/dspy_v3_env/lib/python3.10/site-packages/dspy/teleprompt/copro_optimizer.py:160](http://localhost:8888/lab/tree/~/Documents/venvs/dspy_v3_env/lib/python3.10/site-packages/dspy/teleprompt/copro_optimizer.py#line=159), in COPRO.compile(self, student, trainset, eval_kwargs)
    158 basic_instruction = None
    159 basic_prefix = None
--> 160 *_, last_key = self._get_signature(predictor).fields.keys()
    161 basic_instruction = self._get_signature(predictor).instructions
    162 basic_prefix = self._get_signature(predictor).fields[last_key].json_schema_extra["prefix"]

AttributeError: 'list' object has no attribute 'keys'