Unable to locate notebook _resources/00-init in the git repo for Dolly chatbot

[CreativityAI]

I'm trying to run the chat bot as mentioned in the demos. However, I encountered an issue with the langchain pipeline not supporting summarization yet. As a temporary fix, the companion notebook mentions the addition of the following code snippet:

hf_summary = HuggingFacePipeline_WithSummarization(pipeline=pipe_summary)

The notebook refers to a file named _resources/00-init, but I am unable to find it in the git repository. Could you please guide me on where to find the notebook or provide an alternative solution to address the issue?

Thank you for your assistance!

[QuentinAmbard]

hi @CreativityAI , are you installing the notebook running %pip install dbdemos

and then import dbdemos dbdemos.install('llm-dolly-chatbot')

The installer will load all the companion notebooks

[CreativityAI]

That works fine now when I install pip install dbdemos. Thanks @QuentinAmbard!

[AjinkyaBankar]

I am interested in developing this ChatBot, but I am not Databricks user. So cant install above given demo. Can someone share the definition of HuggingFacePipeline_WithSummarization() method?

[QuentinAmbard]

hey, I beleive it has been merged in langchain: https://github.com/langchain-ai/langchain/blob/61dd92f8215daef3d9cf1734b0d1f8c70c1571c3/libs/langchain/langchain/llms/huggingface_pipeline.py#L174C1-L174C1 So you can use the standard HuggingFacePipeline if you use the last version

[AjinkyaBankar]

Thanks for your response. That's great! I checked the summarization pipeline with memory as:

from langchain.llms import HuggingFacePipeline
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer, pipeline
from langchain.memory import ConversationSummaryBufferMemory
import torch

summarize_model = AutoModelForSeq2SeqLM.from_pretrained("facebook/bart-large-cnn", device_map="auto", torch_dtype=torch.bfloat16, trust_remote_code=True)
summarize_tokenizer = AutoTokenizer.from_pretrained("facebook/bart-large-cnn", padding_side="left")
pipe_summary = pipeline("summarization", model=summarize_model, tokenizer=summarize_tokenizer) #, max_new_tokens=500, min_new_tokens=300

hf_summary = HuggingFacePipeline(pipeline=pipe_summary)
memory=ConversationSummaryBufferMemory(llm=hf_summary, max_token_limit=10)

Added chat history to the memory and observed the memory afterwards as:

memory.save_context({"input": "hi"}, {"output": "whats up"})
memory.save_context({"input": "not much you"}, {"output": "not much"})
memory.save_context({"input": "what's my name"}, {"output": "AJ"})

memory.load_memory_variables({})

It returned:

{'history': "System: The human asks what the AI thinks of artificial intelligence. The AI thinks artificial intelligence is a force for good because it will help humans reach their full potential. Progressively summarize the lines of conversation provided, adding onto the previous summary returning a new summary. The human asks the AI: Why do you think artificial Intelligence is a Force for good? The AI: Because artificial intelligence will help human reach their potential.\nHuman: what's my name\nAI: AJ"}

This doesn't summarize the actual chat history, but returns a common text The human asks what the AI thinks of artificial intelligence. The AI thinks artificial intelligence is a force for good because it will help humans reach their full potential. Progressively summarize the lines of conversation provided, adding onto the previous summary returning a new summary. The human asks the AI: Why do you think artificial Intelligence is a Force for good? The AI: Because artificial intelligence will help human reach their potential.

This would pass wrong prompt to the LLM for question-answering. Can you please help to debug this problem?