krishnaik06 / Complete-Langchain-Tutorials

GNU General Public License v2.0
351 stars 289 forks source link

Unused variable `doc` assigned but not being returned. Proper chunked data not being processed #11

Open robhitt opened 7 months ago

robhitt commented 7 months ago

In LLM Generic APP > test.ipynb in the method chunk_data it is currently returning docs and should return doc. In its current form, doc is an unused variable.

Currently this function doesn't do anything other than pass through the original docs argument .

See below where doc is currently being unused:

def chunk_data(docs,chunk_size=800,chunk_overlap=50):
    text_splitter=RecursiveCharacterTextSplitter(chunk_size=chunk_size,chunk_overlap=chunk_overlap)
    doc=text_splitter.split_documents(docs)  # <-- doc unused
    return docs  # <-- returning initial docs argument