search

gordonwatts / snowmass-chat

Experiments exploring the US Snowmass Process documents using LLM
Apache License 2.0
2 stars 0 forks source link

Use the RecursiveChunk alg from langchain #33

Closed gordonwatts closed 7 months ago

gordonwatts commented 7 months ago

That splits things by sentence, etc.

gordonwatts commented 7 months ago

Found something on it here.

gordonwatts commented 7 months ago

Turns out - we are already using it! But not with a great set of separators. So fixed that up a little bit!