gordonwatts / snowmass-chat

Experiments exploring the US Snowmass Process documents using LLM
Apache License 2.0
2 stars 0 forks source link

Do not include references #43

Open gordonwatts opened 4 months ago

gordonwatts commented 4 months ago

The chunking happily includes the references in a document. This is not great - when lookup includes that, it isn't helpful even a little bit. We should have some pre-processing to remove that.