search

revelrylabs / text_chunker_ex

A library for semantically coherent text chunking
MIT License
66 stars 4 forks source link

Separators: Implement html chunking strategy. #23

Closed cpursley closed 4 months ago

cpursley commented 4 months ago

I actually thought about not including article and section - perhaps they are actually unnecessary. Thoughts?

If you think we should keep, moving up prob makes sense. If so - where?

grossvogel commented 4 months ago

@cpursley I can't convince myself that moving (or removing) them would be a definite improvement, so I'm happy with this until somebody comes along with a stronger argument 😄

Thanks so much for this contribution!

cpursley commented 4 months ago

You're welcome.

Y'all should do a detailed blog post about RAG stuff in Elixir (how you're working with the chunked data, etc)!