search

IBM / data-prep-kit

Open source project for data preparation of LLM application builders
https://ibm.github.io/data-prep-kit/
Apache License 2.0
171 stars 111 forks source link

doc_id and source_doc_id params in doc_chunk #598

Closed dolfim-ibm closed 2 weeks ago

dolfim-ibm commented 2 weeks ago

Why are these changes needed?

This expands the doc_chunk parameters with the possibility to tune the propagation of the doc_id from the input tables to the results.

Related issue number (if any).

https://github.com/IBM/data-prep-kit/issues/585