Right now, when one (or multiple) file(s) in data/ipcc_text_reports have changed, the entire chunking and embedding will take place again and get added on top of the already existing chunks, even if they already exist.
Desired behaviour: only the file(s) that has/have changed or were newly added shall get chunked and embedded to save resources.
Right now, when one (or multiple) file(s) in
data/ipcc_text_reportshave changed, the entire chunking and embedding will take place again and get added on top of the already existing chunks, even if they already exist. Desired behaviour: only the file(s) that has/have changed or were newly added shall get chunked and embedded to save resources.