I haven't quite nailed it down yet, but the system will sometimes crash with specific files... that I haven't been able to pin down. I'm guessing they're images as the offending line seems to be in src/loader.py, specifically the text = splitter.split_text("\n".join([d.text for d in docs]))[0] line. It seems as if the application is unable to handle if there is no text within the document(?).
I've added a try - except case in that to try to pin down exactly what's happening, but because I've only spent about an hour looking into it I haven't figured out a full fix.
It might be best to split the single-line into the full functions and then run an if-statement in order to get the best result. I'll see if I have some time this weekend to get a working fix that isn't an exception catch, but I can provide no promises.
I haven't quite nailed it down yet, but the system will sometimes crash with specific files... that I haven't been able to pin down. I'm guessing they're images as the offending line seems to be in src/loader.py, specifically the
text = splitter.split_text("\n".join([d.text for d in docs]))[0]line. It seems as if the application is unable to handle if there is no text within the document(?).I've added a try - except case in that to try to pin down exactly what's happening, but because I've only spent about an hour looking into it I haven't figured out a full fix.
It might be best to split the single-line into the full functions and then run an if-statement in order to get the best result. I'll see if I have some time this weekend to get a working fix that isn't an exception catch, but I can provide no promises.