This PR is made for solving Issue #48.
What I've discovered in my research of this problem was that a malformed JSON was made when the number of chunks was generally bigger. For example, I've found this for the actual details that @mikhailocampo provided:
I've also tested them with the following files and found the same problem:
https://hagan.okstate.edu/NNDesign.pdf with the topic "Neural Networks"
https://diposit.ub.edu/dspace/bitstream/2445/180441/2/tfm_lichtner_bajjaoui_aisha.pdf
So, I've discovered that the malformed JSON had a "properties" key in its composition, making it unappropriate for the actual Pydantic Schema. So, I developed a function to test if the response has that. If so, then it will re-try to generate another question, providing appropriate results with bigger PDF files.
Although accurate, I think this is separate from the #48 Issue given that the issue encountered here is related to ValidationErrors rather than the mapping error from #48
This PR is made for solving Issue #48. What I've discovered in my research of this problem was that a malformed JSON was made when the number of chunks was generally bigger. For example, I've found this for the actual details that @mikhailocampo provided:
I've also tested them with the following files and found the same problem:
https://hagan.okstate.edu/NNDesign.pdf with the topic "Neural Networks"
https://diposit.ub.edu/dspace/bitstream/2445/180441/2/tfm_lichtner_bajjaoui_aisha.pdf
So, I've discovered that the malformed JSON had a "properties" key in its composition, making it unappropriate for the actual Pydantic Schema. So, I developed a function to test if the response has that. If so, then it will re-try to generate another question, providing appropriate results with bigger PDF files.
![image](https://github.com/radicalxdev/kai-ai-backend/assets/113047749/c5ec0983-7a38-4c6a-b601-c0d9d17f0795)