search

run-llama / llama_parse

Parse files for optimal RAG
https://www.llamaindex.ai
MIT License
3.08k stars 299 forks source link

Issue with LlamaParse #483

Closed vicacid closed 1 day ago

vicacid commented 1 day ago

Describe the bug If I work with low-quality files (bad scans/photos and etc.), the result turns out to be incorrect—the model simply hallucinates, making up content. The main problem is not that I'm getting an incorrect result, but rather that I have no way of knowing this. There's no confidence rate or similar parameter that I could rely on, nor do I receive any error in response.

Using OCR tools for such files (like docTR), I get correct results.

Job ID 70288cfd-b87f-4b39-8df3-901a8920e761 42362160-7a18-434f-8e51-405eb711ec7b - without parsing instructions 1d45ede8-18d2-4c36-bd79-06ac5f110c34 - without parsing instructions

Client:

Additional context I understand that this is a tricky question from a business process perspective—how the parser should behave. It might make sense to use tools to identify such file types before sending them to LLamaParse, but maybe there are some internal tools that I haven't been able to find.

BinaryBrain commented 1 day ago

Hi @vicacid, You can get confidence levels of the OCR in the Json output (.images[0].ocr[0].confidence).

I just checked this job: 1d45ede8-18d2-4c36-bd79-06ac5f110c34 and I don't see how we could correctly OCR the photo of a page in 480x640. I haven't tried it with docTR, but I doubt it can output credible results.

vicacid commented 1 day ago

@BinaryBrain thank you for the answer.

I believe confidence param will work!

I also thought that 1d45ede8-18d2-4c36-bd79-06ac5f110c34 is not readable - but they did it (of course there is minor mismatches - but you can check the result)

BinaryBrain commented 1 day ago

I'd love to see the output of docTR if you have it.

vicacid commented 1 day ago

I'd love to see the output of docTR if you have it.

@BinaryBrain I'm not sure it's a good idea to share it here. Do you have any support email or other messenger channels for the communication? I've tried email from llamacloud which is under "Email us" but that one is dead.

BinaryBrain commented 1 day ago

You can email me at sacha@runllama.ai