This paper presents a comprehensive evaluation of the Optical CharacterRecognition (OCR) capabilities of the recently released GPT-4V(ision), a LargeMultimodal Model (LMM). We assess the model's performance across a range of OCRtasks, including scene text recognition, handwritten text recognition,handwritten mathematical expression recognition, table structure recognition,and information extraction from visually-rich document. The evaluation revealsthat GPT-4V performs well in recognizing and understanding Latin contents, butstruggles with multilingual scenarios and complex tasks. Based on theseobservations, we delve deeper into the necessity of specialized OCR models anddeliberate on the strategies to fully harness the pretrained general LMMs likeGPT-4V for OCR downstream tasks. The study offers a critical reference forfuture research in OCR with LMMs. Evaluation pipeline and results are availableat https://github.com/SCUT-DLVCLab/GPT-4V_OCR.
URL
Affiliations
Abstract
Translation (by gpt-3.5-turbo)
Summary (by gpt-3.5-turbo)