trOCR base model 30% CER on IAM word dataset vs 4% for IAM line dataset, is this normal?

[slender9168]

Describe the bug Model I am using: trocr-base-handwritten

Dataset:

• IAM word dataset: https://www.kaggle.com/datasets/nibinv23/iam-handwriting-word-database
• IAM line dataset: https://huggingface.co/datasets/Teklia/IAM-line

The problem arises when using:

• [ ] my own modified scripts: ` self.device = torch.device("cuda" if torch.cuda.is_available() else "cpu") print(self.device)

  # Initialize processor and model
  self.processor = TrOCRProcessor.from_pretrained("microsoft/trocr-base-handwritten")
  self.model = VisionEncoderDecoderModel.from_pretrained("microsoft/trocr-base-handwritten").to(self.device)
  # Prepare image and move pixel values to the device
  image = Image.open(image_path).convert("RGB")
  pixel_values = self.processor(image, return_tensors="pt").pixel_values.to(self.device)
  
  # Generate text
  generated_ids = self.model.generate(pixel_values)
  generated_text = self.processor.batch_decode(generated_ids, skip_special_tokens=True)[0]

`

A clear and concise description of what the bug is: when running microsoft/trocr-base-handwritten against the IAM word dataset ( single words ), I got a CER of about 30% when running it against the IAM line dataset, the CER is about 4%

1. is this expected?
2. can I train the model on single word images to enhance its performance on single words to 4% CER? or is it inherintally bad on single words?
3. is the model being trained on full lines instead of single words, the reason for the 30% CER?

To Reproduce Steps to reproduce the behavior:

1. use the sample code with microsoft/trocr-base-handwritten against the IAM word dataset, the CER will be aroud 30%

• Platform: windows 10
• Python version: 3.8
• PyTorch version (GPU?): 2.5.1+cu124