custom ctc_char_map.json - train_text_recognition.py causes KeyError

[FelixSchwarz]

I'm trying to get acquainted with the networks in this repo and I decided to start as simple as possible (text_recognition model).

I generated myself a custom dataset which only contains 8-digit "words" (cropped, no skew, no noise).

I generated my custom charmap like this:

from collections import OrderedDict
import json, sys
alphabet = sys.argv[1]
mapping = OrderedDict({"0": 9250})
for idx, char_ in enumerate(sorted(alphabet)):
    mapping[str(idx+1)] = ord(char_)
print(json.dumps(mapping, sort_keys=False, indent=4))

so the final json is:

{
    "0": 9250,
    "1": 43,
    "2": 48,
    "3": 49,
    "4": 50,
    "5": 51,
    "6": 52,
    "7": 53,
    "8": 54,
    "9": 55,
    "10": 56,
    "11": 57
}

Now I'm trying to use it in my training:

train_text_recognition.py --blank-label 0 --char-map $CHARMAP --gpus 0 --batch-size 400 $DATASET/curriculum.json $DATASET/logs

This leads to an exception:

Traceback (most recent call last):
  File "train_text_recognition.py", line 295, in <module>
    trainer.run()
  File "…/site-packages/chainer/training/trainer.py", line 313, in run
    six.reraise(*sys.exc_info())
  File "…/site-packages/six.py", line 693, in reraise
    raise value
  File "…/site-packages/chainer/training/trainer.py", line 299, in run
    update()
  File "…/site-packages/chainer/training/updater.py", line 223, in update
    self.update_core()
  File "…/site-packages/chainer/training/updaters/multiprocess_parallel_updater.py", line 207, in update_core
    loss = _calc_loss(self._master, batch)
  File "…/site-packages/chainer/training/updaters/multiprocess_parallel_updater.py", line 236, in _calc_loss
    return model(*in_arrays)
  File "…/see.git/chainer/utils/multi_accuracy_classifier.py", line 49, in __call__
    reported_accuracies = self.accfun(self.y, t)
  File "…/see.git/chainer/metrics/textrec_metrics.py", line 47, in calc_accuracy
    word = "".join(map(self.label_to_char, word))
  File "…/see.git/chainer/metrics/loss_metrics.py", line 182, in label_to_char
    return chr(self.char_map[str(label)])
KeyError: '14'

(The actual number is different for each run.)

Is there a certain number of items I need to have in my charmap? ctc_char_map.json in "text_recognition_model" contains 52 keys.

And one related/follow-up question: If I use the original ctc_char_map.json training works but the model seems not recognize my test images. I ran the training for 10 epochs with 5000 images in my training set. How many images should I train (for how many epochs) to get somewhat good results?

---

Just for completeness - my train.csv looks like this:

8   1
…/train/82368333.jpg    82368333

[Bartzi]

Yes,

your problem is this line, here you have to add the length of your char_map

[FelixSchwarz]

Thanks - that fixed it :-)