Issues with training using IAM dataset

[shenw000]

I cloned your repo on an Ubuntu 20.04 server, and tested train and inference commands with your included data and they all run correctly. Next, I ran train command for sanity check using IAM. For training data, I have first partition the IAM into train and val folder (90%+10% random split). For instance, in the train folder: a01-000u-00.png a01-000u-01.png a01-000u-02.png a01-000u-03.png a01-000u-05.png a01-000u-06.png a01-000x-02.png a01-000x-03.png a01-000x-04.png .....

Then in the gt folder, I created a labels.csv file using the same format as you included in your repo. here are a few lines at the beginning of the file: a01-000u-00.png,A MOVE to stop Mr. Gaitskell from a01-000u-01.png,nominating any more Labour life Peers a01-000u-02.png,is to be made at a meeting of Labour a01-000u-03.png,Ps tomorrow . Mr. Michael Foot has a01-000u-04.png,put down a resolution on the subject a01-000u-05.png,and he is to be backed by Mr. Will a01-000u-06.png,P for Manchester Exchange . a01-000x-00.png,A MOVE to stop Mr. Gaitskell from nominating a01-000x-01.png,any more Labour life Peers is to be made at a a01-000x-02.png,Ps tomorrow . Mr. Michael a01-000x-03.png,Foot has put down a resolution on the subject a01-000x-04.png,and he is to be backed by Mr. Will Griffiths , a01-000x-05.png,P for Manchester Exchange .

In other words, labels for all files in train and val folders are all included in the gt/labels.csv. Is that format correct for gt/labels.csv?

After I converted the IAM data to your above format, I ran the command python -m src train, which produces error as below:

Traceback (most recent call last): File "/home/shenw/workspace/anaconda3/envs/trocr-2/lib/python3.9/runpy.py", line 197, in _run_module_as_main return _run_code(code, main_globals, None, File "/home/shenw/workspace/anaconda3/envs/trocr-2/lib/python3.9/runpy.py", line 87, in _run_code exec(code, run_globals) File "/home/shenw/workspace/sandbox/htr/trocr-2/src/main.py", line 6, in main() File "/home/shenw/workspace/anaconda3/envs/trocr-2/lib/python3.9/site-packages/typer/main.py", line 214, in call return get_command(self)(*args, kwargs) File "/home/shenw/workspace/anaconda3/envs/trocr-2/lib/python3.9/site-packages/click/core.py", line 1130, in call return self.main(args, kwargs) File "/home/shenw/workspace/anaconda3/envs/trocr-2/lib/python3.9/site-packages/click/core.py", line 1055, in main rv = self.invoke(ctx) File "/home/shenw/workspace/anaconda3/envs/trocr-2/lib/python3.9/site-packages/click/core.py", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/shenw/workspace/anaconda3/envs/trocr-2/lib/python3.9/site-packages/click/core.py", line 1404, in invoke return ctx.invoke(self.callback, ctx.params) File "/home/shenw/workspace/anaconda3/envs/trocr-2/lib/python3.9/site-packages/click/core.py", line 760, in invoke return __callback(args, kwargs) File "/home/shenw/workspace/anaconda3/envs/trocr-2/lib/python3.9/site-packages/typer/main.py", line 500, in wrapper return callback(**use_params) # type: ignore File "/home/shenw/workspace/sandbox/htr/trocr-2/src/cli.py", line 10, in train main_train(local_model) File "/home/shenw/workspace/sandbox/htr/trocr-2/src/main.py", line 30, in main_train train_dataset = HCRDataset(paths.train_dir, processor) File "/home/shenw/workspace/sandbox/htr/trocr-2/src/dataset.py", line 54, in init self.image_name_list, self.label_list = load_filepaths_and_labels(data_dir) File "/home/shenw/workspace/sandbox/htr/trocr-2/src/dataset.py", line 41, in load_filepaths_and_labels assert file_name in label_dict, f"No label for image '{file_name}'" AssertionError: No label for image 'n02-045-01.png'

It appears that the label for n02-045-01.png is missing in gt/labels.csv. So I checked the label for n02-045-01.png in the gt/labels.csv, and it's there and I can display it correctly. Any idea why it produces above errors? Your help will be greatly appreciated.

[shenw000]

I found out that the problem of not recognizing the label for n02-045-01.png is due to the label contains double quotes. After removing double quotes from the labels.csv, the training went smoothly. My question is how can we include the double quotes in the training data without crashing the training process?

[rsommerfeld]

If you don't mind skipping examples that contain the double quotes, you can replace the assertion in line 41 of dataset.py with a continue statement, like:

if not file_name in label_dict:
                continue

However, on my machines (Linux / MacOS) adding double quotes works perfectly fine. Even the unicode char "😀" works as a label. So while I cannot reproduce it, maybe you need to escape your label strings to make it work. Alternatively you could replace all double quotes with some other special char

[shenw000]

Yes, it works by replacing double quote with a less frequently used symbol in the label filed of the labels.csv. Thank you very much!!!

[rsommerfeld]

Sure, glad it works now!