search

mittagessen / kraken

OCR engine for all the languages
http://kraken.re
Apache License 2.0
724 stars 130 forks source link

maximum recursion during a test #530

Closed lauxley closed 5 months ago

lauxley commented 1 year ago

Here is the full (skipping the duplicate lines) traceback:

test_train_new_transcription_model (core.tests.test_tasks.TasksTestCase) ... GPU available: False, used: False
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
`Trainer(val_check_interval=1.0)` was configured so validation will run at the end of the training epoch..
┏━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
┃    ┃ Name      ┃ Type            ┃ Params ┃       In sizes ┃       Out sizes ┃
┡━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
│ 0  │ val_cer   │ _CharErrorRate  │      0 │              ? │               ? │
│ 1  │ val_wer   │ _WordErrorRate  │      0 │              ? │               ? │
│ 2  │ net       │ MultiParamSequ… │  4.0 M │   [[1, 1, 120, │     [[1, 54, 1, │
│    │           │                 │        │     400], '?'] │       50], '?'] │
│ 3  │ net.C_0   │ ActConv2D       │  1.3 K │   [[1, 1, 120, │   [[1, 32, 120, │
│    │           │                 │        │     400], '?'] │      400], '?'] │
│ 4  │ net.Do_1  │ Dropout         │      0 │  [[1, 32, 120, │   [[1, 32, 120, │
│    │           │                 │        │     400], '?'] │      400], '?'] │
│ 5  │ net.Mp_2  │ MaxPool         │      0 │  [[1, 32, 120, │    [[1, 32, 60, │
│    │           │                 │        │     400], '?'] │      200], '?'] │
│ 6  │ net.C_3   │ ActConv2D       │ 40.0 K │   [[1, 32, 60, │    [[1, 32, 60, │
│    │           │                 │        │     200], '?'] │      200], '?'] │
│ 7  │ net.Do_4  │ Dropout         │      0 │   [[1, 32, 60, │    [[1, 32, 60, │
│    │           │                 │        │     200], '?'] │      200], '?'] │
│ 8  │ net.Mp_5  │ MaxPool         │      0 │   [[1, 32, 60, │    [[1, 32, 30, │
│    │           │                 │        │     200], '?'] │      100], '?'] │
│ 9  │ net.C_6   │ ActConv2D       │ 55.4 K │   [[1, 32, 30, │    [[1, 64, 30, │
│    │           │                 │        │     100], '?'] │      100], '?'] │
│ 10 │ net.Do_7  │ Dropout         │      0 │   [[1, 64, 30, │    [[1, 64, 30, │
│    │           │                 │        │     100], '?'] │      100], '?'] │
│ 11 │ net.Mp_8  │ MaxPool         │      0 │   [[1, 64, 30, │    [[1, 64, 15, │
│    │           │                 │        │     100], '?'] │       50], '?'] │
│ 12 │ net.C_9   │ ActConv2D       │  110 K │   [[1, 64, 15, │    [[1, 64, 15, │
│    │           │                 │        │      50], '?'] │       50], '?'] │
│ 13 │ net.Do_10 │ Dropout         │      0 │   [[1, 64, 15, │    [[1, 64, 15, │
│    │           │                 │        │      50], '?'] │       50], '?'] │
│ 14 │ net.S_11  │ Reshape         │      0 │   [[1, 64, 15, │    [[1, 960, 1, │
│    │           │                 │        │      50], '?'] │       50], '?'] │
│ 15 │ net.L_12  │ TransposedSumm… │  1.9 M │   [[1, 960, 1, │    [[1, 400, 1, │
│    │           │                 │        │      50], '?'] │       50], '?'] │
│ 16 │ net.Do_13 │ Dropout         │      0 │   [[1, 400, 1, │    [[1, 400, 1, │
│    │           │                 │        │      50], '?'] │       50], '?'] │
│ 17 │ net.L_14  │ TransposedSumm… │  963 K │   [[1, 400, 1, │    [[1, 400, 1, │
│    │           │                 │        │      50], '?'] │       50], '?'] │
│ 18 │ net.Do_15 │ Dropout         │      0 │   [[1, 400, 1, │    [[1, 400, 1, │
│    │           │                 │        │      50], '?'] │       50], '?'] │
│ 19 │ net.L_16  │ TransposedSumm… │  963 K │   [[1, 400, 1, │    [[1, 400, 1, │
│    │           │                 │        │      50], '?'] │       50], '?'] │
│ 20 │ net.Do_17 │ Dropout         │      0 │   [[1, 400, 1, │    [[1, 400, 1, │
│    │           │                 │        │      50], '?'] │       50], '?'] │
│ 21 │ net.O_18  │ LinSoftmax      │ 21.7 K │   [[1, 400, 1, │     [[1, 54, 1, │
│    │           │                 │        │      50], '?'] │       50], '?'] │
└────┴───────────┴─────────────────┴────────┴────────────────┴─────────────────┘
Trainable params: 4.0 M                                                         
Non-trainable params: 0                                                         
Total params: 4.0 M                                                             
Total estimated model params size (MB): 16                                      
maximum recursion depth exceeded while calling a Python object
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/kraken/lib/dataset/recognition.py", line 400, in __getitem__
    im, _ = next(extract_polygons(im, {'type': 'baselines',
  File "/usr/local/lib/python3.8/site-packages/kraken/lib/segmentation.py", line 987, in extract_polygons
    raise KrakenInputException('Line polygon outside of image bounds')
kraken.lib.exceptions.KrakenInputException: Line polygon outside of image bounds
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/kraken/lib/dataset/recognition.py", line 400, in __getitem__
    im, _ = next(extract_polygons(im, {'type': 'baselines',
  File "/usr/local/lib/python3.8/site-packages/kraken/lib/segmentation.py", line 987, in extract_polygons
    raise KrakenInputException('Line polygon outside of image bounds')
kraken.lib.exceptions.KrakenInputException: Line polygon outside of image bounds
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/kraken/lib/dataset/recognition.py", line 400, in __getitem__
    im, _ = next(extract_polygons(im, {'type': 'baselines',
  File "/usr/local/lib/python3.8/site-packages/kraken/lib/segmentation.py", line 987, in extract_polygons
    raise KrakenInputException('Line polygon outside of image bounds')
kraken.lib.exceptions.KrakenInputException: Line polygon outside of image bounds
[...]
Traceback (most recent call last):
  File "/home/robin/Projects/escriptorium/app/env/lib/python3.9/site-packages/kraken/lib/dataset/recognition.py", line 400, in __getitem__
    im, _ = next(extract_polygons(im, {'type': 'baselines',
  File "/home/robin/Projects/escriptorium/app/env/lib/python3.9/site-packages/kraken/lib/segmentation.py", line 987, in extract_polygons
    raise KrakenInputException('Line polygon outside of image bounds')
kraken.lib.exceptions.KrakenInputException: Line polygon outside of image bounds

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/robin/Projects/escriptorium/app/env/lib/python3.9/site-packages/kraken/lib/dataset/recognition.py", line 400, in __getitem__
    im, _ = next(extract_polygons(im, {'type': 'baselines',
  File "/home/robin/Projects/escriptorium/app/env/lib/python3.9/site-packages/kraken/lib/segmentation.py", line 987, in extract_polygons
    raise KrakenInputException('Line polygon outside of image bounds')
kraken.lib.exceptions.KrakenInputException: Line polygon outside of image bounds

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/robin/Projects/escriptorium/app/env/lib/python3.9/site-packages/kraken/lib/dataset/recognition.py", line 400, in __getitem__
    im, _ = next(extract_polygons(im, {'type': 'baselines',
  File "/home/robin/Projects/escriptorium/app/env/lib/python3.9/site-packages/kraken/lib/segmentation.py", line 987, in extract_polygons
    raise KrakenInputException('Line polygon outside of image bounds')
kraken.lib.exceptions.KrakenInputException: Line polygon outside of image bounds

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/robin/Projects/escriptorium/app/env/lib/python3.9/site-packages/kraken/lib/dataset/recognition.py", line 400, in __getitem__
    im, _ = next(extract_polygons(im, {'type': 'baselines',
  File "/home/robin/Projects/escriptorium/app/env/lib/python3.9/site-packages/kraken/lib/segmentation.py", line 976, in extract_polygons
    im = np.array(im)
  File "/home/robin/Projects/escriptorium/app/env/lib/python3.9/site-packages/PIL/Image.py", line 673, in __array_interface__
    new["data"] = self.tobytes()
  File "/home/robin/Projects/escriptorium/app/env/lib/python3.9/site-packages/PIL/Image.py", line 732, in tobytes
    self.load()
  File "/home/robin/Projects/escriptorium/app/env/lib/python3.9/site-packages/PIL/ImageFile.py", line 249, in load
    s = read(self.decodermaxblock)
  File "/home/robin/Projects/escriptorium/app/env/lib/python3.9/site-packages/PIL/PngImagePlugin.py", line 929, in load_read
    cid, pos, length = self.png.read()
  File "/home/robin/Projects/escriptorium/app/env/lib/python3.9/site-packages/PIL/PngImagePlugin.py", line 166, in read
    length = i32(s)
  File "/home/robin/Projects/escriptorium/app/env/lib/python3.9/site-packages/PIL/_binary.py", line 85, in i32be
    return unpack_from(">I", c, o)[0]
RecursionError: maximum recursion depth exceeded in comparison

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/robin/Projects/escriptorium/app/env/lib/python3.9/site-packages/torch/utils/data/_utils/worker.py", line 302, in _worker_loop
    data = fetcher.fetch(index)
  File "/home/robin/Projects/escriptorium/app/env/lib/python3.9/site-packages/torch/utils/data/_utils/fetch.py", line 58, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/robin/Projects/escriptorium/app/env/lib/python3.9/site-packages/torch/utils/data/_utils/fetch.py", line 58, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/robin/Projects/escriptorium/app/env/lib/python3.9/site-packages/torch/utils/data/dataset.py", line 295, in __getitem__
    return self.dataset[self.indices[idx]]
  File "/home/robin/Projects/escriptorium/app/env/lib/python3.9/site-packages/kraken/lib/dataset/recognition.py", line 423, in __getitem__
    return self[idx]
  File "/home/robin/Projects/escriptorium/app/env/lib/python3.9/site-packages/kraken/lib/dataset/recognition.py", line 423, in __getitem__
    return self[idx]
  File "/home/robin/Projects/escriptorium/app/env/lib/python3.9/site-packages/kraken/lib/dataset/recognition.py", line 423, in __getitem__
    return self[idx]
  [Previous line repeated 877 more times]
  File "/home/robin/Projects/escriptorium/app/env/lib/python3.9/site-packages/kraken/lib/dataset/recognition.py", line 421, in __getitem__
    logger.debug(traceback.format_exc())
  File "/usr/lib/python3.9/traceback.py", line 167, in format_exc
    return "".join(format_exception(*sys.exc_info(), limit=limit, chain=chain))
  File "/usr/lib/python3.9/traceback.py", line 120, in format_exception
    return list(TracebackException(
  File "/usr/lib/python3.9/traceback.py", line 517, in __init__
    self.stack = StackSummary.extract(
  File "/usr/lib/python3.9/traceback.py", line 366, in extract
    f.line
  File "/usr/lib/python3.9/traceback.py", line 288, in line
    self._line = linecache.getline(self.filename, self.lineno).strip()
  File "/usr/lib/python3.9/linecache.py", line 30, in getline
    lines = getlines(filename, module_globals)
  File "/usr/lib/python3.9/linecache.py", line 46, in getlines
    return updatecache(filename, module_globals)
  File "/usr/lib/python3.9/linecache.py", line 136, in updatecache
    with tokenize.open(fullname) as fp:
  File "/usr/lib/python3.9/tokenize.py", line 392, in open
    buffer = _builtin_open(filename, 'rb')
RecursionError: maximum recursion depth exceeded while calling a Python object

The code of the test can be found here: https://gitlab.com/scripta/escriptorium/-/blob/develop/app/apps/core/tests/test_tasks.py#L70
To note that the next test, that train on top of another model with the same data doesn't have the same issue.

mittagessen commented 1 year ago

Where do you define the data for the task? It looks like it is some problem in PIL but than it should happen in the following test as well. And is it happening consistently on each run?

lauxley commented 1 year ago

Yes it happens consistently and on different systems.