PonteIneptique / YALTAi

You Actually Look Twice At it
GNU General Public License v3.0
29 stars 1 forks source link

TypeError: 'tuple' object does not support item assignment #22

Closed johnlockejrr closed 1 month ago

johnlockejrr commented 2 months ago
OS: Ubuntu 22.04.4 LTS x64 under WSL2
Python: 3.11.0rc1
Virtualenv

Trying to set my dataset then train but I get this error:

(env-py3.11) incognito@DESKTOP-H1BS9PO:~/YALTAi$ yaltai alto-to-yolo sam_new/*.xml sam_seg --shuffle .1 --segmonto region
Using list of inputs.
Found 166 to convert.
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /home/incognito/YALTAi/env-py3.11/bin/yaltai:8 in <module>                                       │
│                                                                                                  │
│   5 from yaltai.yaltai import cli                                                                │
│   6 if __name__ == '__main__':                                                                   │
│   7 │   sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])                         │
│ ❱ 8 │   sys.exit(cli())                                                                          │
│   9                                                                                              │
│                                                                                                  │
│ /home/incognito/YALTAi/env-py3.11/lib/python3.11/site-packages/click/core.py:1157 in __call__    │
│                                                                                                  │
│ /home/incognito/YALTAi/env-py3.11/lib/python3.11/site-packages/click/core.py:1078 in main        │
│                                                                                                  │
│ /home/incognito/YALTAi/env-py3.11/lib/python3.11/site-packages/click/core.py:1688 in invoke      │
│                                                                                                  │
│ /home/incognito/YALTAi/env-py3.11/lib/python3.11/site-packages/click/core.py:1434 in invoke      │
│                                                                                                  │
│ /home/incognito/YALTAi/env-py3.11/lib/python3.11/site-packages/click/core.py:783 in invoke       │
│                                                                                                  │
│ /home/incognito/YALTAi/env-py3.11/lib/python3.11/site-packages/yaltai/yaltai.py:98 in convert    │
│                                                                                                  │
│    95 │   if val:                                                                                │
│    96 │   │   message(f"{len(val)} image for validation.", fg='green')                           │
│    97 │   elif shuffle:                                                                          │
│ ❱  98 │   │   random.shuffle(input_paths)                                                        │
│    99 │   │   val_idx = int(len(input_paths) * shuffle)                                          │
│   100 │   │   message(f"{val_idx+1}/{len(input_paths)} image for validation.", fg='green')       │
│   101                                                                                            │
│                                                                                                  │
│ /usr/lib/python3.11/random.py:380 in shuffle                                                     │
│                                                                                                  │
│   377 │   │   for i in reversed(range(1, len(x))):                                               │
│   378 │   │   │   # pick an element in x[:i+1] with which to exchange x[i]                       │
│   379 │   │   │   j = randbelow(i + 1)                                                           │
│ ❱ 380 │   │   │   x[i], x[j] = x[j], x[i]                                                        │
│   381 │                                                                                          │
│   382 │   def sample(self, population, k, *, counts=None):                                       │
│   383 │   │   """Chooses k unique random elements from a population sequence.                    │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
TypeError: 'tuple' object does not support item assignment
PonteIneptique commented 2 months ago

Hi @johnlockejrr I found the bug today as well but was not able to find the time to solve it. My recommandation:

I hope I'll be able to get to the issue next week.

johnlockejrr commented 2 months ago

Ok, I'll do that until you fix it!

By the way, if I don't shuffle I get this error:

NameError: name 'parse_xml' is not defined

PonteIneptique commented 2 months ago

This error is definitely due to the new version. I am now worried the other bug is not connected to the new version, please keep me up to date

johnlockejrr commented 2 months ago

I trined a model with YALTAi 1.0.2. I realized it extracted only the text regions from my PAGE XML (exported from eScriptorium) and not the lines... is that normal? Last days I "eat" my brains working with YOLOv8 segmentation, wrote some scripts to extract polygons from PAGE XML datasets and convert them to YOLOv8 OBB, thats how I found (refound actually) your project. I needed a good text segmentation to extract text lines and send them to a PyLaia model for text recognition (https://huggingface.co/spaces/johnlockejrr/PyLaia-heb_sam_v1).

johnlockejrr commented 2 months ago

About the second error, just reading your code I see in yaltai.py that parse_xml is commented:

#from kraken.lib.xml import parse_xml

Why? :)

Edit...

ImportError: cannot import name 'parse_xml' from 'kraken.lib.xml' (/home/incognito/YALTAi/train-2.0.1-py3.11/lib/python3.11/site-packages/kraken/lib/xml.py)
johnlockejrr commented 2 months ago

Seems like in kraken now you should do from kraken.lib import xml then xml.XMLPage(doc, format_type) etc.

From kraken 5.x release notes:

While 5.x preserves the general OCR functional blocks, the existing dictionary-based data structures have been replaced with [container classes](https://kraken.re/5.2/api_docs.html#kraken-containers-module) and the XML parser has been reworked

See https://github.com/mittagessen/kraken/releases/tag/5.2 under API changes

PonteIneptique commented 2 months ago

Coming back to it today :)

johnlockejrr commented 2 months ago

Thanks to God! :D Can't wait!

PonteIneptique commented 1 month ago

This issue should be fixed but warning I changed the command to simplify things:

yaltai alto-to-yolo sam_new/*.xml sam_seg --shuffle .1 --segmonto region

becomes yaltai convert alto-to-yolo ....

johnlockejrr commented 1 month ago

Perfect! I will clone the latest and give it a try. And perfect you changed the command. Should I install with pip install YALTAi already or dirrectly from github? And what python version do you recommand?

PonteIneptique commented 1 month ago

pip install 'yaltai==2.0.2' should be good

PonteIneptique commented 1 month ago

Can you confirm this one so that I could close this one at least ? :)

johnlockejrr commented 1 month ago

Same error:

(yaltai-2.0.2-py3.10) incognito@DESKTOP-H1BS9PO:~/YALTAi$ yaltai convert alto-to-yolo teyman_alto/*.xml datasets/teyman_alto --shuffle .1
Using list of inputs.
Found 70 to convert.
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /home/incognito/YALTAi/yaltai-2.0.2-py3.10/bin/yaltai:8 in <module>                              │
│                                                                                                  │
│   5 from yaltai.cli.yaltai import yaltai_cli                                                     │
│   6 if __name__ == '__main__':                                                                   │
│   7 │   sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])                         │
│ ❱ 8 │   sys.exit(yaltai_cli())                                                                   │
│   9                                                                                              │
│                                                                                                  │
│ /home/incognito/YALTAi/yaltai-2.0.2-py3.10/lib/python3.10/site-packages/click/core.py:1157 in    │
│ __call__                                                                                         │
│                                                                                                  │
│ /home/incognito/YALTAi/yaltai-2.0.2-py3.10/lib/python3.10/site-packages/click/core.py:1078 in    │
│ main                                                                                             │
│                                                                                                  │
│ /home/incognito/YALTAi/yaltai-2.0.2-py3.10/lib/python3.10/site-packages/click/core.py:1688 in    │
│ invoke                                                                                           │
│                                                                                                  │
│ /home/incognito/YALTAi/yaltai-2.0.2-py3.10/lib/python3.10/site-packages/click/core.py:1688 in    │
│ invoke                                                                                           │
│                                                                                                  │
│ /home/incognito/YALTAi/yaltai-2.0.2-py3.10/lib/python3.10/site-packages/click/core.py:1434 in    │
│ invoke                                                                                           │
│                                                                                                  │
│ /home/incognito/YALTAi/yaltai-2.0.2-py3.10/lib/python3.10/site-packages/click/core.py:783 in     │
│ invoke                                                                                           │
│                                                                                                  │
│ /home/incognito/YALTAi/yaltai-2.0.2-py3.10/lib/python3.10/site-packages/yaltai/cli/yaltai.py:98  │
│ in alto_to_yolo                                                                                  │
│                                                                                                  │
│    95 │   if val:                                                                                │
│    96 │   │   message(f"{len(val)} image for validation.", fg='green')                           │
│    97 │   elif shuffle:                                                                          │
│ ❱  98 │   │   random.shuffle(input_paths)                                                        │
│    99 │   │   val_idx = int(len(input_paths) * shuffle)                                          │
│   100 │   │   message(f"{val_idx+1}/{len(input_paths)} image for validation.", fg='green')       │
│   101                                                                                            │
│                                                                                                  │
│ /usr/lib/python3.10/random.py:394 in shuffle                                                     │
│                                                                                                  │
│   391 │   │   │   for i in reversed(range(1, len(x))):                                           │
│   392 │   │   │   │   # pick an element in x[:i+1] with which to exchange x[i]                   │
│   393 │   │   │   │   j = randbelow(i + 1)                                                       │
│ ❱ 394 │   │   │   │   x[i], x[j] = x[j], x[i]                                                    │
│   395 │   │   else:                                                                              │
│   396 │   │   │   _warn('The *random* parameter to shuffle() has been deprecated\n'              │
│   397 │   │   │   │     'since Python 3.9 and will be removed in a subsequent '                  │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
TypeError: 'tuple' object does not support item assignment
PonteIneptique commented 1 month ago

I missinterpreted your problem, and just fixed it I guess. You can confirm by installing 2.0.3

johnlockejrr commented 1 month ago

Works now, but with a little problem that I saw in the old ones, it finds only one class even I have two. In my case here, I have textzone and textline, the textline is not detected. But I think is not YALTAi problem but an eScriptorium one, maybe not even eScriptorium problem but mine, I named my lines textline and eScriptorium/kraken got confused because ALTO has a tag named TextLine.

(yaltai-2.0.3-py3.11) incognito@DESKTOP-H1BS9PO:~/YALTAi$ yaltai convert alto-to-yolo teyman_alto/*.xml teyman_alto_test --shuffle .1
Using list of inputs.
Found 70 to convert.
HELLLOOO MOTHER FUCKER
8/70 image for validation.
Shuffling data with a ratio of 0.1 for validation.
70it [00:00, 213.93it/s]
70 ground truth XML files converted.
Configuration available at teyman_alto_test/config.yml.
Label Map available at teyman_alto_test/labelmap.txt.
Regions count:
        - 00070 textzone
johnlockejrr commented 1 month ago

One question: is this really YOLOv8 because looks for me to be YOLOv5?

PonteIneptique commented 1 month ago

Yes it is :)

johnlockejrr commented 1 month ago

Yes it is :)

You are right.

(yaltai-2.0.3-py3.11) incognito@DESKTOP-H1BS9PO:~/YALTAi$ yolo version
8.0.209