search

VoxelCubes / PanelCleaner

An AI-powered tool to clean manga panels.
GNU General Public License v3.0
242 stars 18 forks source link

Running Denoiser... (Stuck) #14

Open justme7678876788 opened 1 year ago

justme7678876788 commented 1 year ago

Hi . after testing this on windows and not working i went to linux . after i installed it in linux i had some problems and they were all about the requirements versions which i found the right version and everything seems to go fine until the last step when it does this :

abder@abder-X75A1:~/Downloads$ /home/abder/.local/bin/pcleaner clean index.jpeg
Found 1 image.
Running text detection AI model...
Using device for text detection model: cpu
Using 1 processes for text detection.
100%|█████████████████████████████████████████████| 1/1 [00:13<00:00, 13.91s/it]

Running box data Pre-Processor...
2023-05-31 15:51:41.961 | INFO     | manga_ocr.ocr:__init__:13 - Loading OCR model from kha-white/manga-ocr-base
2023-05-31 15:51:53.992 | INFO     | manga_ocr.ocr:__init__:22 - Using CPU
2023-05-31 15:51:57.097 | INFO     | manga_ocr.ocr:__init__:29 - OCR ready
100%|█████████████████████████████████████████████| 1/1 [00:03<00:00,  3.72s/it]

OCR Analytics
-------------
Number of boxes: 2 | Number of small boxes: 2 (100%)
Number of removed boxes: 0 (0% total, 0% of small boxes)

Small box sizes:
   0- 500:  0 / 0
 500-1000:  0 / 0
1000-1500:  0 / 0
1500-2000:  0 / 0
2000-2500: ███████████████████████████████████████████████████████ 0 / 2
2500-3000:  0 / 0

█ Small boxes | █ Removed boxes

Removed bubbles:

Running Masker...
100%|█████████████████████████████████████████████| 1/1 [00:01<00:00,  1.64s/it]

Mask Fitment Analytics
----------------------
Total boxes: 2 | Masks succeeded: 0 (0%) | Masks failed: 2
Perfect masks: 0 (N/A) | Average border deviation: N/A

Mask usage by mask size (smallest to largest):
Box mask:  0 / 0

█ Perfect | █ Total

Pages with failures / total:
index.png: 2 / 2

Running Denoiser...
  0%|                                                     | 0/1 [00:00<?, ?it/s]
multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/home/abder/.local/lib/python3.10/site-packages/pcleaner/denoiser.py", line 29, in denoise_page
    cleaned_image.paste(mask_image, (0, 0), mask_image)
  File "/usr/lib/python3/dist-packages/PIL/Image.py", line 1557, in paste
    self.im.paste(im, box, mask.im)
ValueError: bad transparency mask
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/abder/.local/bin/pcleaner", line 8, in <module>
    sys.exit(main())
  File "/home/abder/.local/lib/python3.10/site-packages/pcleaner/main.py", line 219, in main
    run_cleaner(
  File "/home/abder/.local/lib/python3.10/site-packages/pcleaner/main.py", line 418, in run_cleaner
    for analytic in tqdm(pool.imap(dn.denoise_page, data), total=len(data)):
  File "/home/abder/.local/lib/python3.10/site-packages/tqdm/std.py", line 1178, in __iter__
    for obj in iterable:
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 873, in next
    raise value
ValueError: bad transparency mask

and here it is with another image : 

abder@abder-X75A1:/media/abder/Stuffs/Manga/Blade of the Immortal (1997-2015) (Digital) (danke-Empire)/Blade of the Immortal v01 - Blood of a Thousand (1997) (Digital) (danke-Empire)$ /home/abder/.local/bin/pcleaner clean im7.jpg
Found 1 image.
Running text detection AI model...
Using device for text detection model: cpu
Using 1 processes for text detection.
100%|█████████████████████████████████████████████| 1/1 [00:12<00:00, 12.52s/it]

Running box data Pre-Processor...
2023-05-31 15:54:55.353 | INFO     | manga_ocr.ocr:__init__:13 - Loading OCR model from kha-white/manga-ocr-base
2023-05-31 15:55:00.636 | INFO     | manga_ocr.ocr:__init__:22 - Using CPU
2023-05-31 15:55:02.815 | INFO     | manga_ocr.ocr:__init__:29 - OCR ready
100%|████████████████████████████████████████████| 1/1 [00:00<00:00, 478.36it/s]

OCR Analytics
-------------
Number of boxes: 2 | Number of small boxes: 0 (0%)
Number of removed boxes: 0 (0% total, N/A of small boxes)
No not-removed small boxes found.

Removed bubbles:

Running Masker...
100%|█████████████████████████████████████████████| 1/1 [00:02<00:00,  2.02s/it]

Mask Fitment Analytics
----------------------
Total boxes: 2 | Masks succeeded: 2 (100%) | Masks failed: 0
Perfect masks: 1 (50%) | Average border deviation: 0.06

Mask usage by mask size (smallest to largest):
Mask 0  :  0 / 0
Mask 1  :  0 / 0
Mask 2  :  0 / 0
Mask 3  :  0 / 0
Mask 4  :  0 / 0
Mask 5  :  0 / 0
Mask 6  :  0 / 0
Mask 7  :  0 / 0
Mask 8  :  0 / 0
Mask 9  :  0 / 0
Mask 10 : ████████████████████████████████████████████████████████ 1 / 1
Box mask: ████████████████████████████████████████████████████████ 0 / 1

█ Perfect | █ Total

Pages with failures / total:

Running Denoiser...
  0%|                                                     | 0/1 [00:00<?, ?it/s]
multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/home/abder/.local/lib/python3.10/site-packages/pcleaner/denoiser.py", line 29, in denoise_page
    cleaned_image.paste(mask_image, (0, 0), mask_image)
  File "/usr/lib/python3/dist-packages/PIL/Image.py", line 1557, in paste
    self.im.paste(im, box, mask.im)
ValueError: bad transparency mask
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/abder/.local/bin/pcleaner", line 8, in <module>
    sys.exit(main())
  File "/home/abder/.local/lib/python3.10/site-packages/pcleaner/main.py", line 219, in main
    run_cleaner(
  File "/home/abder/.local/lib/python3.10/site-packages/pcleaner/main.py", line 418, in run_cleaner
    for analytic in tqdm(pool.imap(dn.denoise_page, data), total=len(data)):
  File "/home/abder/.local/lib/python3.10/site-packages/tqdm/std.py", line 1178, in __iter__
    for obj in iterable:
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 873, in next
    raise value
ValueError: bad transparency mask
justme7678876788 commented 1 year ago

Btw . here's my problem in the windows version : 

E:\Manga\Blade of the Immortal (1997-2015) (Digital) (danke-Empire)\Blade of the Immortal v01 - Blood of a Thousand (1997) (Digital) (danke-Empire)>pcleaner clean im6.jpg
Found 1 image.
Running text detection AI model...
Using device for text detection model: cpu
Using 1 processes for text detection.
100%|████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:18<00:00, 18.77s/it]

Running box data Pre-Processor...
2023-05-31 19:33:32.080 | INFO     | manga_ocr.ocr:__init__:13 - Loading OCR model from kha-white/manga-ocr-base
2023-05-31 19:33:40.137 | INFO     | manga_ocr.ocr:__init__:22 - Using CPU
2023-05-31 19:33:44.564 | INFO     | manga_ocr.ocr:__init__:29 - OCR ready
  0%|                                                                                            | 0/1 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "C:\Users\Harrou abderrahmane\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\Harrou abderrahmane\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\Users\Harrou abderrahmane\AppData\Local\Programs\Python\Python310\Scripts\pcleaner.exe\__main__.py", line 7, in <module>
  File "C:\Users\Harrou abderrahmane\AppData\Local\Programs\Python\Python310\lib\site-packages\pcleaner\main.py", line 221, in main
    run_cleaner(
  File "C:\Users\Harrou abderrahmane\AppData\Local\Programs\Python\Python310\lib\site-packages\pcleaner\main.py", line 350, in run_cleaner
    ocr_analytic = pp.prep_json_file(
  File "C:\Users\Harrou abderrahmane\AppData\Local\Programs\Python\Python310\lib\site-packages\pcleaner\pre_processor.py", line 105, in prep_json_file
    page_data.grow_boxes(scale_len(pre_processor_conf.box_padding_initial), st.BoxType.BOX)
  File "C:\Users\Harrou abderrahmane\AppData\Local\Programs\Python\Python310\lib\site-packages\pcleaner\structures.py", line 105, in grow_boxes
    image_width, image_height = self.image_size
  File "C:\Users\Harrou abderrahmane\AppData\Local\Programs\Python\Python310\lib\site-packages\pcleaner\structures.py", line 78, in image_size
    metadata = magic.from_file(self.image_path)
AttributeError: module 'magic' has no attribute 'from_file'. Did you mean: '_image_size'?

it looks like a mistake in the code it self

justme7678876788 commented 1 year ago

after downloading manually the libmagic it seems like the error changed

E:\Manga\Blade of the Immortal (1997-2015) (Digital) (danke-Empire)\Blade of the Immortal v01 - Blood of a Thousand (1997) (Digital) (danke-Empire)>pcleaner clean im6.jpg
Found 1 image.
Running text detection AI model...
Using device for text detection model: cpu
Using 1 processes for text detection.
100%|████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:17<00:00, 17.89s/it]

Running box data Pre-Processor...
2023-05-31 20:28:42.812 | INFO     | manga_ocr.ocr:__init__:13 - Loading OCR model from kha-white/manga-ocr-base
2023-05-31 20:28:52.518 | INFO     | manga_ocr.ocr:__init__:22 - Using CPU
2023-05-31 20:28:56.608 | INFO     | manga_ocr.ocr:__init__:29 - OCR ready
  0%|                                                                                            | 0/1 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "C:\Users\Harrou abderrahmane\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\Harrou abderrahmane\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\Users\Harrou abderrahmane\AppData\Local\Programs\Python\Python310\Scripts\pcleaner.exe\__main__.py", line 7, in <module>
  File "C:\Users\Harrou abderrahmane\AppData\Local\Programs\Python\Python310\lib\site-packages\pcleaner\main.py", line 221, in main
    run_cleaner(
  File "C:\Users\Harrou abderrahmane\AppData\Local\Programs\Python\Python310\lib\site-packages\pcleaner\main.py", line 350, in run_cleaner
    ocr_analytic = pp.prep_json_file(
  File "C:\Users\Harrou abderrahmane\AppData\Local\Programs\Python\Python310\lib\site-packages\pcleaner\pre_processor.py", line 105, in prep_json_file
    page_data.grow_boxes(scale_len(pre_processor_conf.box_padding_initial), st.BoxType.BOX)
  File "C:\Users\Harrou abderrahmane\AppData\Local\Programs\Python\Python310\lib\site-packages\pcleaner\structures.py", line 105, in grow_boxes
    image_width, image_height = self.image_size
  File "C:\Users\Harrou abderrahmane\AppData\Local\Programs\Python\Python310\lib\site-packages\pcleaner\structures.py", line 79, in image_size
    size_str = re.search(r"(\d+) x (\d+)", metadata).groups()
AttributeError: 'NoneType' object has no attribute 'groups'. Did you mean: '_image_size'?

i followed this : https://github.com/hey-red/Mime/issues/9

VoxelCubes commented 1 year ago

Yeah, I haven't had the time to redo my windows vm and trying to get a build working there, so thanks for the tips on how I might be able to do that.

As for the denoiser, I personally have never run into that issue and wasn't able to reproduce it, but I can try catching whatever error that is and moving on, though I can't know for sure if it's fixed. Are you able to share one or more images with this problem (in a way that preserves the original file, so that it hopefully works the same)?

In the meantime, add the -D flag (or --skip-denoising) to your command or alternatively use a profile where denoising is explicitly disabled.

Thanks!

VoxelCubes commented 1 year ago

Unfortunately I still haven't been able to reproduce this issue, so I don't know to fix it. If you have images that cause issues, please email them to me at voxel.aur@gmail.com