[Data] Annotation 실습 결과물 학습

[km9mn]

What?

• Upstage annotation tool로 캠퍼들이 만든 1288장의 데이터 baseline code로 학습

Todo

• [ ] Annotation 실습 결과물로만 학습
• [ ] ICDAR17_Korean (default)와 합쳐서 학습

[km9mn]

Traceback (most recent call last):
  File "train2.py", line 118, in <module>
    main(args)
  File "train2.py", line 111, in main
    do_training(**args.__dict__)
  File "train2.py", line 79, in do_training
    for img, gt_score_map, gt_geo_map, roi_mask in train_loader:
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 435, in __next__
    data = self._next_data()
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1085, in _next_data
    return self._process_data(data)
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1111, in _process_data
    data.reraise()
  File "/opt/conda/lib/python3.8/site-packages/torch/_utils.py", line 428, in reraise
    raise self.exc_type(msg)
OSError: Caught OSError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/_utils/worker.py", line 198, in _worker_loop
    data = fetcher.fetch(index)
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/opt/ml/code/east_dataset.py", line 136, in __getitem__
    image, word_bboxes, roi_mask = self.dataset[idx]
  File "/opt/ml/code/dataset.py", line 365, in __getitem__
    image, vertices = resize_img(image, vertices, self.image_size)
  File "/opt/ml/code/dataset.py", line 262, in resize_img
    img = img.resize((size, int(h * ratio)), Image.BILINEAR)
  File "/opt/conda/lib/python3.8/site-packages/PIL/Image.py", line 1989, in resize
    self.load()
  File "/opt/conda/lib/python3.8/site-packages/PIL/ImageFile.py", line 249, in load
    raise OSError(
OSError: image file is truncated (46 bytes not processed)

에러 발생

[km9mn]

dataset.py에

from PIL import ImageFile
ImageFile.LOAD_TRUNCATED_IMAGES = True

추가

Traceback (most recent call last):
  File "train2.py", line 118, in <module>
    main(args)
  File "train2.py", line 111, in main
    do_training(**args.__dict__)
  File "train2.py", line 79, in do_training
    for img, gt_score_map, gt_geo_map, roi_mask in train_loader:
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 435, in __next__
    data = self._next_data()
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1085, in _next_data
    return self._process_data(data)
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1111, in _process_data
    data.reraise()
  File "/opt/conda/lib/python3.8/site-packages/torch/_utils.py", line 428, in reraise
    raise self.exc_type(msg)
ValueError: Caught ValueError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/_utils/worker.py", line 198, in _worker_loop
    data = fetcher.fetch(index)
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/opt/ml/code/east_dataset.py", line 136, in __getitem__
    image, word_bboxes, roi_mask = self.dataset[idx]
  File "/opt/ml/code/dataset.py", line 361, in __getitem__
    vertices, labels = np.array(vertices, dtype=np.float32), np.array(labels, dtype=np.int64)
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (14,) + inhomogeneous part.

새로운 에러

[updaun]

  File "train.py", line 118, in <module>
    main(args)
  File "train.py", line 111, in main
    do_training(**args.__dict__)
  File "train.py", line 79, in do_training
    for img, gt_score_map, gt_geo_map, roi_mask in train_loader:
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 435, in __next__
    data = self._next_data()
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1085, in _next_data
    return self._process_data(data)
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1111, in _process_data
    data.reraise()
  File "/opt/conda/lib/python3.8/site-packages/torch/_utils.py", line 428, in reraise
    raise self.exc_type(msg)
ValueError: Caught ValueError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/_utils/worker.py", line 198, in _worker_loop
    data = fetcher.fetch(index)
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/opt/ml/code/east_dataset.py", line 136, in __getitem__
    image, word_bboxes, roi_mask = self.dataset[idx]
  File "/opt/ml/code/dataset.py", line 360, in __getitem__
    vertices, labels = np.array(vertices, dtype=np.float32), np.array(labels, dtype=np.int64)
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (63,) + inhomogeneous part.

저도 에러가 나네요! 규민님과 쪼금 다른 것 같긴한데 비슷한 에러인듯합니다!

[jeongjae96]

1632643837411-7.jpg
[array([ 564.64,  553.15, 1081.94,  553.15, 1081.94,  673.54,  564.64,
        673.54]), array([1211.69,  336.33, 1278.27,  336.33, 1278.27,  370.47, 1211.69,
        370.47]), array([755.44, 665.83, 798.81, 665.83, 798.81, 688.02, 755.44, 688.02]), array([798.81, 665.83, 830.61, 665.83, 830.61, 688.02, 798.81, 688.02]), array([842.17, 665.83, 881.69, 665.83, 881.69, 688.02, 842.17, 688.02])] [1, 1, 1, 1, 1]
  0%|                                                                                                                                                                     | 0/108 [00:01<?, ?it/s]
Traceback (most recent call last):
  File "train.py", line 118, in <module>
    main(args)
  File "train.py", line 111, in main
    do_training(**args.__dict__)
  File "train.py", line 79, in do_training
    for img, gt_score_map, gt_geo_map, roi_mask in train_loader:
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 435, in __next__
    data = self._next_data()
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1085, in _next_data
    return self._process_data(data)
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1111, in _process_data
    data.reraise()
  File "/opt/conda/lib/python3.8/site-packages/torch/_utils.py", line 428, in reraise
    raise self.exc_type(msg)
ValueError: Caught ValueError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/_utils/worker.py", line 198, in _worker_loop
    data = fetcher.fetch(index)
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/opt/ml/code/east_dataset.py", line 137, in __getitem__
    image, word_bboxes, roi_mask = self.dataset[idx]
  File "/opt/ml/code/dataset.py", line 366, in __getitem__
    vertices, labels = np.array(vertices, dtype=np.float32), np.array(labels, dtype=np.int64)
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (28,) + inhomogeneous part.

여기서 나는 에러 같은데, 뭐가 문제일까요? train.py의 train_loader shuffle=False로 하고 진행중인데, 똑같은 곳에서 오류가 항상 뜨네요.

[jeongjae96]

vertices = np.array(vertices, dtype=np.float32)

이 과정에서 오류가 발생하는 것 같네요.

[jeongjae96]

1595697184898-0.jpg 에서 문제가 생기는 것으로 보이네요! 이 파일의 annotation이 어떻게 됐는지 분석해보면 해결할 수 있을 것 같습니다.