"IndexError: too many indices for array" in lib/datasets/layout_coco.py

[haruka0000]

There is like below error after epoch 000 learned. I followed README however I couldn't. Are other libraries' vesions different. For example COCOAPI, numpy, opencv?

Epoch 000, iter 0013410:
loss:  7.487740596005087 0.0 0.0
accu:  [0.53364956 0.00988664 0.3089692  0.45671326]
-------------------------
0 0
0 1
0 2
0 3
0 4
0 5
0 6
0 7
0 8
0 9
0 10
0 11
0 12
0 13
0 14
Traceback (most recent call last):
  File "./tools/train_layout.py", line 46, in <module>
    train_model(config)
  File "./tools/train_layout.py", line 31, in train_model
    trainer.train(train_db, test_db, val_db)   
  File "/home/haruka/Workspace/Text2Scene/tools/../lib/modules/layout_trainer.py", line 218, in train
    val_loss, val_accu, val_infos = self.validate_epoch(val_db)
  File "/home/haruka/Workspace/Text2Scene/tools/../lib/modules/layout_trainer.py", line 348, in validate_epoch
    for cnt, batched in enumerate(val_loader):
  File "/opt/conda/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 615, in __next__
    batch = self.collate_fn([self.dataset[i] for i in indices])
  File "/opt/conda/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 615, in <listcomp>
    batch = self.collate_fn([self.dataset[i] for i in indices])
  File "/home/haruka/Workspace/Text2Scene/tools/../lib/datasets/layout_coco.py", line 102, in __getitem__
    out_inds, out_msks = self.scene_to_output_inds(scene)  
  File "/home/haruka/Workspace/Text2Scene/tools/../lib/datasets/layout_coco.py", line 502, in scene_to_output_inds
    other_inds = self.boxes2indices(boxes)
  File "/home/haruka/Workspace/Text2Scene/tools/../lib/datasets/layout_coco.py", line 479, in boxes2indices
    coord_inds = self.loc_map.coords2indices(boxes[:,:2])
IndexError: too many indices for array

These are libraries' versions I uesd.

annoy==1.16.2
asn1crypto==0.24.0
attrs==19.3.0
backcall==0.1.0
beautifulsoup4==4.7.1
bleach==3.1.0
certifi==2019.9.11
cffi==1.12.3
chardet==3.0.4
cloudpickle==1.2.2
conda==4.7.12
conda-build==3.17.8
conda-package-handling==1.6.0
cryptography==2.6.1
cycler==0.10.0
Cython==0.29.13
cytoolz==0.10.0
dask==2.6.0
decorator==4.4.0
defusedxml==0.6.0
entrypoints==0.3
filelock==3.0.10
glob2==0.6
idna==2.8
imageio==2.6.1
importlib-metadata==0.23
ipykernel==5.1.3
ipython==7.5.0
ipython-genutils==0.2.0
ipywidgets==7.5.1
jedi==0.13.3
Jinja2==2.10.1
jsonschema==3.1.1
jupyter==1.0.0
jupyter-client==5.3.4
jupyter-console==6.0.0
jupyter-core==4.6.1
kiwisolver==1.1.0
libarchive-c==2.8
lief==0.9.0
MarkupSafe==1.1.1
matplotlib==3.1.1
mistune==0.8.4
mkl-fft==1.0.12
mkl-random==1.0.2
more-itertools==7.2.0
nbconvert==5.6.1
nbformat==4.4.0
networkx==2.4
nltk==3.4.5
notebook==6.0.1
numpy==1.16.3
oauthlib==3.1.0
olefile==0.46
pandas==0.25.2
pandocfilters==1.4.2
parso==0.4.0
patsy==0.5.1
pexpect==4.7.0
pickleshare==0.7.5
Pillow==6.0.0
pkginfo==1.5.0.1
prometheus-client==0.7.1
prompt-toolkit==2.0.9
psutil==5.6.2
ptyprocess==0.6.0
pycairo==1.18.1
pycocotools==2.0
pycosat==0.6.3
pycparser==2.19
Pygments==2.3.1
pyOpenSSL==19.0.0
pyparsing==2.4.2
pyrsistent==0.15.4
PySocks==1.6.8
python-dateutil==2.8.0
pytz==2019.1
PyWavelets==1.1.1
PyYAML==5.1
pyzmq==18.1.0
qtconsole==4.5.5
requests==2.21.0
requests-oauthlib==1.3.0
ruamel-yaml==0.15.46
scikit-image==0.15.0
scipy==1.2.1
seaborn==0.9.0
Send2Trash==1.5.0
six==1.12.0
soupsieve==1.8
statsmodels==0.10.1
terminado==0.8.2
testpath==0.4.2
toolz==0.10.0
torch==1.0.1
torchtext==0.4.0
torchvision==0.2.1
tornado==6.0.3
tqdm==4.31.1
traitlets==4.3.2
urllib3==1.24.2
wcwidth==0.1.7
webencodings==0.5.1
widgetsnbextension==3.5.1
zipp==0.6.0

[haruka0000]

I added "print" to that error line.

lib/datasets/layout_coco.py

def boxes2indices(self, boxes):
        print(boxes.shape)  # This line added
        coord_inds = self.loc_map.coords2indices(boxes[:,:2])
        trans_inds = self.trans_map.whs2indices(boxes[:,2:])
        out_inds = np.concatenate([coord_inds.reshape((-1, 1)), trans_inds], -1)
        return out_inds

And, I checked output. There is a different shape. Maybe, that training data of layout COCO contains wrong data?

0 14
(4, 4)
(6, 4)
(2, 4)
(9, 4)
(2, 4)
(8, 4)
(9, 4)
(24, 4)
(3, 4)
(4, 4)
(0,) # This

I didn't check which data yet.

[fwtan]

Hi haruka0000, it seems the model predicts no box in this case. I will double check the codes and get back to you by this weekend.

[haruka0000]

Thank you for your reply. Okay, I'll wait.

[haruka0000]

I fixed a bit like below. After that, it is training well. But not finished yet. lib/datasets/layout_coco.py line number 134

        # scenedb = [self.load_coco_annotation(index) for index in self.image_index]
        """
        Exclude boxes with zero dim.
        """
        scenedb = []
        for index in self.image_index:
            if  self.load_coco_annotation(index)['boxes'].shape != np.array([]).shape:
                scenedb.append(self.load_coco_annotation(index))

It can train scenes having boxes temporarily. However, it can't use all exiting data.

I'm sorry to post a lot of comments.

[fwtan]

The error is fixed. Basically it is because some of the images have no box annotations. I actually remove all of these cases for the training and validation splits. But I want to keep such images in the test set so that we can evaluate all images in the testset. The error is in the training script where in my old codes, the val2017 split of COCO was named "val" but was actually used as "test" set.

[fwtan]

Also, for the layout generation model, the default training epoch number is 50. Actually the model will converge in fewer epochs, so 50 is actually the upper bound. You may want to check the validation accuracy for each epoch and pick the one with the best val performance. You can also speed up the training using multiple GPUs, by adding --parallel in the training script.

[haruka0000]

Thank you for your answering. I understood why that error didn't happen before. I'll try to train with fixed scripts and epoch 50.