search

microsoft / table-transformer

Table Transformer (TATR) is a deep learning model for extracting tables from unstructured documents (PDFs and images). This is also the official repository for the PubTables-1M dataset and GriTS evaluation metric.
MIT License
2.31k stars 256 forks source link

TypeError: 'numpy.float64' object cannot be interpreted as an integer during evaluation #67

Closed LxYuan-Handshakes closed 2 years ago

LxYuan-Handshakes commented 2 years ago

Hi,

I encountered this numpy type error during the evaluation phase. Any idea how to fix this?

How to reproduce the error

(env)$ python main.py 
  --data_type detection 
  --config_file detection_config.json 
  --data_root_dir ~/../pubtables/PubTables1M-Detection-PASCAL-VOC/

Error Message

{'lr': 5e-05, 'lr_backbone': 1e-05, 'batch_size': 2, 'weight_decay': 0.0001, 'epochs': 20, 'lr_drop': 1, 'lr_gamma': 0.9, 'clip_max_norm': 0.1, 'backbone': 'resnet18', 'num_classes': 2, 'dilation': False, 'position_
embedding': 'sine', 'emphasized_weights': {}, 'enc_layers': 6, 'dec_layers': 6, 'dim_feedforward': 2048, 'hidden_dim': 256, 'dropout': 0.1, 'nheads': 8, 'num_queries': 15, 'pre_norm': True, 'masks': False, 'aux_loss
': False, 'mask_loss_coef': 1, 'dice_loss_coef': 1, 'ce_loss_coef': 1, 'bbox_loss_coef': 5, 'giou_loss_coef': 2, 'eos_coef': 0.4, 'set_cost_class': 1, 'set_cost_bbox': 5, 'set_cost_giou': 2, 'device': 'cuda', 'seed'
: 42, 'start_epoch': 0, 'num_workers': 1, 'data_root_dir': '/home/lxyuan/../pubtables/PubTables1M-Detection-PASCAL-VOC/', 'config_file': 'detection_config.json', 'data_type': 'detection', 'model_load_path': None, 'l
oad_weights_only': False, 'model_save_dir': None, 'metrics_save_filepath': '', 'debug_save_dir': 'debug', 'table_words_dir': None, 'mode': 'train', 'debug': False, 'checkpoint_freq': 1, 'train_max_size': None, 'val_
max_size': None, 'test_max_size': None, 'eval_pool_size': 1, 'eval_step': 1, '__module__': '__main__', '__dict__': <attribute '__dict__' of 'Args' objects>, '__weakref__': <attribute '__weakref__' of 'Args' objects>
, '__doc__': None}
----------------------------------------------------------------------------------------------------
loading model
loading data
loading data
creating index...
index created!
finished loading data in : 0:00:04.291752
Max batches per epoch: 230294
Output directory:  /home/lxyuan/../pubtables/PubTables1M-Detection-PASCAL-VOC/output/20220815202559
Output model path:  /home/lxyuan/../pubtables/PubTables1M-Detection-PASCAL-VOC/output/20220815202559/model.pth
Start training
----------------------------------------------------------------------------------------------------
Epoch: [0]  [     0/230294]  eta: 21:14:22  lr: 0.000050  class_error: 33.33  loss: 7.6202 (7.6202)  loss_ce: 1.3217 (1.3217)  loss_bbox: 4.0440 (4.0440)  loss_giou: 2.2545 (2.2545)  loss_ce_unscaled: 1.3217 (1.3217
)  class_error_unscaled: 33.3333 (33.3333)  loss_bbox_unscaled: 0.8088 (0.8088)  loss_giou_unscaled: 1.1273 (1.1273)  cardinality_error_unscaled: 12.5000 (12.5000)  time: 0.3320  data: 0.1073  max mem: 796
Epoch: [0]  [  1000/230294]  eta: 5:37:18  lr: 0.000050  class_error: 100.00  loss: 2.3491 (3.9134)  loss_ce: 0.4271 (0.4534)  loss_bbox: 1.0936 (2.2053)  loss_giou: 0.8212 (1.2548)  loss_ce_unscaled: 0.4271 (0.4534
)  class_error_unscaled: 100.0000 (96.9789)  loss_bbox_unscaled: 0.2187 (0.4411)  loss_giou_unscaled: 0.4106 (0.6274)  cardinality_error_unscaled: 1.0000 (1.0919)  time: 0.0870  data: 0.0046  max mem: 1393
Epoch: [0]  [  2000/230294]  eta: 5:37:05  lr: 0.000050  class_error: 100.00  loss: 3.2153 (3.2962)  loss_ce: 0.3987 (0.4452)  loss_bbox: 1.6650 (1.7887)  loss_giou: 1.0126 (1.0623)  loss_ce_unscaled: 0.3987 (0.4452
)  class_error_unscaled: 100.0000 (94.8372)  loss_bbox_unscaled: 0.3330 (0.3577)  loss_giou_unscaled: 0.5063 (0.5312)  cardinality_error_unscaled: 1.0000 (1.0160)  time: 0.0845  data: 0.0045  max mem: 1393
Epoch: [0]  [  3000/230294]  eta: 5:35:23  lr: 0.000050  class_error: 100.00  loss: 2.4226 (2.9530)  loss_ce: 0.3809 (0.4328)  loss_bbox: 1.1276 (1.5422)  loss_giou: 0.8819 (0.9780)  loss_ce_unscaled: 0.3809 (0.4328
)  class_error_unscaled: 100.0000 (92.2951)  loss_bbox_unscaled: 0.2255 (0.3084)  loss_giou_unscaled: 0.4409 (0.4890)  cardinality_error_unscaled: 1.0000 (0.9888)  time: 0.0883  data: 0.0045  max mem: 1393
Epoch: [0]  [  4000/230294]  eta: 5:35:24  lr: 0.000050  class_error: 0.00  loss: 1.8408 (2.7103)  loss_ce: 0.3210 (0.4222)  loss_bbox: 0.7209 (1.3707)  loss_giou: 0.6109 (0.9174)  loss_ce_unscaled: 0.3210 (0.4222)
 class_error_unscaled: 50.0000 (89.8711)  loss_bbox_unscaled: 0.1442 (0.2741)  loss_giou_unscaled: 0.3055 (0.4587)  cardinality_error_unscaled: 0.5000 (0.9609)  time: 0.0906  data: 0.0049  max mem: 1393
Epoch: [0]  [  5000/230294]  eta: 5:34:34  lr: 0.000050  class_error: 0.00  loss: 2.0806 (2.5365)  loss_ce: 0.3440 (0.4120)  loss_bbox: 0.7721 (1.2546)  loss_giou: 0.7145 (0.8699)  loss_ce_unscaled: 0.3440 (0.4120)
 class_error_unscaled: 75.0000 (86.8418)  loss_bbox_unscaled: 0.1544 (0.2509)  loss_giou_unscaled: 0.3572 (0.4349)  cardinality_error_unscaled: 0.5000 (0.9352)  time: 0.0928  data: 0.0048  max mem: 1393
Epoch: [0]  [  6000/230294]  eta: 5:33:42  lr: 0.000050  class_error: 50.00  loss: 1.5561 (2.4004)  loss_ce: 0.3442 (0.4008)  loss_bbox: 0.5955 (1.1669)  loss_giou: 0.5303 (0.8327)  loss_ce_unscaled: 0.3442 (0.4008)
  class_error_unscaled: 66.6667 (82.8963)  loss_bbox_unscaled: 0.1191 (0.2334)  loss_giou_unscaled: 0.2652 (0.4163)  cardinality_error_unscaled: 0.5000 (0.8982)  time: 0.0910  data: 0.0048  max mem: 1393
Epoch: [0]  [  7000/230294]  eta: 5:32:53  lr: 0.000050  class_error: 100.00  loss: 1.9024 (2.2844)  loss_ce: 0.2432 (0.3884)  loss_bbox: 0.6760 (1.0965)  loss_giou: 0.6833 (0.7995)  loss_ce_unscaled: 0.2432 (0.3884
)  class_error_unscaled: 50.0000 (79.1719)  loss_bbox_unscaled: 0.1352 (0.2193)  loss_giou_unscaled: 0.3416 (0.3998)  cardinality_error_unscaled: 0.5000 (0.8579)  time: 0.0856  data: 0.0047  max mem: 1393
Epoch: [0]  [  8000/230294]  eta: 5:31:30  lr: 0.000050  class_error: 50.00  loss: 1.3197 (2.1904)  loss_ce: 0.2045 (0.3753)  loss_bbox: 0.5773 (1.0416)  loss_giou: 0.6363 (0.7734)  loss_ce_unscaled: 0.2045 (0.3753)
  class_error_unscaled: 33.3333 (75.1935)  loss_bbox_unscaled: 0.1155 (0.2083)  loss_giou_unscaled: 0.3182 (0.3867)  cardinality_error_unscaled: 0.0000 (0.8116)  time: 0.0903  data: 0.0047  max mem: 1393
Epoch: [0]  [  9000/230294]  eta: 5:30:25  lr: 0.000050  class_error: 100.00  loss: 1.2540 (2.1009)  loss_ce: 0.2317 (0.3612)  loss_bbox: 0.4740 (0.9915)  loss_giou: 0.5079 (0.7482)  loss_ce_unscaled: 0.2317 (0.3612
)  class_error_unscaled: 50.0000 (71.3004)  loss_bbox_unscaled: 0.0948 (0.1983)  loss_giou_unscaled: 0.2539 (0.3741)  cardinality_error_unscaled: 0.5000 (0.7655)  time: 0.0909  data: 0.0048  max mem: 1393

<truncated>

Epoch: [0]  [230293/230294]  eta: 0:00:00  lr: 0.000050  class_error: 0.00  loss: 0.2878 (0.4740)  loss_ce: 0.0005 (0.0355)  loss_bbox: 0.1188 (0.2152)  loss_giou: 0.1408 (0.2233)  loss_ce_unscaled: 0.0005 (0.0355)
 class_error_unscaled: 0.0000 (4.7529)  loss_bbox_unscaled: 0.0238 (0.0430)  loss_giou_unscaled: 0.0704 (0.1116)  cardinality_error_unscaled: 0.0000 (0.0790)  time: 0.0888  data: 0.0057  max mem: 1393
Epoch: [0] Total time: 5:45:45 (0.0901 s / it)
Averaged stats: lr: 0.000050  class_error: 0.00  loss: 0.2878 (0.4740)  loss_ce: 0.0005 (0.0355)  loss_bbox: 0.1188 (0.2152)  loss_giou: 0.1408 (0.2233)  loss_ce_unscaled: 0.0005 (0.0355)  class_error_unscaled: 0.00
00 (4.7529)  loss_bbox_unscaled: 0.0238 (0.0430)  loss_giou_unscaled: 0.0704 (0.1116)  cardinality_error_unscaled: 0.0000 (0.0790)
Epoch completed in  5:45:45.451181
    main()
  File "/home/lxyuan/playground/table-transformer/src/main.py", line 368, in main
    train(args, model, criterion, postprocessors, device)
  File "/home/lxyuan/playground/table-transformer/src/main.py", line 317, in train
    pubmed_stats, coco_evaluator = evaluate(model, criterion,
  File "/home/lxyuan/playground/table-transformer/env/lib64/python3.9/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/home/lxyuan/playground/table-transformer/src/../detr/engine.py", line 81, in evaluate
    coco_evaluator = CocoEvaluator(base_ds, iou_types)
  File "/home/lxyuan/playground/table-transformer/src/../detr/datasets/coco_eval.py", line 31, in __init__
    self.coco_eval[iou_type] = COCOeval(coco_gt, iouType=iou_type)
  File "/home/lxyuan/playground/table-transformer/env/lib64/python3.9/site-packages/pycocotools/cocoeval.py", line 76, in __init__
    self.params = Params(iouType=iouType) # parameters
  File "/home/lxyuan/playground/table-transformer/env/lib64/python3.9/site-packages/pycocotools/cocoeval.py", line 527, in __init__
    self.setDetParams()
  File "/home/lxyuan/playground/table-transformer/env/lib64/python3.9/site-packages/pycocotools/cocoeval.py", line 507, in setDetParams
    self.iouThrs = np.linspace(.5, 0.95, np.round((0.95 - .5) / .05) + 1, endpoint=True)
  File "<__array_function__ internals>", line 180, in linspace
  File "/home/lxyuan/playground/table-transformer/env/lib64/python3.9/site-packages/numpy/core/function_base.py", line 120, in linspace
    num = operator.index(num)
TypeError: 'numpy.float64' object cannot be interpreted as an integer

It seems like i was able to complete one training epoch but got the numpy error message when we were trying to evaluate model performance on the validation set (i.e., src/main:L317)

Similar error when I tried to use main.py to evaluate model performance directly.

How to reproduce the error

(env)$ python main.py 
  --mode eval 
  --data_type detection 
  --config_file detection_config.json
  --data_root_dir ~/../pubtables/PubTables1M-Detection-PASCAL-VOC/ 
  --model_load_path ../pretrained_models/pubtables1m_detection_detr_r18.pth

Error Message

{'lr': 5e-05, 'lr_backbone': 1e-05, 'batch_size': 2, 'weight_decay': 0.0001, 'epochs': 20, 'lr_drop': 1, 'lr_gamma': 0.9, 'clip_max_norm': 0.1, 'backbone': 'resnet18', 'num_classes': 2, 'dilation': False, 'pos
ition_embedding': 'sine', 'emphasized_weights': {}, 'enc_layers': 6, 'dec_layers': 6, 'dim_feedforward': 2048, 'hidden_dim': 256, 'dropout': 0.1, 'nheads': 8, 'num_queries': 15, 'pre_norm': True, 'masks': Fals
e, 'aux_loss': False, 'mask_loss_coef': 1, 'dice_loss_coef': 1, 'ce_loss_coef': 1, 'bbox_loss_coef': 5, 'giou_loss_coef': 2, 'eos_coef': 0.4, 'set_cost_class': 1, 'set_cost_bbox': 5, 'set_cost_giou': 2, 'devic
e': 'cuda', 'seed': 42, 'start_epoch': 0, 'num_workers': 1, 'data_root_dir': '/home/lxyuan/mini-pubtables/PubTables1M-Dectection-PASCAL-VOC/', 'config_file': 'detection_config.json', 'data_type': 'detection',
'model_load_path': '../pretrained_models/pubtables1m_detection_detr_r18.pth', 'load_weights_only': False, 'model_save_dir': None, 'metrics_save_filepath': '', 'debug_save_dir': 'debug', 'table_words_dir': None
, 'mode': 'eval', 'debug': False, 'checkpoint_freq': 1, 'train_max_size': None, 'val_max_size': None, 'test_max_size': None, 'eval_pool_size': 1, 'eval_step': 1, '__module__': '__main__', '__dict__': <attribut
e '__dict__' of 'Args' objects>, '__weakref__': <attribute '__weakref__' of 'Args' objects>, '__doc__': None}
----------------------------------------------------------------------------------------------------
loading model
loading model from checkpoint
loading data
creating index...
index created!
Traceback (most recent call last):
  File "/home/lxyuan/playground/table-transformer/src/main.py", line 375, in <module>
    main()
  File "/home/lxyuan/playground/table-transformer/src/main.py", line 371, in main
    eval_coco(args, model, criterion, postprocessors, data_loader_test, dataset_test, device)
  File "/home/lxyuan/playground/table-transformer/src/eval.py", line 693, in eval_coco
    pubmed_stats, coco_evaluator = evaluate(args, model, criterion, postprocessors,
  File "/home/lxyuan/playground/table-transformer/env/lib64/python3.9/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/home/lxyuan/playground/table-transformer/src/eval.py", line 586, in evaluate
    coco_evaluator = CocoEvaluator(base_ds, iou_types)
  File "/home/lxyuan/playground/table-transformer/src/../detr/datasets/coco_eval.py", line 31, in __init__
    self.coco_eval[iou_type] = COCOeval(coco_gt, iouType=iou_type)
  File "/home/lxyuan/playground/table-transformer/env/lib64/python3.9/site-packages/pycocotools/cocoeval.py", line 76, in __init__
    self.params = Params(iouType=iouType) # parameters
  File "/home/lxyuan/playground/table-transformer/env/lib64/python3.9/site-packages/pycocotools/cocoeval.py", line 527, in __init__
    self.setDetParams()
  File "/home/lxyuan/playground/table-transformer/env/lib64/python3.9/site-packages/pycocotools/cocoeval.py", line 507, in setDetParams
    self.iouThrs = np.linspace(.5, 0.95, np.round((0.95 - .5) / .05) + 1, endpoint=True)
  File "<__array_function__ internals>", line 180, in linspace
  File "/home/lxyuan/playground/table-transformer/env/lib64/python3.9/site-packages/numpy/core/function_base.py", line 120, in linspace
    num = operator.index(num)
TypeError: 'numpy.float64' object cannot be interpreted as an integer

NOTE: I am using numpy==1.23.2 and python3.9

LxYuan-Handshakes commented 2 years ago

I have tried downgraded numpy to 1.17.4 and 1.19.3 but it didn't work.

The error message i got:

(env)$ python main.py --data_type detection --config_file detection_config.json --data_root_dir ~/../pubtables/PubTables1M-Detection-PASCAL-VOC/ 
Traceback (most recent call last):
  File "/home/lxyuan/playground/table-transformer/src/main.py", line 16, in <module>
    from engine import evaluate, train_one_epoch
  File "/home/lxyuan/playground/table-transformer/src/../detr/engine.py", line 13, in <module>
    from datasets.coco_eval import CocoEvaluator
  File "/home/lxyuan/playground/table-transformer/src/../detr/datasets/__init__.py", line 5, in <module>
    from .coco import build as build_coco
  File "/home/lxyuan/playground/table-transformer/src/../detr/datasets/coco.py", line 12, in <module>
    from pycocotools import mask as coco_mask
  File "/home/lxyuan/playground/table-transformer/env/lib64/python3.9/site-packages/pycocotools/mask.py", line 3, in <module>
    import pycocotools._mask as _mask
  File "pycocotools/_mask.pyx", line 1, in init pycocotools._mask
ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 80 from PyObject
LxYuan-Handshakes commented 2 years ago

Solution: