search

microsoft / Graphormer

Graphormer is a general-purpose deep learning backbone for molecular modeling.
MIT License
2.08k stars 334 forks source link

OverflowError: can't convert negative value to unsigned int #123

Closed JiaYuanChng closed 2 years ago

JiaYuanChng commented 2 years ago

Hi developers,

I am trying to train a regression model using my own dataset. I have a bash script (in fact, exactly the same command lines but different numbers for epochs and number of updates) almost similar to the ZINC example, but I have not been able to resolve the following error: OverflowError: can't convert negative value to unsigned int

Could you please help shed some light on how do I go about troubleshooting this, or is there something I might have overlooked?

ps. in the examples provided, only the oc20.sh runs. I get the above error with zinc.sh too.

Thank you.

Austinzhenghua commented 2 years ago

Hi, I have the same problem, but I think this problem is related to these two lines:

pip install . --use-feature=in-tree-build
python setup.py build_ext --inplace
JiaYuanChng commented 2 years ago

In case anyone who could help requires a bit more info:

2022-06-21 01:01:51 | INFO | fairseq_cli.train | task: GraphPredictionTask
2022-06-21 01:01:51 | INFO | fairseq_cli.train | model: GraphormerModel
2022-06-21 01:01:51 | INFO | fairseq_cli.train | criterion: GraphPredictionL1Loss
2022-06-21 01:01:51 | INFO | fairseq_cli.train | num. shared model params: 959,073 (num. trained: 959,073)
2022-06-21 01:01:51 | INFO | fairseq_cli.train | num. expert model params: 0 (num. trained: 0)
2022-06-21 01:01:51 | INFO | graphormer.tasks.graph_prediction | Loaded valid with #samples: 627
2022-06-21 01:01:51 | INFO | fairseq_cli.train | training on 1 devices (GPUs/TPUs)
2022-06-21 01:01:51 | INFO | fairseq_cli.train | max tokens per device = None and max sentences per device = 32
2022-06-21 01:01:51 | INFO | fairseq.trainer | Preparing to load checkpoint ./ckpts/checkpoint_last.pt
2022-06-21 01:01:51 | INFO | fairseq.trainer | No existing checkpoint found ./ckpts/checkpoint_last.pt
2022-06-21 01:01:51 | INFO | fairseq.trainer | loading train data for epoch 1
2022-06-21 01:01:51 | INFO | graphormer.tasks.graph_prediction | Loaded train with #samples: 5641
Traceback (most recent call last):
  File "/share/apps/miniconda3/envs/graphormer/bin/fairseq-train", line 8, in <module>
    sys.exit(cli_main())
  File "/share/apps/miniconda3/envs/graphormer/lib/python3.9/site-packages/fairseq_cli/train.py", line 528, in cli_main
    distributed_utils.call_main(cfg, main)
  File "/share/apps/miniconda3/envs/graphormer/lib/python3.9/site-packages/fairseq/distributed/utils.py", line 369, in call_main
    main(cfg, **kwargs)
  File "/share/apps/miniconda3/envs/graphormer/lib/python3.9/site-packages/fairseq_cli/train.py", line 162, in main
    extra_state, epoch_itr = checkpoint_utils.load_checkpoint(
  File "/share/apps/miniconda3/envs/graphormer/lib/python3.9/site-packages/fairseq/checkpoint_utils.py", line 270, in load_checkpoint
    epoch_itr = trainer.get_train_iterator(
  File "/share/apps/miniconda3/envs/graphormer/lib/python3.9/site-packages/fairseq/trainer.py", line 655, in get_train_iterator
    self.reset_dummy_batch(batch_iterator.first_batch)
  File "/share/apps/miniconda3/envs/graphormer/lib/python3.9/site-packages/fairseq/data/iterators.py", line 334, in first_batch
    return self.collate_fn([self.dataset[i] for i in self.frozen_batches[0]])
  File "/share/apps/miniconda3/envs/graphormer/lib/python3.9/site-packages/fairseq/data/iterators.py", line 334, in <listcomp>
    return self.collate_fn([self.dataset[i] for i in self.frozen_batches[0]])
  File "/share/apps/miniconda3/envs/graphormer/lib/python3.9/site-packages/fairseq/data/nested_dictionary_dataset.py", line 70, in __getitem__
    return OrderedDict((k, ds[index]) for k, ds in self.defn.items())
  File "/share/apps/miniconda3/envs/graphormer/lib/python3.9/site-packages/fairseq/data/nested_dictionary_dataset.py", line 70, in <genexpr>
    return OrderedDict((k, ds[index]) for k, ds in self.defn.items())
  File "/share/apps/miniconda3/envs/graphormer/Graphormer/graphormer/data/dataset.py", line 31, in __getitem__
    item = self.dataset[int(index)]
  File "/share/apps/miniconda3/envs/graphormer/Graphormer/graphormer/data/dgl_datasets/dgl_dataset.py", line 156, in __getitem__
    return self.__preprocess_dgl_graph(graph, y, idx)
  File "/share/apps/miniconda3/envs/graphormer/Graphormer/graphormer/data/dgl_datasets/dgl_dataset.py", line 131, in __preprocess_dgl_graph
    edge_input = algos.gen_edge_input(max_dist, path, attn_edge_type.numpy())
  File "algos.pyx", line 86, in graphormer.data.algos.gen_edge_input
  File "algos.pyx", line 58, in graphormer.data.algos.get_all_edges
OverflowError: can't convert negative value to unsigned int
JiaYuanChng commented 2 years ago

It seems to have been fixed recently #121