elliptic (not temporal) is missing

[achmand]

running an experiment on elliptic without temporal meaning setting 'data' parameter in yaml file as elliptic doesn't work.

Also the 'elliptic_dl' script is missing which is used in the 'run_exp.py'. Is it possible to upload the python script which handles the non temporal elliptic dataset?

On another note will the preprocessed version of the elliptic dataset be publicly available since Im trying to reproduce results, the following link is not publicly accessible

https://ibm.box.com/s/j04m8lwoqktjixke2gj7lgllrvvdidme

I am trying to produce results found in the following paper 'Anti-Money Laundering in Bitcoin: Experimenting with Graph Convolutional Networks for Financial Forensics'

[zkn0629]

I am having the same issue, please help. Thanks.

[leeQT]

The script for loading AMLSim dataset is also missing, is this repo still maintained?

[mosheber]

I wrote a new script, which does the transformations required for the config in experiments/parameters_elliptic_egcn_h.yaml. Feel free to edit and use it :) elliptic_temporal_data_transform_for_config.ipynb.zip

[mdanb]

@mosheber it works for parameters_elliptic_egcn_o.yaml too right?

[mosheber]

@mdanb it should, since they have the same file configuration and naming.

[mdanb]

@mosheber do you happen to have gotten the following error while running the command python run_exp.py --config_file ./experiments/parameters_elliptic_egcn_h.yaml:

(base) root@01617a075e3a:/evolveGCN# python run_exp.py --config_file ./experiments/parameters_example.yaml
/evolveGCN/utils.py:136: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  data = yaml.load(args.config_file)
use CUDA: True - device: cuda
MPI backend not preset. Set process rank to 0 (out of 1)
Traceback (most recent call last):
  File "run_exp.py", line 218, in <module>
    dataset = build_dataset(args)
  File "run_exp.py", line 116, in build_dataset
    return sbm.sbm_dataset(args)
  File "/evolveGCN/sbm_dl.py", line 16, in __init__
    edges = self.load_edges(args.sbm_args)
  File "/evolveGCN/sbm_dl.py", line 62, in load_edges
    with open(file) as f:
FileNotFoundError: [Errno 2] No such file or directory: './data/sbm_50t_1000n_adj.csv'
(base) root@01617a075e3a:/evolveGCN# python run_exp.py --config_file ./experiments/parameters_elliptic_egcn_h.yaml 
/evolveGCN/utils.py:136: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  data = yaml.load(args.config_file)
use CUDA: True - device: cuda
MPI backend not preset. Set process rank to 0 (out of 1)
Dataset splits sizes:  train 25 dev 5 test 14
CLS num_feats 76
Log file: log/log_elliptic_temporal_node_cls_egcn_h_20200528002454_r0.log
ERROR: Unexpected bus error encountered in worker. This might be caused by insufficient shared memory (shm).
ERROR: Unexpected bus error encountered in worker. This might be caused by insufficient shared memory (shm).
ERROR: Unexpected bus error encountered in worker. This might be caused by insufficient shared memory (shm).
ERROR: Unexpected bus error encountered in worker. This might be caused by insufficient shared memory (shm).
ERROR: Unexpected bus error encountered in worker. This might be caused by insufficient shared memory (shm).
Traceback (most recent call last):
  File "/root/miniconda/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 761, in _try_get_data
    data = self._data_queue.get(timeout=timeout)
  File "/root/miniconda/lib/python3.7/multiprocessing/queues.py", line 104, in get
    if not self._poll(timeout):
  File "/root/miniconda/lib/python3.7/multiprocessing/connection.py", line 257, in poll
    return self._poll(timeout)
  File "/root/miniconda/lib/python3.7/multiprocessing/connection.py", line 414, in _poll
    r = wait([self], timeout)
  File "/root/miniconda/lib/python3.7/multiprocessing/connection.py", line 920, in wait
    ready = selector.select(timeout)
  File "/root/miniconda/lib/python3.7/selectors.py", line 415, in select
    fd_event_list = self._selector.poll(timeout)
  File "/root/miniconda/lib/python3.7/site-packages/torch/utils/data/_utils/signal_handling.py", line 66, in handler
    _error_if_any_worker_fails()
RuntimeError: DataLoader worker (pid 41) is killed by signal: Bus error. It is possible that dataloader's workers are out of shared memory. Please try to raise your shared memory limit.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "run_exp.py", line 238, in <module>
    trainer.train()
  File "/evolveGCN/trainer.py", line 63, in train
    eval_train, nodes_embs = self.run_epoch(self.splitter.train, e, 'TRAIN', grad = True)
  File "/evolveGCN/trainer.py", line 93, in run_epoch
    for s in split:
  File "/root/miniconda/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 345, in __next__
    data = self._next_data()
  File "/root/miniconda/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 841, in _next_data
    idx, data = self._get_data()
  File "/root/miniconda/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 808, in _get_data
    success, data = self._try_get_data()
  File "/root/miniconda/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 774, in _try_get_data
    raise RuntimeError('DataLoader worker (pid(s) {}) exited unexpectedly'.format(pids_str))
RuntimeError: DataLoader worker (pid(s) 41) exited unexpectedly

I can't seem to figure out how to fix this. I looked up the error online, the suggestion is to add --ipc=host or --shm-size when running the command, but that doesn't seem to do it for me.

[mosheber]

@mdanb It depends on your hardware settings, make sure you configure it so that you have enough memory in your GPU. If not, try taking a subsample of the original data, that way it can fit your current build.

[mdanb]

@mosheber hmm now I get a different error. When I run !cd EvolveGCN && python3 run_exp.py --config_file ./experiments/parameters_elliptic_egcn_h.yaml in the Jupyter notebook I get AttributeError: 'splitter' object has no attribute 'train_idx' Did you get this?

EDIT: It seems like there is a bug in the code. In particular, train_idx is set for static datasets in splitter, but not for dynamic datasets. I ended up changing the parameter for saving node embeddings to false, and that did it

[dgx11678832]

@mosheber嗯，现在我得到了另一个错误。当我!cd EvolveGCN && python3 run_exp.py --config_file ./experiments/parameters_elliptic_egcn_h.yaml在Jupyter笔记本中运行时，我明白AttributeError: 'splitter' object has no attribute 'train_idx'了吗？

编辑：似乎代码中有错误。特别是，train_idx是为拆分器中的静态数据集设置的，而没有为动态数据集设置的。我最终将保存节点嵌入的参数更改为false，然后做到了

Have you run through the code? Can you talk about it?

[mosheber]

I haven't used the code in a while, I will try to see what the problem is soon

On Mon, Sep 7, 2020, 10:30 AM dgx11678832 notifications@github.com wrote:

@mosheber https://github.com/mosheber嗯，现在我得到了另一个错误。当我!cd EvolveGCN && python3 run_exp.py --config_file ./experiments/parameters_elliptic_egcn_h.yaml在Jupyter笔记本中运行时，我明白AttributeError: 'splitter' object has no attribute 'train_idx'了吗？

编辑：似乎代码中有错误。特别是，train_idx是为拆分器中的静态数据集设置的，而没有为动态数据集设置的。我最终将保存节点嵌入的参数更改为false，然后做到了

Have you run through the code? Can you talk about it?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/IBM/EvolveGCN/issues/2#issuecomment-688102642, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFJUZUVK4TL5U3HH6BWM653SESDZNANCNFSM4J3O5GYQ .

[dgx11678832]

@mosheber Is your code successful? Could you leave your email?

[dgx11678832]

Traceback (most recent call last): File "/home/dgx/PycharmProjects/EvolveGCN-master/test_args.py", line 237, in trainer.train() File "/home/dgx/PycharmProjects/EvolveGCN-master/trainer.py", line 63, in train eval_train, nodes_embs = self.run_epoch(self.splitter.train, e, 'TRAIN', grad = True) File "/home/dgx/PycharmProjects/EvolveGCN-master/trainer.py", line 104, in run_epoch loss = self.comp_loss(predictions,s.label_sp['vals']) File "/home/dgx/anaconda3/envs/EvolveGCN-master/lib/python3.6/site-packages/torch/nn/modules/module.py", line 722, in _call_impl result = self.forward(*input, **kwargs) File "/home/dgx/PycharmProjects/EvolveGCN-master/Cross_Entropy.py", line 47, in forward alpha = self.weights(labels)[labels].view(-1,1) IndexError: index 2 is out of bounds for dimension 0 with size

Have you encountered this problem? Is there a solution?

[dgx11678832]

Traceback (most recent call last): File "/home/dgx/PycharmProjects/EvolveGCN-master/test_args.py", line 237, in trainer.train() File "/home/dgx/PycharmProjects/EvolveGCN-master/trainer.py", line 63, in train eval_train, nodes_embs = self.run_epoch(self.splitter.train, e, 'TRAIN', grad = True) File "/home/dgx/PycharmProjects/EvolveGCN-master/trainer.py", line 104, in run_epoch loss = self.comp_loss(predictions,s.label_sp['vals']) File "/home/dgx/anaconda3/envs/EvolveGCN-master/lib/python3.6/site-packages/torch/nn/modules/module.py", line 722, in _call_impl result = self.forward(*input, **kwargs) File "/home/dgx/PycharmProjects/EvolveGCN-master/Cross_Entropy.py", line 47, in forward alpha = self.weights(labels)[labels].view(-1,1) IndexError: index 2 is out of bounds for dimension 0 with size

Have you encountered this problem? Is there a solution?

I studied the code and found the problem. Labels is only used as an index, but the value of labels is (1, 2) and the index is (0, 1), so you should put labels-1 in the file Cross_Entropy.py

[dgx11678832]

I found that the difference between my experiment and the paper was a bit big. I don't know what went wrong. This is the validation set of the last epochs that egcn-h ran. Among them, class0 is legal, and class1 is illegal. INFO:root:################ VALID epoch 304 ################### INFO:root:VALID mean losses tensor(1.6735, device='cuda:0') INFO:root:VALID mean errors 0.20141655206680298 INFO:root:VALID mean MRR 0.0 - mean MAP 0.1279858496659947 INFO:root:VALID tp {0: tensor(3541, device='cuda:0'), 1: tensor(67, device='cuda:0')},fn {0: tensor(457, device='cuda:0'), 1: tensor(453, device='cuda:0')},fp {0: tensor(453, device='cuda:0'), 1: tensor(457, device='cuda:0')} INFO:root:VALID measures microavg - precision 0.7986 - recall 0.7986 - f1 0.7986 INFO:root:VALID measures for class 0 - precision 0.8866 - recall 0.8857 - f1 0.8861 INFO:root:VALID measures for class 1 - precision 0.1279 - recall 0.1288 - f1 0.1284 INFO:root:VALID measures@10 microavg - precision 0.4500 - recall 0.0100 - f1 0.0195 INFO:root:VALID measures@10 for class 0 - precision 0.7600 - recall 0.0095 - f1 0.0188 INFO:root:VALID measures@10 for class 1 - precision 0.1400 - recall 0.0135 - f1 0.0246 INFO:root:VALID measures@100 microavg - precision 0.5356 - recall 0.1067 - f1 0.1779 INFO:root:VALID measures@100 for class 0 - precision 0.8600 - recall 0.1076 - f1 0.1912 INFO:root:VALID measures@100 for class 1 - precision 0.1300 - recall 0.1000 - f1 0.1130 INFO:root:VALID measures@1000 microavg - precision 0.7957 - recall 0.7742 - f1 0.7848 INFO:root:VALID measures@1000 for class 0 - precision 0.8861 - recall 0.8582 - f1 0.8719 INFO:root:VALID measures@1000 for class 1 - precision 0.1279 - recall 0.1288 - f1 0.1284 INFO:root:VALID Total epoch time: 3.797350121982163

[lionheartStark]

@dgx11678832 do you know why？

[vatsalpatels]

Hi,

I'm trying to access the processed elliptic data set but I'm unable to find it. Can you please help?

https://ibm.box.com/s/j04m8lwoqktjixke2gj7lgllrvvdidme

[dgx11678832]

@lionheartStark Which question ？