Multi-process running errors

[JennyyiqiW]

Hi! There are some errors when I try to run the code in the multi-process mode.(i.e. without --debug)

The errors are shown below:

[Errno 2] No such file or directory: 'test-results/GraphSAGE_ENZYMES_assessment/10_NESTED_CV/OUTER_FOLD_1/HOLDOUT_MS/config_1/config_results.json' [Errno 2] No such file or directory: 'test-results/GraphSAGE_ENZYMES_assessment/10_NESTED_CV/OUTER_FOLD_1/HOLDOUT_MS/config_2/config_results.json' [Errno 2] No such file or directory: 'test-results/GraphSAGE_ENZYMES_assessment/10_NESTED_CV/OUTER_FOLD_2/HOLDOUT_MS/config_1/config_results.json' [Errno 2] No such file or directory: 'test-results/GraphSAGE_ENZYMES_assessment/10_NESTED_CV/OUTER_FOLD_2/HOLDOUT_MS/config_2/config_results.json' [Errno 2] No such file or directory: 'test-results/GraphSAGE_ENZYMES_assessment/10_NESTED_CV/OUTER_FOLD_3/HOLDOUT_MS/config_1/config_results.json' [Errno 2] No such file or directory: 'test-results/GraphSAGE_ENZYMES_assessment/10_NESTED_CV/OUTER_FOLD_3/HOLDOUT_MS/config_2/config_results.json' [Errno 2] No such file or directory: 'test-results/GraphSAGE_ENZYMES_assessment/10_NESTED_CV/OUTER_FOLD_4/HOLDOUT_MS/config_1/config_results.json' [Errno 2] No such file or directory: 'test-results/GraphSAGE_ENZYMES_assessment/10_NESTED_CV/OUTER_FOLD_4/HOLDOUT_MS/config_2/config_results.json' [Errno 2] No such file or directory: 'test-results/GraphSAGE_ENZYMES_assessment/10_NESTED_CV/OUTER_FOLD_5/HOLDOUT_MS/config_1/config_results.json' [Errno 2] No such file or directory: 'test-results/GraphSAGE_ENZYMES_assessment/10_NESTED_CV/OUTER_FOLD_5/HOLDOUT_MS/config_2/config_results.json' [Errno 2] No such file or directory: 'test-results/GraphSAGE_ENZYMES_assessment/10_NESTED_CV/OUTER_FOLD_6/HOLDOUT_MS/config_1/config_results.json' [Errno 2] No such file or directory: 'test-results/GraphSAGE_ENZYMES_assessment/10_NESTED_CV/OUTER_FOLD_6/HOLDOUT_MS/config_2/config_results.json' [Errno 2] No such file or directory: 'test-results/GraphSAGE_ENZYMES_assessment/10_NESTED_CV/OUTER_FOLD_7/HOLDOUT_MS/config_1/config_results.json' [Errno 2] No such file or directory: 'test-results/GraphSAGE_ENZYMES_assessment/10_NESTED_CV/OUTER_FOLD_7/HOLDOUT_MS/config_2/config_results.json' [Errno 2] No such file or directory: 'test-results/GraphSAGE_ENZYMES_assessment/10_NESTED_CV/OUTER_FOLD_8/HOLDOUT_MS/config_1/config_results.json' [Errno 2] No such file or directory: 'test-results/GraphSAGE_ENZYMES_assessment/10_NESTED_CV/OUTER_FOLD_8/HOLDOUT_MS/config_2/config_results.json' [Errno 2] No such file or directory: 'test-results/GraphSAGE_ENZYMES_assessment/10_NESTED_CV/OUTER_FOLD_9/HOLDOUT_MS/config_1/config_results.json' [Errno 2] No such file or directory: 'test-results/GraphSAGE_ENZYMES_assessment/10_NESTED_CV/OUTER_FOLD_9/HOLDOUT_MS/config_2/config_results.json' [Errno 2] No such file or directory: 'test-results/GraphSAGE_ENZYMES_assessment/10_NESTED_CV/OUTER_FOLD_10/HOLDOUT_MS/config_1/config_results.json' [Errno 2] No such file or directory: 'test-results/GraphSAGE_ENZYMES_assessment/10_NESTED_CV/OUTER_FOLD_10/HOLDOUT_MS/config_2/config_results.json' [Errno 2] No such file or directory: 'test-results/GraphSAGE_ENZYMES_assessment/10_NESTED_CV/OUTER_FOLD_1/outer_results.json' [Errno 2] No such file or directory: 'test-results/GraphSAGE_ENZYMES_assessment/10_NESTED_CV/OUTER_FOLD_2/outer_results.json' [Errno 2] No such file or directory: 'test-results/GraphSAGE_ENZYMES_assessment/10_NESTED_CV/OUTER_FOLD_3/outer_results.json' [Errno 2] No such file or directory: 'test-results/GraphSAGE_ENZYMES_assessment/10_NESTED_CV/OUTER_FOLD_4/outer_results.json' [Errno 2] No such file or directory: 'test-results/GraphSAGE_ENZYMES_assessment/10_NESTED_CV/OUTER_FOLD_5/outer_results.json' [Errno 2] No such file or directory: 'test-results/GraphSAGE_ENZYMES_assessment/10_NESTED_CV/OUTER_FOLD_6/outer_results.json' [Errno 2] No such file or directory: 'test-results/GraphSAGE_ENZYMES_assessment/10_NESTED_CV/OUTER_FOLD_7/outer_results.json' [Errno 2] No such file or directory: 'test-results/GraphSAGE_ENZYMES_assessment/10_NESTED_CV/OUTER_FOLD_8/outer_results.json' [Errno 2] No such file or directory: 'test-results/GraphSAGE_ENZYMES_assessment/10_NESTED_CV/OUTER_FOLD_9/outer_results.json' [Errno 2] No such file or directory: 'test-results/GraphSAGE_ENZYMES_assessment/10_NESTED_CV/OUTER_FOLD_10/outer_results.json' /mnt/ufs18/home-118/wangy206/torch-projects/self-surpervised-GNN/gnn-comparison/evaluation/risk_assessment/K_Fold_Assessment.py:53: RuntimeWarning: Mean of empty slice. assessment_results['avg_TR_score'] = outer_TR_scores.mean() /mnt/home/wangy206/anaconda3/envs/gnn-comparison/lib/python3.7/site-packages/numpy/core/_methods.py:161: RuntimeWarning: invalid value encountered in double_scalars ret = ret.dtype.type(ret / rcount) /mnt/home/wangy206/anaconda3/envs/gnn-comparison/lib/python3.7/site-packages/numpy/core/_methods.py:217: RuntimeWarning: Degrees of freedom <= 0 for slice keepdims=keepdims) /mnt/home/wangy206/anaconda3/envs/gnn-comparison/lib/python3.7/site-packages/numpy/core/_methods.py:186: RuntimeWarning: invalid value encountered in true_divide arrmean, rcount, out=arrmean, casting='unsafe', subok=False) /mnt/home/wangy206/anaconda3/envs/gnn-comparison/lib/python3.7/site-packages/numpy/core/_methods.py:209: RuntimeWarning: invalid value encountered in double_scalars ret = ret.dtype.type(ret / rcount) /mnt/ufs18/home-118/wangy206/torch-projects/self-surpervised-GNN/gnn-comparison/evaluation/risk_assessment/K_Fold_Assessment.py:55: RuntimeWarning: Mean of empty slice. assessment_results['avg_TS_score'] = outer_TS_scores.mean()

It seems that the errors come from HoldOutSeletor.process_results(), where the son processes are killed before they finish writing "config_results.json" or that "best_config = self.process_results(HOLDOUT_MS_FOLDER, config_id)" begins to run before the processes in the pool finish.

[diningphil]

Hi!

Do you get any errors with the debug option? If everything is OK with debug, then it should also run in parallel.

[JennyyiqiW]

Hi!

I haven got any errors with the debug option. Thanks.

On Fri, Jun 19, 2020 at 2:03 AM Federico Errica notifications@github.com wrote:

Hi!

Do you get any errors with the debug option?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/diningphil/gnn-comparison/issues/8#issuecomment-646452665, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANOVUFCZTPPQFM4Y3KEW4XLRXL5TBANCNFSM4OCG4WTQ .

[diningphil]

Strange, this has never occured to me (nor to anyone else it seems). It is quite difficult to tell where the problem could be with such little information.

Can you paste the command you used to launch the experiments?

[JennyyiqiW]

Hi Federico, it can run in parallel now. It turns out that I used to run the parallel code in Cuda so it failed. When I use CPU, it can run in parallel.

Great codes! Thanks!