Open lwwlwwl opened 2 years ago
Hi,
I am trying to preprocess ogb_mag240m with marius_preprocess --dataset ogb_mag240m --output_dir datasets/ogb_mag240m/ while it was killed due to oom.
marius_preprocess --dataset ogb_mag240m --output_dir datasets/ogb_mag240m/
The dataset.yaml was half-way generated:
dataset_dir: /marius/datasets/ogb_mag240m/ num_edges: 1297748926 num_nodes: 121751666 num_relations: 1 num_train: 1297748926 num_valid: -1 num_test: -1 node_feature_dim: -1 rel_feature_dim: -1 num_classes: -1 initialized: false
The cpu mem is as high as I am able to get (312GB). I am wondering if there is any way around if I want to run ogb_mag240m on this machine. Thank you.
Hi,
I am trying to preprocess ogb_mag240m with
marius_preprocess --dataset ogb_mag240m --output_dir datasets/ogb_mag240m/while it was killed due to oom.The dataset.yaml was half-way generated:
The cpu mem is as high as I am able to get (312GB). I am wondering if there is any way around if I want to run ogb_mag240m on this machine. Thank you.