search

marius-team / marius

Large scale graph learning on a single machine.
https://marius-project.org
Apache License 2.0
160 stars 45 forks source link

Add functions to store and use custom dataset statistics #77

Closed AnzeXie closed 2 years ago

AnzeXie commented 3 years ago

Describe the pull request. This PR enables the preprocess to store the statistics of the preprocessed dataset into a JSON file. This PR also enables the config_generator to read the JSON file stored by the preprocess and put the statistics into the configuration file. Users can invoke this function by passing the option --dataset_stats_path with the path to the JSON file.

How was this tested? This is tested by adding the test test_save_load_custom_dataset_stats (test_config_generator_cmd_opt_parsing.py line 183-198).