Output directory fix for preprocessor

marius-team / marius

Large scale graph learning on a single machine.

https://marius-project.org

Apache License 2.0

160 stars 45 forks source link

Output directory fix for preprocessor #62

Closed JasonMoho closed 3 years ago

JasonMoho commented 3 years ago

Minor bug with the preprocessor for custom datasets. The output directory was interpreted as a list when it should have been a string.

shivaram commented 3 years ago

Any ideas how our tests didn't catch this? Do we need a new test

JasonMoho commented 3 years ago

Yeah I think the test case for this is insufficient. https://github.com/marius-team/marius/blob/main/test/python/preprocessing/test_preprocess_cmd_opt_parsing.py#L166

This will only check if a configuration file was generated. Doesn't check the other output files. @AnzeXie can you create an issue and get working on a fix?

AnzeXie commented 3 years ago

The features supporting the custom dataset preprocessing was migrated from csv_converter.py . The argparse of csv_converter.py was a little different from that in preprocess.py. Yes, I'll add additional tests for the argparse in custom dataset cases.