Validation is slow for large graphs

ldbc / ldbc_graphalytics

Generic driver for LDBC Graphalytics implementation

https://ldbcouncil.org/benchmarks/graphalytics/

Apache License 2.0

81 stars 34 forks source link

Validation is slow for large graphs #205

Closed szarnyasg closed 1 year ago

szarnyasg commented 3 years ago

Make validation code multi-threaded - or reimplement it in C++/Julia: https://github.com/ldbc/ldbc_graphalytics/tree/master/graphalytics-core/src/main/java/science/atlarge/graphalytics/validation

Update (August 2022): Using DuckDB through JDBC could be an alternative solution. This gives us a fast relational engine without having to move the entire framework to another language.

szarnyasg commented 2 years ago

Example SQL scripts for cross-validation: https://github.com/ldbc/ldbc_graphalytics/tree/output-validation-using-matching-in-sql/matching