kstaats / karoo_gp

A Genetic Programming platform for Python with TensorFlow for wicked-fast CPU and GPU support.
Other
159 stars 61 forks source link

Add a method that creates a JSON report. #65

Closed ezio-melotti closed 2 years ago

ezio-melotti commented 2 years ago

This PR creates a JSON report that includes both the initial configuration and results as discussed in #62.

Implementation-wise, I copied the fx_data_params_write, created a fx_eval_fittest function that contains some duplicated logic, and created a new fx_data_params_write_json that creates the report.

Here are some sample JSONs for the classification and regression kernels:

Classification kernel ```json { "package": "Karoo GP", "launched": "2022-05-31_08-05-57-998149", "dataset": "karoo_gp/files/data_CLASSIFY.csv", "kernel": "c", "precision": 6, "tree_type": "g", "tree_depth_base": 3, "tree_depth_max": 4, "min_node_count": 3, "genetic_operators": { "reproduction": 0.1, "point_mutation": 0.1, "branch_mutation": 0.2, "crossover": 0.6 }, "tournament_size": 7, "population": 10, "number_of_generations": 10, "outcome": "SUCCESS", "fittest_tree_id": 10, "expression": "pw*sl*sw", "fitness_score": 6.0, "classification_report": { "0.0": { "precision": 0.0, "recall": 0.0, "f1-score": 0.0, "support": 11 }, "1.0": { "precision": 0.0, "recall": 0.0, "f1-score": 0.0, "support": 13 }, "2.0": { "precision": 0.2, "recall": 1.0, "f1-score": 0.33333333333333337, "support": 6 }, "accuracy": 0.2, "macro avg": { "precision": 0.06666666666666667, "recall": 0.3333333333333333, "f1-score": 0.11111111111111112, "support": 30 }, "weighted avg": { "precision": 0.04000000000000001, "recall": 0.2, "f1-score": 0.06666666666666667, "support": 30 } }, "confusion_matrix": [ [ 0, 0, 11 ], [ 0, 0, 13 ], [ 0, 0, 6 ] ] } ```
Regression kernel ```json { "package": "Karoo GP", "launched": "2022-05-31_08-09-14-382026", "dataset": "karoo_gp/files/data_REGRESS.csv", "kernel": "r", "precision": 6, "tree_type": "g", "tree_depth_base": 3, "tree_depth_max": 4, "min_node_count": 3, "genetic_operators": { "reproduction": 0.1, "point_mutation": 0.1, "branch_mutation": 0.2, "crossover": 0.6 }, "tournament_size": 7, "population": 10, "number_of_generations": 10, "outcome": "SUCCESS", "fittest_tree_id": 9, "expression": "1", "fitness_score": 0.04999995231628418, "mean_squared_error": 7.777762948535383e-05 } ```

We could/should bikeshed a bit on names and format, in particular:

ezio-melotti commented 2 years ago

After discussing with Grant, I made the following changes:

Here are some example outputs:

Regression kernel ```json { "package": "Karoo GP", "launched": "2022-06-12_05-32-27-376325", "dataset": "karoo_gp/files/data_CLASSIFY.csv", "config": { "kernel": "r", "precision": 6, "tree_type": "g", "tree_depth_base": 3, "tree_depth_max": 4, "min_node_count": 3, "genetic_operators": { "reproduction": 0.1, "point_mutation": 0.1, "branch_mutation": 0.2, "crossover": 0.6 }, "tournament_size": 7, "population": 100, "number_of_generations": 10 }, "outcome": "SUCCESS", "fittest_tree": { "id": 100, "expression": "pl*pw/sl" }, "score": { "fitness": 5.721993446350098, "mean_squared_error": 0.0737130418419838 } } ```
Classification kernel ```json { "package": "Karoo GP", "launched": "2022-06-12_05-46-26-552193", "dataset": "karoo_gp/files/data_CLASSIFY.csv", "config": { "kernel": "c", "precision": 6, "tree_type": "g", "tree_depth_base": 3, "tree_depth_max": 4, "min_node_count": 3, "genetic_operators": { "reproduction": 0.1, "point_mutation": 0.1, "branch_mutation": 0.2, "crossover": 0.6 }, "tournament_size": 7, "population": 100, "number_of_generations": 10 }, "outcome": "SUCCESS", "fittest_tree": { "id": 96, "expression": "pw - pw/sw - 1/pw" }, "score": { "fitness": 29.0, "classification_report": { "0.0": { "precision": 1.0, "recall": 1.0, "f1-score": 1.0, "support": 9 }, "1.0": { "precision": 1.0, "recall": 0.9166666666666666, "f1-score": 0.9565217391304348, "support": 12 }, "2.0": { "precision": 0.9, "recall": 1.0, "f1-score": 0.9473684210526316, "support": 9 }, "accuracy": 0.9666666666666667, "macro avg": { "precision": 0.9666666666666667, "recall": 0.9722222222222222, "f1-score": 0.9679633867276888, "support": 30 }, "weighted avg": { "precision": 0.9700000000000001, "recall": 0.9666666666666667, "f1-score": 0.9668192219679634, "support": 30 } }, "confusion_matrix": [ [9, 0, 0], [0, 11, 1], [0, 0, 9] ] } } ```