Closed jeff-regier closed 6 years ago
Scores following this PR, for a synthetic catalog and synthetic imagery:
Predictions:
│ Row │ N │ first │ field │ ├─────┼─────┼───────────┼─────────────────┤ │ 1 │ 131 │ 0.0610687 │ missed_stars │ │ 2 │ 347 │ 0.037464 │ missed_galaxies │ │ 3 │ 478 │ 0.144662 │ position │ │ 4 │ 478 │ 0.143964 │ flux_r_mag │ │ 5 │ 478 │ 0.309982 │ flux_r_nmgy │ │ 6 │ 50 │ 8.31035 │ gal_angle_deg │ │ 7 │ 94 │ 0.141638 │ gal_frac_dev │ │ 8 │ 94 │ 0.140924 │ gal_axis_ratio │ │ 9 │ 94 │ 0.624384 │ gal_radius_px │ │ 10 │ 478 │ 0.406908 │ color_ug │ │ 11 │ 478 │ 0.156452 │ color_gr │ │ 12 │ 478 │ 0.124973 │ color_ri │ │ 13 │ 478 │ 0.206888 │ color_iz │
Uncertainty quantification:
│ Row │ field │ within_half_sd │ within_1_sd │ within_2_sd │ within_3_sd │ ├─────┼─────────────────┼────────────────┼─────────────┼─────────────┼─────────────┤ │ 1 │ color_gr │ 0.267223 │ 0.438413 │ 0.7119 │ 0.845511 │ │ 2 │ color_iz │ 0.292276 │ 0.542797 │ 0.822547 │ 0.924843 │ │ 3 │ color_ri │ 0.223382 │ 0.43215 │ 0.693111 │ 0.830898 │ │ 4 │ color_ug │ 0.240084 │ 0.498956 │ 0.793319 │ 0.889353 │ │ 5 │ log_flux_r_nmgy │ 0.183716 │ 0.329854 │ 0.51357 │ 0.686848 │
@andymiller Here are the commands I used to generate these tables, all executed in the benchmark/accuracy directory:
benchmark/accuracy
$ ./write_ground_truth_catalog_csv.jl prior $ ./generate_synthetic_field.jl 4263 5 119 output/prior_<hash1>.csv $ export JULIA_NUM_THREADS=4 $ ./run_celeste_on_field.jl --images-jld output/prior_<hash1>_synthetic_<hash2>.jld output/prior_<hash1>.csv --use-full-initialization $ ./score_predictions.jl output/prior_<hash1>.csv --use-full-initialization output/prior_<hash1>_synthetic_<hash2>_predictions_<hash3>.csv $ ./score_uncertainty.jl output/prior_<hash1>.csv output/prior_<hash1>_synthetic_<hash2>_predictions_<hash3>.csv
Scores following this PR, for a synthetic catalog and synthetic imagery:
Predictions:
Uncertainty quantification:
@andymiller Here are the commands I used to generate these tables, all executed in the
benchmark/accuracy
directory: