lsst-sims / legacy_sims_maf

LSST Simulations package for the metrics analysis framework (MAF)
13 stars 19 forks source link

adding figure of merit metric #174

Closed hsnee closed 4 years ago

hsnee commented 4 years ago

Adding the static-probe statistical figure of merit emulator metric, to easily replicate what was used in the DESC observing strategy whitepaper on new sims. This can be run as a summary metric. It requires a version of ExgalM5 that has ebv and depth cuts so adding that as well. Thanks to @yoachim for help with writing this.

Example code to run this metric (suppressing imports to declutter)

outDir = 'temp'
resultsDb = db.ResultsDb(outDir=outDir)
dbFile = '/global/cscratch1/sd/awan/dbs_post_wp_v2/add_mag_clouds_v1.3_10yrs.db'
conn = db.OpsimDatabase(dbFile)
sql = 'filter = "i" and note not like "DD%"'
metric = ExgalM5_cut(depth_cut=26)
slicer = slicers.HealpixSlicer(nside=nside, useCache=False)
summary = [FoMEmulatorMetric()]
bundleList = [metricBundles.MetricBundle(metric, slicer, sql, summaryMetrics=summary)]
bd = metricBundles.makeBundlesDictFromList(bundleList)
bg = metricBundles.MetricBundleGroup(bd, conn, outDir=outDir, resultsDb=resultsDb)
bg.runAll()

# result: 
(bg.bundleDict['opsim_ExgalM5_cut_fiveSigmaDepth_i_and_note_not_like_DD%_HEAL']).summaryValues
hsnee commented 4 years ago

Thanks for the comments, I've expanded more on the documentation for both metrics. They're actually quite different, the FoM measures the statistical power of the combined static probes, while the WeakLensingNvisits is proxy for WL systematics. This has been documented in the docstring now. Let me know if this makes sense/seems enough to avoid confusion in the future. I've also changed the name of the FoM metric.

hsnee commented 4 years ago

Due to #194 the sql constraint should no longer specify the filter.