search

astrochun / Metallicity_Stack_Commons

Set of common codes used in metallicity studies that use stacking techniques
MIT License
0 stars 1 forks source link

feature/dust_implement #72

Closed astrochun closed 4 years ago

astrochun commented 4 years ago

See iss #67

A couple of issues:

  1. [x] There seems to be an issue with getting the same results with MC runs. Some of the flux ratios differ. [79790f2]
  2. [x] I noticed that the EBV_HdHb was much different between raw=True and raw=False [f986726]
  3. [x] Need to change filename_dict convention to distinguish when revised=False or True

Addressing these issues below in other comments.

The following tests are needed to confirm the dust attenuation is working with error_prop and ratios:

Zcalbase_gal

No Randomization (raw=True)

    • [x] Run error_prop.fluxes_derived_prop with raw=True, binned_data=True, apply_dust=False, revised=False
      • [x] Output file called bin_derived_properties.valid1.tbl
      • [x] Contains flux ratios: two_beta_composite, three_beta_composite, R_composite
      • [x] Contains flux ratios: logR23_composite, logO32_composite
      • [x] Contains derived properties: T_e, 12+log(O/H), log(O+/H), log(O++/H), O+/H, O++/H
      • [x] Include HgHb_composite, HdHb_composite (this is needed later since bin_derived_properties.tbl initial values are needed)
      • [x] Pass Chun's local test
      • [x] Pass Reagen's local test
    • [x] Run error_prop.fluxes_derived_prop with raw=True, binned_data=True, apply_dust=True, revised=False
      • [x] Output file called bin_derived_properties.dustcorr.valid1.tbl
      • [x] Contains flux ratios: two_beta_composite, three_beta_composite, R_composite
      • [x] Contains flux ratios: logR23_composite, logO32_composite
      • [x] Contains derived properties: T_e, 12+log(O/H), log(O+/H), log(O++/H), O+/H, O++/H
      • [x] Contains dust info: HgHb, HdHb, EBV_HgHb, and EBV_HdHb
      • [x] Pass Chun's local test
      • [x] Pass Reagen's local test
    • [x] Run error_prop.fluxes_derived_prop with raw=True, binned_data=True, apply_dust=True, revised=True
      • [x] Output file called bin_derived_properties.tbl
      • [x] Pass Chun's local test

Monte Carlo Randomization (raw=False)

    • [x] Run error_prop.fluxes_derived_prop with raw=False, binned_data=True, apply_dust=False, revised=False
      • [x] Output file called bin_derived_properties.valid1.MC.tbl
      • [x] Contains flux ratios: two_beta_composite, three_beta_composite, R_composite
      • [x] Contains flux ratios: logR23_composite, logO32_composite
      • [x] Contains errors for flux ratios: two_beta_composite, three_beta_composite, R_composite
      • [x] Contains errors for flux ratios: logR23_composite, logO32_composite
      • [x] Contains derived properties: T_e, 12+log(O/H), log(O+/H), log(O++/H), O+/H, O++/H
      • [x] Contains errors for derived properties: T_e, 12+log(O/H), log(O+/H), log(O++/H), O+/H, O++/H
      • [x] Include HgHb_composite, HdHb_composite
      • [x] Include errors for HgHb_composite, HdHb_composite
      • [x] Pass Chun's local test
      • [x] Pass Reagen's local test
    • [x] Run error_prop.fluxes_derived_prop with raw=False, binned_data=True, apply_dust=False, revised=True
      • [x] Output file called bin_derived_properties.MC.tbl
      • [x] Contains flux ratios: two_beta_composite, three_beta_composite, R_composite
      • [x] Contains flux ratios: logR23_composite, logO32_composite
      • [x] Contains errors for flux ratios: two_beta_composite, three_beta_composite, R_composite
      • [x] Contains errors for flux ratios: logR23_composite, logO32_composite
      • [x] Contains derived properties: T_e, 12+log(O/H), log(O+/H), log(O++/H), O+/H, O++/H
      • [x] Contains errors for derived properties: T_e, 12+log(O/H), log(O+/H), log(O++/H), O+/H, O++/H
      • [x] Include HgHb_composite, HdHb_composite
      • [x] Include errors for HgHb_composite, HdHb_composite
      • [x] Pass Chun's local test
      • [x] Pass Reagen's local test
    • [x] Run error_prop.fluxes_derived_prop with raw=False, binned_data=True, apply_dust=True, revised=True
      • [x] Output file called 'bin_derived_properties.MC.dustcorr.tbl'
      • [x] Contains flux ratios: two_beta_composite, three_beta_composite, R_composite
      • [x] Contains flux ratios: logR23_composite, logO32_composite
      • [x] Contains errors for flux ratios: two_beta_composite, three_beta_composite, R_composite
      • [x] Contains errors for flux ratios: logR23_composite, logO32_composite
      • [x] Contains derived properties: T_e, 12+log(O/H), log(O+/H), log(O++/H), O+/H, O++/H
      • [x] Contains errors for derived properties: T_e, 12+log(O/H), log(O+/H), log(O++/H), O+/H, O++/H
      • [x] Contains dust info: HgHb_composite, HdHb_composite, EBV_HgHb, and EBV_HdHb
      • [x] Contains errors for dust info: HgHb_composite, HdHb_composite, EBV_HgHb, and EBV_HdHb
      • [x] Pass Chun's local test
      • [x] Pass Reagen's local test

Evolution_of_Galaxies

No Randomization (raw=True)

    • [x] Run error_prop.fluxes_derived_prop with raw=True, binned_data=True, apply_dust=False, revised=False
      • [x] Output file called bin_derived_properties.valid1.tbl
      • [x] Contains flux ratios: two_beta_composite, three_beta_composite, R_composite
      • [x] Contains flux ratios: logR23_composite, logO32_composite
      • [x] Contains derived properties: T_e, 12+log(O/H), log(O+/H), log(O++/H), O+/H, O++/H
      • [x] Include HgHb_composite, HdHb_composite (this is needed later since bin_derived_properties.tbl initial values are needed)
      • [x] Pass Chun's local test
      • [x] Pass Caroline's local test
        • [x] Mass bins
        • [x] Mass-LHbeta bins
    • [x] Run error_prop.fluxes_derived_prop with raw=True, binned_data=True, apply_dust=True, revised=False
      • [x] Output file called bin_derived_properties.dustcorr.valid1.tbl
      • [x] Contains flux ratios: two_beta_composite, three_beta_composite, R_composite
      • [x] Contains flux ratios: logR23_composite, logO32_composite
      • [x] Contains derived properties: T_e, 12+log(O/H), log(O+/H), log(O++/H), O+/H, O++/H
      • [x] Contains dust info: HgHb_composite, HdHb_composite, EBV_HgHb, and EBV_HdHb
      • [x] Pass Chun's local test
      • [x] Pass Caroline's local test
        • [x] Mass bins
        • [x] Mass-LHbeta bins

Monte Carlo Randomization (raw=False)

    • [x] Run error_prop.fluxes_derived_prop with raw=False, binned_data=True, apply_dust=False, revised=False
      • [x] Output file called bin_derived_properties.valid1.MC.tbl
      • [x] Contains flux ratios: two_beta_composite, three_beta_composite, R_composite
      • [x] Contains flux ratios: logR23_composite, logO32_composite
      • [x] Contains errors for flux ratios: two_beta_composite, three_beta_composite, R_composite
      • [x] Contains errors for flux ratios: logR23_composite, logO32_composite
      • [x] Contains derived properties: T_e, 12+log(O/H), log(O+/H), log(O++/H), O+/H, O++/H
      • [x] Contains errors for derived properties: T_e, 12+log(O/H), log(O+/H), log(O++/H), O+/H, O++/H
      • [x] Include HgHb_composite, HdHb_composite
      • [x] Include errors for HgHb_composite, HdHb_composite
      • [x] Pass Chun's local test
      • [x] Pass Caroline's local test
        • [x] Mass bins
        • [x] Mass-LHbeta bins
    • [x] Run error_prop.fluxes_derived_prop with raw=False, binned_data=True, apply_dust=False, revised=True
      • [x] Output file called bin_derived_properties.MC.tbl
      • [x] Contains flux ratios: two_beta_composite, three_beta_composite, R_composite
      • [x] Contains flux ratios: logR23_composite, logO32_composite
      • [x] Contains errors for flux ratios: two_beta_composite, three_beta_composite, R_composite
      • [x] Contains errors for flux ratios: logR23_composite, logO32_composite
      • [x] Contains derived properties: T_e, 12+log(O/H), log(O+/H), log(O++/H), O+/H, O++/H
      • [x] Contains errors for derived properties: T_e, 12+log(O/H), log(O+/H), log(O++/H), O+/H, O++/H
      • [x] Include HgHb_composite, HdHb_composite
      • [x] Pass Chun's local test
      • [x] Pass Caroline's local test
        • [x] Mass bins
        • [x] Mass-LHbeta bins
    • [x] Run error_prop.fluxes_derived_prop with raw=False, binned_data=True, apply_dust=True, revised=True
      • [x] Output file called 'bin_derived_properties.MC.dustcorr.tbl'
      • [x] Contains flux ratios: two_beta_composite, three_beta_composite, R_composite
      • [x] Contains flux ratios: logR23_composite, logO32_composite
      • [x] Contains errors for flux ratios: two_beta_composite, three_beta_composite, R_composite
      • [x] Contains errors for flux ratios: logR23_composite, logO32_composite
      • [x] Contains derived properties: T_e, 12+log(O/H), log(O+/H), log(O++/H), O+/H, O++/H
      • [x] Contains errors for derived properties: T_e, 12+log(O/H), log(O+/H), log(O++/H), O+/H, O++/H
      • [x] Contains dust info: HgHb_composite, HdHb_composite, EBV_HgHb, and EBV_HdHb
      • [x] Contains errors for dust info: HgHb_composite, HdHb_composite, EBV_HgHb, and EBV_HdHb
      • [x] Pass Chun's local test
      • [x] Pass Caroline's local test
        • [x] Mass bins
        • [x] Mass-LHbeta bins
astrochun commented 4 years ago

Ran on Evolution of Galaxies for first case using 10182020 set. Executed fine. HgHb and HdHb needs to include _composite. This is now fixed in c7dc377

astrochun commented 4 years ago

Initially I had remove (using dict.pop) HgHb_composite and HdHb_composite for apply_dust=False in part because we didn't need it. But as it turns out, calling error_prop.fluxes_derived_prop with raw=False will use the bin_derived_properties.tbl, which is generated with raw=True. Without those columns, below is the resulting message:

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-4-bc142c0a09ca> in <module>
----> 1 error_prop.fluxes_derived_prop('/Users/cly/GoogleDrive/Research/MZEvolve/massbin/10192020/dust_implement_test_N_format_test1/', raw=False, binned_data=True, apply_dust=False, revised=False)

~/codes/python/Metallicity_Stack_Commons/Metallicity_Stack_Commons/analysis/error_prop.py in fluxes_derived_prop(path, raw, binned_data, apply_dust, revised)
    205         for name0 in flux_ratios_dict.keys():
    206             err_prop, peak_prop = compute_onesig_pdf(flux_ratios_dict[name0],
--> 207                                                      prop_tab0[name0],
    208                                                      usepeak=True)
    209 

/Applications/anaconda3/lib/python3.7/site-packages/astropy/table/table.py in __getitem__(self, item)
   1347     def __getitem__(self, item):
   1348         if isinstance(item, str):
-> 1349             return self.columns[item]
   1350         elif isinstance(item, (int, np.integer)):
   1351             return self.Row(self, item)

/Applications/anaconda3/lib/python3.7/site-packages/astropy/table/table.py in __getitem__(self, item)
    189         """
    190         if isinstance(item, str):
--> 191             return OrderedDict.__getitem__(self, item)
    192         elif isinstance(item, (int, np.integer)):
    193             return self.values()[item]

KeyError: 'HgHb_composite'
astrochun commented 4 years ago

I ran error_prop.derived_flux_prop() twice with raw=True and got the same results (empty diff):

$ diff bin_derived_properties_run1.tbl bin_derived_properties.tbl

However, with raw=False I kept getting different two_beta_composite values.

This turned out to be a bug. It is fixed in 79790f2

astrochun commented 4 years ago

Attempted to change column order but that's a bit tough with use of different keywords [360cb99]