Paper Feedback Round 2: @RickKessler

[aimalz]

I'm seeking volunteers to implement the changes that are not tagged with my name. If you'd like to help, please make a branch, leave a comment on this issue with a link to the branch, check off boxes in this issue as you go, and make a PR as soon as you've gotten through everything you wanted to take on. Thank you!

ABSTRACT

• [x] 2nd line, I was initially confused about "probabilistic data products" but after reading a bit I realized that this should be "probabilistic classifications"
• [x] optimal global -> optimal
• [x] "broader community outside astronomy" implies that astronomy community is not invited to participate.
• [x] reassuring discovery -> reassuring conclusion [discovery is a bit over-stated]
• [x] I don't quite understand "superior sensitivity to the most concerning potential failure modes" What does it mean to be sensitive to a failure mode? And there is no context here for anything to fail; it's just good or bad classifications.

The sentence has been rephrased. --@aimalz

INTRO

• [x] L102: remove 'naturally'
• [x] L419-422: I don't understand this sentence. I am not aware of any attempt to use SNIbc for cosmology analysis.

The sentence has been rephrased to clarify that the SN Ibc are contaminants accidentally included as if they were SN Ia, not intentionally used with some unspecified procedure. --@aimalz

• [x] L438: successfully prevents -> minimizes
• [x] L495: "photometric cosmology analysis" is jargon. It is a cosmology analysis using photometrically identified SNIa.
• [x] L531-532: this statement is incorrect. The SNPHOTCC metric was not based on spectroscopic resources, but instead based on contamination of the Hubble diagram.
• [x] L574: remove 'directly'
• [x] L591: remove 'explicitly'
• [x] L611: must -> is designed to
• [x] L611: remove 'all'
• [x] L616: remove 'obviously have to'
• [x] After L634, remove 'systematically'
• [x] L646: Because of this CAVEAT
• [x] Fig 5: I didn't understand the main points of this discussion, and the symbols are hard to distinguish. And why not plot vertical axis on log-scale to avoid successive magnifcation of vertical ranges ?

A log scaling doesn't actually fix the problem, unfortunately. The changes in horizontal ranges are also necessary, and when both are log-scaled, everything gets compressed into the corner even worse. --@aimalz

• [x] L732: remove 'also'
• [x] Table 3: would be easier to parse if each column is mapped so that MIN,MAX -> 0,1. The L738 statement that FOM differs from the others is pointless, since it could be a monotonic mapping. L740: agreeing over winner and loser is technically correct, but they don't agree in between, suggesting some luck in picking the same winner & loser. Would help to see these (table or plotted) after re-mapping MIN/MAX -> 0/1.

To preserve interpretability, I'm not going to impose a linear normalization. However, I added some discussion of the magnitude of differences to address this point. --@aimalz

Sec 5:

• [x] L745: first statement is misleading since there is only one metric in PLASTICC. Should clearly state what is used for PLASTICC itself, then start on discussion about use of other metrics.
• [x] L754: There exist plans for -> We are planning
• [x] L757: BELOW, we proceed to ...
• [x] L762: of -> for
• [x] L765: to fitting -> for fitting
• [x] L769: in addition to spectroscopic follow-up, should mention other kinds of potential follow-up: NIR, HST/JWST (deeper NIR), X-ray. I'm sure there are other science cases I have left out.
• [x] L771: wasted -> spent [mis-classified events may still be useful, just not as useful as one had hoped. And let's not get in the habit of telling the community that we might waste spectroscopic resources]
• [x] L774: might -> should. [if you really aren't sure, should indicate why]
• [x] L783: optimally metric ??
• [x] L788: might enable it. -> is appropriate. [after reading further, you didn't actually provide a metric for the cases below]
• [x] L808-818: since you give SNIa type in previous paragraph, here you could give KN event as example ... if there are 5 likely candidates, we might be willing to take a spectrum for all 5.

Sec 6

• [x] L850-852: I worry that this sentence will alarm PLASTICC participants about what the real metric is ... the final choice of log-loss isn't presented until the end. Suggest starting by presenting our choice, and then go into the 2-metric discussion.
• [x] "Author contributions" ? Do journals allow this ? I have never seen this before. And I didn't know you could acknowledge your own collaborators.

This is actually part of the LSST-DESC Publication Policy. There have not been problems with including it in LSST-DESC papers thus far. --@aimalz

[juramaga]

Hi Alex, I'll work in this branch: https://github.com/aimalz/proclam/tree/corrections_RM I will start going down the list.

[aimalz]

@juramaga Were you able to push changes? I don't see any edits in your branch, but you've checked off some boxes on the issue. Thanks!

[juramaga]

Hi,

Was in a conference and had to interrupt the work. Starting again now. Will push changes during the day.

Rafael

On Thu, Sep 6, 2018 at 8:58 AM, Alex Malz notifications@github.com wrote:

@juramaga https://github.com/juramaga Were you able to push changes? I don't see any edits in your branch, but you've checked off some boxes on the issue. Thanks!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/aimalz/proclam/issues/62#issuecomment-418986229, or mute the thread https://github.com/notifications/unsubscribe-auth/AFP5A_1-k5MFQkD25p4Q15iNIClQqx-Sks5uYMeRgaJpZM4WVvP4 .

--

J. Rafael Martinez-Galarza Harvard-Smithsonian Center for Astrophysics 60 Garden Street - MS 66 Cambridge, MA 02138 U. S. A. +1 617 495 7027 jmartine@cfa.harvard.edu

[juramaga]

Hi Alex,

I just pushed some changes to my branch.

Rafael

On Thu, Sep 6, 2018 at 8:58 AM, Alex Malz notifications@github.com wrote:

@juramaga https://github.com/juramaga Were you able to push changes? I don't see any edits in your branch, but you've checked off some boxes on the issue. Thanks!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/aimalz/proclam/issues/62#issuecomment-418986229, or mute the thread https://github.com/notifications/unsubscribe-auth/AFP5A_1-k5MFQkD25p4Q15iNIClQqx-Sks5uYMeRgaJpZM4WVvP4 .

--

J. Rafael Martinez-Galarza Harvard-Smithsonian Center for Astrophysics 60 Garden Street - MS 66 Cambridge, MA 02138 U. S. A. +1 617 495 7027 jmartine@cfa.harvard.edu

[reneehlozek]

Finished addressing RK comments.