Should there be science cases described without a metric in the overview paper?

[rbiswas4]

@drphilmarshall @egawiser @MichelleLochner

The current focus of the white paper is on developing metrics. On the other hand, we recognize that the whitepaper will be a living document, where additions will be made in future.

If there are science cases that we find interesting, but do not have metrics for them yet, should we be putting descriptions of the science cases into the whitepaper with the (statement and) understanding that the metrics will be developed later, or should we keep the description of such cases out of the white paper altogether till the metrics are ready?

This is a question that came up for me in a discussion with Bob Nichol and Dario Scovacricchi (but I don't know their github ids).

[egawiser]

As mentioned at the DESC meeting, we are willing to include science cases with not-yet-well-developed metrics as long as there is a discussion of how the metric will be developed. However, the goal of the white paper, as I understand it, is not to be an exhaustive list of science cases. Hence something you find "interesting" that doesn't have a metric seems best to leave out, or mention briefly, but a critical science case that doesn't have a metric but for which the path to one can be described should be included. That's my answer, but let's see what @drphilmarshall says... :)

[drphilmarshall]

@rbiswas4 wrote:

If there are science cases that we find interesting, but do not have metrics for them yet, should we be putting descriptions of the science cases into the whitepaper with the (statement and) understanding that the metrics will be developed later, or should we keep the description of such cases out of the white paper altogether till the metrics are ready?

This is an excellent question, and one that needs discussing by the group, I think. This is because it is closely related to the question, "how are we going to meet our submission deadline?" :-) I spoke to @bethwillman and @ivezic today, and we talked about this, as well as the #29 ("Should we publish the white paper in a journal").

@egawiser is right, that the white paper is not meant to be exhaustive - so for now, I would focus on the science cases that have well-defined metrics with quantified results, and preferably those where we can already answer the question of how good simulated cadence X would be for my science - and leave the important but as yet unquantifiable science cases for later versions. Critical science cases that do not yet have metrics can be referred to in the chapter or section introductions.

[fedhere]

in the transients group this gets tricky: we want to mention gravitational waves, which need ToO. we are not planning on developing a metric for it, although we have simulated killonova lightcurves the case that no cadences would cut it is fairly clean cut. thoughts?

[drphilmarshall]

I think ToO observations for GW sources is too important not to mention anywhere, but if we do not yet have any simulations that contain simulated ToO observations, it'll be hard to show improvement in your metrics. I guess what you could do is design a metric and compute it for the baseline cadence - you might get some GW location-matched kilonovae just in the universal cadence survey. And then if we can produce an OpSim run that emulates GW ToOs, you could show by how much the metrics improve. However, at this point in time, if there is not yet a metric plan, I think GW ToOs might have to be a paragraph in the transients chapter, and perhaps also in 2.6 (Example Observing Strategies, Future Work). How does that sound?

[StephenRidgway]

A GW ToO will probably execute a visit pattern over one or more positional error windows. Since it would be done presumably with high priority, it would be accomplished subject to feasibility. I’m not sure that a metric on that execution will tell anything at all about the performance of the scheduler.

However, the real measure of the impact of that ToO is in all the other metrics. Ideally, the ToO would just hit some fields outside of the normal sequence, but those visits would be counted towards the totals for those fields. So possibly other metrics would not suffer much or not at all if no sequences were broken.

On Apr 11, 2016, at 4:23 PM, Phil Marshall notifications@github.com wrote:

I think ToO observations for GW sources is too important not to mention anywhere, but if we do not yet have any simulations that contain simulated ToO observations, it'll be hard to show improvement in your metrics. I guess what you could do is design a metric and compute it for the baseline cadence - you might get some GW location-matched kilonovae just in the universal cadence survey. And then if we can produce an OpSim run that emulates GW ToOs, you could show by how much the metrics improve. However, at this point in time, if there is not yet a metric plan, I think GW ToOs might have to be a paragraph in the transients chapter, and perhaps also in 2.6 (Example Observing Strategies, Future Work). How does that sound?

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub https://github.com/LSSTScienceCollaborations/ObservingStrategy/issues/291#issuecomment-208613633

[AshishMahabal]

Hi,

One of the useful numbers will be the depth of galaxy catalogs corresponding to the depth of LSST observations. That will determine how long LSST will need to integrate for counterparts possibly associated with known galaxies. Unless the error circle is at most a few LSST fields, tiling with LSST to break cadence of too many other fields may not be feasible (and will affect their metrics, as Steve says).

Yes, we should mention GW ToOs. During the recent roadmap meeting the GW follow-up group has become somewhat active. We will be able to get some text from them.

-ashish

On Apr 11, 2016, at 4:45 PM, StephenRidgway notifications@github.com wrote:

A GW ToO will probably execute a visit pattern over one or more positional error windows. Since it would be done presumably with high priority, it would be accomplished subject to feasibility. I’m not sure that a metric on that execution will tell anything at all about the performance of the scheduler.

However, the real measure of the impact of that ToO is in all the other metrics. Ideally, the ToO would just hit some fields outside of the normal sequence, but those visits would be counted towards the totals for those fields. So possibly other metrics would not suffer much or not at all if no sequences were broken.

On Apr 11, 2016, at 4:23 PM, Phil Marshall notifications@github.com wrote:

I think ToO observations for GW sources is too important not to mention anywhere, but if we do not yet have any simulations that contain simulated ToO observations, it'll be hard to show improvement in your metrics. I guess what you could do is design a metric and compute it for the baseline cadence - you might get some GW location-matched kilonovae just in the universal cadence survey. And then if we can produce an OpSim run that emulates GW ToOs, you could show by how much the metrics improve. However, at this point in time, if there is not yet a metric plan, I think GW ToOs might have to be a paragraph in the transients chapter, and perhaps also in 2.6 (Example Observing Strategies, Future Work). How does that sound?

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub https://github.com/LSSTScienceCollaborations/ObservingStrategy/issues/291#issuecomment-208613633

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub https://github.com/LSSTScienceCollaborations/ObservingStrategy/issues/291#issuecomment-208622926

[fedhere]

we have a conference call tomorrow with the people writing the transient chapter. i will discuss all these suggestions there. my opinion, and our current plan, is to discuss GW and the need for ToO for the GW counterpart discovery program, although we are unlikely to develop metrics about this it in time. I will report back after I talk to the paragraph writers.

f

On Mon, Apr 11, 2016 at 7:59 PM, Ashish Mahabal notifications@github.com wrote:

Hi,

One of the useful numbers will be the depth of galaxy catalogs corresponding to the depth of LSST observations. That will determine how long LSST will need to integrate for counterparts possibly associated with known galaxies. Unless the error circle is at most a few LSST fields, tiling with LSST to break cadence of too many other fields may not be feasible (and will affect their metrics, as Steve says).

Yes, we should mention GW ToOs. During the recent roadmap meeting the GW follow-up group has become somewhat active. We will be able to get some text from them.

-ashish

On Apr 11, 2016, at 4:45 PM, StephenRidgway notifications@github.com wrote:

A GW ToO will probably execute a visit pattern over one or more positional error windows. Since it would be done presumably with high priority, it would be accomplished subject to feasibility. I’m not sure that a metric on that execution will tell anything at all about the performance of the scheduler.

However, the real measure of the impact of that ToO is in all the other metrics. Ideally, the ToO would just hit some fields outside of the normal sequence, but those visits would be counted towards the totals for those fields. So possibly other metrics would not suffer much or not at all if no sequences were broken.

On Apr 11, 2016, at 4:23 PM, Phil Marshall notifications@github.com wrote:

I think ToO observations for GW sources is too important not to mention anywhere, but if we do not yet have any simulations that contain simulated ToO observations, it'll be hard to show improvement in your metrics. I guess what you could do is design a metric and compute it for the baseline cadence - you might get some GW location-matched kilonovae just in the universal cadence survey. And then if we can produce an OpSim run that emulates GW ToOs, you could show by how much the metrics improve. However, at this point in time, if there is not yet a metric plan, I think GW ToOs might have to be a paragraph in the transients chapter, and perhaps also in 2.6 (Example Observing Strategies, Future Work). How does that sound?

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub < https://github.com/LSSTScienceCollaborations/ObservingStrategy/issues/291#issuecomment-208613633

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub < https://github.com/LSSTScienceCollaborations/ObservingStrategy/issues/291#issuecomment-208622926

— You are receiving this because you commented. Reply to this email directly or view it on GitHub https://github.com/LSSTScienceCollaborations/ObservingStrategy/issues/291#issuecomment-208626789

/|/\ |* Please consider the environment before printing this e-mail