timj commented 6 years ago

Review of LSST: from Science Drivers to Reference Design and Anticipated Data Products by Ivezic etal... 60 Mbytes!

This is a paper describing the Large Synoptic Survey Telescope and its science goals. Since the hardware is being built now, some of the parameters are very well known while others are estimates based on anticipated performance that will have to be verified during initial operations. Since the LSST is going to be a major source of data in the 2020's, this is a useful contribution to the astronomical literature and should be published after revision. But I do note that this MS is arXiv:0805.2366 and is thus fairly well known already, although this version is v5 on the arxiv, as described in Appendix A.

52: Given the 10 year history of this paper, some of the science discussions are stale and should be updated. I have identified some places where this should be done, but there are no doubt other places that could be improved.

Here are some suggestions:

33: The "Deep-Wide-Fast" survey is supplemented by a "Very Deep" survey and a "Fast time domain" survey [in the abstract]. This is too reminiscent of Starbucks' coffee sizes, where "grande" isn't big.
32: It would be good to have more current seeing data in Figure 1. Seeing always seems to get worse after site selection probably due to regression to the mean.
34: The deep drilling fields in Figure 18 do not appear to be very much deeper than the main survey: 1.8x on the color bar. This figure might need a logarithmic color bar.

And some things to fix:

35: A Figure showing the cadence for 1 or more typical spots would be useful, and it should show which filter is used for each visit. Something like a timeline with different color ticks indicating the visits. This should probably cover one lunation to keep the scale readable.

36: State how long it takes to change filters. This is relevant for getting an SED of a rapidly changing transient.

37: The Zwicky Transient Facility is up and running, and the author list of this paper includes many ZTF team members. Based on ZTF experience, you should have a fairly good updated estimate of what the LSST transient alert stream should look like. While you do mention several million alerts per night, you should mention how these will break down into supernovae, fast transients, stellar flares, and asteroids.

38: Standard sirens will not be "found" by LSST [p 36]. They have to found and measured using LIGO/VIRGO/KAGRA/IndIGO and then the EM counterparts need to be found, and LSST will be a great asset for this follow-up activity. So rewrite this to say LSST will followup GWB alerts and is likely to find 70 EM counterparts if the gravitational wave network is working well.

39: The trailing loss Eqn (8) has weird units. vt/theta should be dimensionless, so express v in arc-sec/sec, t in sec, and theta in arc-sec. This gets rid of the weird 24 in Eqn (9). Eqn (8) is basically noise \propto sqrt(FWHM^2+[trail length]^2) which is pessimistic relative to a search tuned for trails. This would have noise \propto sqrt(FWHM*sqrt(FWHM^2+[trail length]^2)) but would be a substantial computational burden to do a 4-D search for trails. You have to either explain your reasoning or give a citation for Eqn (8).

40: On page 28, H does NOT scale directly with the diameter of an object. That would say that an object with D=1400 m would have H=220 since an object with H=22 has D=140 m for albedo = 14%. H is a logarithmic quantity representing a flux which scales quadratically with diameter. Explain what size distribution is equivalent to your dN/dH~10^{H/3}.

41: The claim in Figure 24 that LSST colors will be twice as accurate as SDSS colors for asteroids depends on being able to phase up very sparse light curves in the different filters, which will be a considerable task, while the SDSS filters were all obtained within about 2 minutes.

42: \S 2.1.2: It should be noted in the text that this requirement: "The photometry should be better than 1'?'€'?'2% to measure asteroids' colors and thus determine their types. The different filters should be observed over a short time span to reduce apparent variations in color due to changes in observing geometry, but should be repeated over many lunations in order to determine phase curves and allow shape modeling." is not met by the proposed main survey. The typical asteroid changes its flux by 1% in 100 seconds or so. The filter change cycle is hard to find - I suppose I could look at minion_1016 - but I believe it is on a much longer time scale than 100 seconds. Thus the caption of Figure 24 stating that LSST colors will be twice as good as SDSS colors is not supported. SDSS observed all 5 bands in O(10^2) seconds, while LSST may well take days.

43: Also the Congressional mandate to find 90% of all NEOs 140 m and up will not be met, although finding a large fraction of NEOs brighter the H=22 is feasible. It's those pesky coal black C types that won't be found by LSST: a quarter of the NEOs at 140 m diameter have H > 23. The albedo used in \S 3.2.2 is hard to find, but the statement that H=22 is equivalent to 140 m gives 14%. Most C-types are much darker than this.

44: Specify the atmospheric dispersion in the filters. You mention that going for wider passbands would make this worse but do not give the magnitude. And call it "atmospheric dispersion", not "chromatic atmospheric refraction", even though these mean the same thing. Some instruments have ADC systems, none have CARC systems. This seems like an important systematic noise term for weak lensing studies.

45: How does the effective number density of weak lensing background galaxies compare with Euclid and WFIRST? Euclid is observing a similarly large area with a better PSF, no seeing, no weather, no Moon, no daytime, and a darker sky.

46: The weak lensing results will be competing with Euclid which has a similar start date. There are several surveys that have been "skimming the cream on weak lensing science", including CFHTLenS, PanSTARRS and DES. So far the cream has not produced spectacular results. Figure 22 seems frozen in the DETF days. Figure 22 should acknowledge the surveys in the can, and show an "All" that include Euclid WL and then "All minus LSST WL". So you should include a fair assessment of how much the LSST will advance the field over where it will be given current and under construction surveys.

47: In discussing the proper motion and parallax accuracy, it should be pointed out that achieving the required 10 mas 1 sigma 1 axis single visit accuracy requires SNR of 30, so the magnitude limit is 2 mag brighter than the single visit SNR 5 limit. Still very good: r = 22.5. Later on p 29 there is a more conservative calculation showing r = 21 for 0.2 mas/yr. So give a magnitude limit during the first proper motion accuracy discussion, and state that accuracy scales like 1/flux for fainter objects.

48: You should cite actual performance of Gaia in addition to the pre-launch projected performance.

49: The telescope latitude and longitude are specified to 0.01" which is 31 cm. Are these values really that good? And if so, what part of the telescope do they correspond too? That should be specified. [p 16]

50: Footnote 14: there should be an S^2 term as well since there are flux errors proportional to the flux. The coefficient will probably be small but the term will be present.

51: Eqn 6 & 7: These are not consistent in their treatment of the exposure time. Eqn(6) is pure sqrt(t) while Eqn(7) is poorly written and not explained well. \Delta C^\infty_m is not the loss due to readout noise when the integration time goes to infinity, because that is zero. Rewrite to explain better.

timj commented 6 years ago

@ivezic Let me know if you would like me to file an issue for each comment separately so that different people can be assigned. It might be easier to track things that way.

cc/ @drphilmarshall @smr456 @RobertLuptonTheGood

drphilmarshall commented 6 years ago

I'd certainly appreciate being assigned the DESC-related fixes so I can delegate them... and I think I'd end up making one issue per fix in the end.

timj commented 6 years ago

I've added individual issues for each referee comment. They are all assigned to @ivezic for now but obviously can be reassigned.

ivezic commented 6 years ago

Thanks Tim! I will start working on it next week.

ivezic commented 6 years ago

Tim, thanks so much for organizing these referee comments into issues! It made it so easy to assign people to comments. I am puzzled why I couldn't find @drphilmarshall (and Leanne) on the list of possible assignees... I will wait for ~2 weeks and then revisit the list and ping people who don't respond. Thanks again!

drphilmarshall commented 6 years ago

I think it's because I don't have Read or Write access to this repo, I can only follow by "Watching" it. Fortunately I am indeed Watching! I saw the DESC assignments, and will get something going.

drphilmarshall commented 6 years ago

PS. We can work in our https://github.com/LSSTDESC/overview_paper fork, no problem - which branch on this base repo should we submit PRs to?

timj commented 6 years ago

Use apj-v1 (see #30)

timj commented 5 years ago

The revised version was submitted on December 20th (b74008c). It was given ID AAS11501R1

lsst-pst / overview_paper

Referee report #1 for ApJ #31

52: Given the 10 year history of this paper, some of the science discussions are stale and should be updated. I have identified some places where this should be done, but there are no doubt other places that could be improved.

33: The "Deep-Wide-Fast" survey is supplemented by a "Very Deep" survey and a "Fast time domain" survey [in the abstract]. This is too reminiscent of Starbucks' coffee sizes, where "grande" isn't big.

32: It would be good to have more current seeing data in Figure 1. Seeing always seems to get worse after site selection probably due to regression to the mean.

34: The deep drilling fields in Figure 18 do not appear to be very much deeper than the main survey: 1.8x on the color bar. This figure might need a logarithmic color bar.

35: A Figure showing the cadence for 1 or more typical spots would be useful, and it should show which filter is used for each visit. Something like a timeline with different color ticks indicating the visits. This should probably cover one lunation to keep the scale readable.

36: State how long it takes to change filters. This is relevant for getting an SED of a rapidly changing transient.

41: The claim in Figure 24 that LSST colors will be twice as accurate as SDSS colors for asteroids depends on being able to phase up very sparse light curves in the different filters, which will be a considerable task, while the SDSS filters were all obtained within about 2 minutes.

45: How does the effective number density of weak lensing background galaxies compare with Euclid and WFIRST? Euclid is observing a similarly large area with a better PSF, no seeing, no weather, no Moon, no daytime, and a darker sky.

48: You should cite actual performance of Gaia in addition to the pre-launch projected performance.

49: The telescope latitude and longitude are specified to 0.01" which is 31 cm. Are these values really that good? And if so, what part of the telescope do they correspond too? That should be specified. [p 16]

50: Footnote 14: there should be an S^2 term as well since there are flux errors proportional to the flux. The coefficient will probably be small but the term will be present.