LSST DESC Note: Twinkles Design

[drphilmarshall]

OK, we have a folder for this note and a number of things we need to do. I think this one could be latex, so that we can a) efficiently self-plagiarize when writing the Twinkles paper and b) cite papers well. Since writing latex is an epic task, here goes:

• [x] Basic latex Note set-up (see https://github.com/DarkEnergyScienceCollaboration/ComputingInfrastructure/issues/32)
• [x] #300 Note layout in the note's own branch, with sections to be completed, including:
  • [x] Introduction
  • [ ] Twinkles: a Tiny Simulated LSST Sky Survey
  • [ ] Pipeline Development
  • [ ] Validation/Science Analysis
  • [ ] Discussion and Conclusions
• [ ] Content based on notes in threads linked to old "Specifications" issue #200
• [ ] Automatic PDF build, even of draft

Comments welcome! For example, @rbiswas4 I'm thinking we can use the science analysis plan section to help us think about what we require from the Run 3 DM L2 processing. @wmwv @cwwalter I'll be bouncing sections to you after making a start. Let us know what else this note should contain.

@sethdigel interested to hear your thoughts on self-plagiarization between Notes and journal papers. It seems somewhat acceptable to do this between eg conference proceedings and papers, and certainly between papers and theses.

NB. This issue was is part of the Run 1 / 1.1 / 2 LSST DESC Notes epic #284

[sethdigel]

Well, self-plagiarism is the sincerest form of self-flattery.

For what it is worth, I don't see a problem with recycling unpublished text from DESC Notes for journal papers. If the overlap would be extensive, it might be worth skipping the overhead of the DESC note stage and starting work on a paper. Or if there's not so much overlap, you might want to consider just referring to a DESC Note from a journal paper. This would require the DESC Note to be publicly available, but I know you have looked into how that might be done. That approach wouldn't be my first choice, for a couple reasons. Also, you have pointed out that preparing DESC Notes on a topic could be a way to objectively demonstrate authorship eligibility for a related journal paper.

[cwwalter]

Dear Seth and Phil,

I also suspect things will be different in the “results papers era”. In that case, I think there is an expectation that people will write research notes to further document the analysis at level beyond what is in the papers themselves. The review committees will use this information to approve results and papers. Some of these may also become public but I would expect large sections of those notes would be lifted and put into the papers themselves. For me that is almost the main point.

-Chris

On Aug 26, 2016, at 2:22 AM, Seth Digel notifications@github.com wrote:

Well, self-plagiarism is the sincerest form of self-flattery.

For what it is worth, I don't see a problem with recycling unpublished text from DESC Notes for journal papers. If the overlap would be extensive, it might be worth skipping the overhead of the DESC note stage and starting work on a paper. Or if there's not so much overlap, you might want to consider just referring to a DESC Note from a journal paper. This would require the DESC Note to be publicly available, but I know you have looked into how that might be done. That approach wouldn't be my first choice, for a couple reasons. Also, you have pointed out that preparing DESC Notes on a topic could be a way to objectively demonstrate authorship eligibility for a related journal paper.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_DarkEnergyScienceCollaboration_Twinkles_issues_299-23issuecomment-2D242640236&d=CwMFaQ&c=imBPVzF25OnBgGmVOlcsiEgHoG1i6YHLR0Sj_gZ4adc&r=-dvMKp7qvRQfGKAINi3CHQ1wJ-kLZBJGuLQvW1Rur8Y&m=O3b4--wKiXnxLw1PEo5yke_pSvT03TKAzMAoGV84Hmg&s=SiVpB9cdbT_iNC7f0aEbfuLez8XvqrR6Y4AYwiWrZls&e=, or mute the thread https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AFThdCQbKqMN2O-5FssSKzJ1Zf68pq-2DPdtks5qjoYfgaJpZM4JtqMQ&d=CwMFaQ&c=imBPVzF25OnBgGmVOlcsiEgHoG1i6YHLR0Sj_gZ4adc&r=-dvMKp7qvRQfGKAINi3CHQ1wJ-kLZBJGuLQvW1Rur8Y&m=O3b4--wKiXnxLw1PEo5yke_pSvT03TKAzMAoGV84Hmg&s=oFSTVzgwGt29JMrhCSsrn-oxCpAXjTGw6G-EpzvYU9A&e=.

[rbiswas4]

@drphilmarshall this looks great, but I am a little confused on whether you wanted me to write something yet, and if so where.

[drphilmarshall]

Why, thank you! Feel free to write down whatever you want and check it in to the folder however you want, and we can refactor later - but it'd be fine to wait until I have put in place a skeleton tex file and pointed you at it. Most checkboxes will get their own issues eventually.

[drphilmarshall]

Cross-reference: lots of useful information in the "Twinkles Specifications" issue #200 - moving those old sub-issues to this epic.

[drphilmarshall]

It's possible that the only place that the Twinkles flowchart ( #30 ) belongs is in this design note - and then only in Scott's suggested reduced form.

[jbkalmbach]

After a conversation here at Hack Week, we need to calculate some numbers on the size of the instance catalog information that we will want to put into databases since there are limits on what will be allowed into the real databases for LSST.

[drphilmarshall]

In particular, @brianv0 mentioned the 10% limit for Level 3 storage. We decided we would scope out the instance catalog table and estimate its size relative to the ForcedSource table - but then also discuss the size of the Twinkles sky area compared to the DDFs. The context is that during operations we can imagine wanting to drop realistic sources into copies of the visit images, and then run the pipelines on them in order to understand our error models - and the Twinkles results will give scaleable (ie, per sq arcmin) answers to the question of how much such a program would cost.