Script to start notes

[nregnault]

Hi @heather999,

this is what we want to do:

 cd doc/desc_notes/
 startnote snia

Would you be able to work on this please ? Thank you ! :-)

[heather999]

Certainly, @nregnault!
Just to confirm, (also tagging @drphilmarshall), snia signifies the filename you would like associated with this new LSST DESC Note?

[drphilmarshall]

Yes: we'd use 'snia' as the name of the sub-folder to contain the Note and its figures etc, and the Note itself would be in main.md.

On Fri, Jul 22, 2016 at 3:19 AM, Heather Kelly notifications@github.com wrote:

Certainly, @nregnault https://github.com/nregnault!

Just to confirm, (also tagging @drphilmarshall https://github.com/drphilmarshall), snia signifies the filename you would like associated with this new LSST DESC Note?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/DarkEnergyScienceCollaboration/ComputingInfrastructure/issues/28#issuecomment-234509032, or mute the thread https://github.com/notifications/unsubscribe-auth/AArY96e2j2X0T3NbhkoweVxbS13xH3iiks5qYJlPgaJpZM4JSoFJ .

[drphilmarshall]

Just catching up on the LSST technotes documentation - looks like they initialize their notes using cookiecutter: https://github.com/lsst-sqre/lsst-technote-bootstrap As @jonathansick pointed out in #24 we could think of the DESC notes being just another series, analogous to the DM notes - but I want to understand the one repo per note approach first. So, I tried starting a DM technote to see what happens, but got an error - I'll issue it to @jonathansick and report back!

[jonathansick]

I just fixed the issue (my bad! didn't fully test a recent change to the cookiecutter template)

[drphilmarshall]

Thanks Jonathan!

So, the following command does most all of what we want:

cookiecutter https://github.com/lsst-sqre/lsst-technote-bootstrap.git

Here's the output I get when I run it:

Cloning into 'lsst-technote-bootstrap'... remote: Counting objects: 124, done. remote: Total 124 (delta 0), reused 0 (delta 0), pack-reused 124 Receiving objects: 100% (124/124), 32.17 KiB | 0 bytes/s, done. Resolving deltas: 100% (66/66), done. Checking connectivity... done. first_author [First Last]: Phil Marshall Select series: 1 - SQR 2 - DMTN 3 - SMTN Choose from 1, 2, 3 [1]: 2 serial_number [000]: 037 title [Document Title]: LSST DESC Notes: Guidelines for Authors repo_name [dmtn-037]: descnotes-037 Select github_org: 1 - lsst-sqre 2 - lsst-dm 3 - lsst-sims Choose from 1, 2, 3 [1]: 2 github_namespace [lsst-dm/descnotes-037]: docushare_url []: url [http://dmtn-037.lsst.io]: description [A short description of this document]: copyright_year [2016]: copyright_holder [AURA/LSST]: LSST DESC

We could think of adding a DESC Notes series as an additional option. I think most of our Notes will exist inside project repositories, though - that's why I wanted to see what the cookiecutter makes. The answer is a folder, just as we had designed. At the moment, each LSST technote inhabits a repo of its own - but it's not obvious that they have to. Heres what cookiecutter made:

pjm@PPA-PC92478 > ls descnotes-037 pjm@PPA-PC92478 > cd descnotes-037/ pjm@PPA-PC92478 > ls LICENSE README.rst conf.py metadata.yaml Makefile _static index.rst requirements.txt

I guess it would not be too hard to make the index.rst the DESC one instead of the standard LSST one; likewise for the LICENSE and README.rst. One reservation I have is that the LSST technotes index.rst files are supposed to not have titles and authors in them, which makes their presentation on GitHub distinctly un-paper-like. For DESC Notes in private repos this is a problem: the product needs to look like a paper.

Having said that, I think we should hold off writing any sort of start up script ourselves for now, as I think the LSST technote cookiecutter might end up doing what we want it to :-)

[jonathansick]

@drphilmarshall Yeah, I've been thinking about having the title go in index.rst rather than the metadata file. This touches a larger discussion I'm having about how DM should handle metadata for its repos. I've been talking to folks like @arfon and @augustfly and I'm now convinced I want to migrate DM from our ad-hoc metadata.yaml to JSON-LD (e.g. http://www.arfon.org/json-ld-for-software-discovery-reuse-and-credit) and then have a metadata server that completes/generates the 'metadata.json' file in a repo with metadata extracted the repo itself (e.g., authors and date from git, title from index.rst, license info from LICENSE, etc.). Anyways, all this to say that I'll be changing the title/author metata, but it'll be part of a bigger effort.

Another thing I want to mention is that you may encounter difficulties rendering reST on GitHub. You can't use any of the role/directive syntax defined by Sphinx. In DM, I've solved that by pushing people to look primarily at the rendered HTML published by LSST the Docs. That's a big motivation for why LSST the Docs automatically publishes every branch in a Git repo.

And yeah, I think it's fine to host DESC notes inside other repos. It might require some work to get that model to work with publishing on LSST the Docs, but probably is not insurmountable. In DM we put technotes in their own repos so that they can have a well-defined git history for each document. That way when a branch is made in a technote repo, LSST the Docs knows that we mean to publish a new edition of that document.

[drphilmarshall]

I see, thanks Jonathan. I notice that math also does not render on GitHub, either in reST or MD, which is a shame. They don't seem to be about to fix that any time soon though. You saw Seth asking about writing Notes in latex; we had also wondered about supporting Notes written as ipython notebooks (which do support math, on GitHub). Interested to hear whether you have thought about this channel.

I'm about to start an experiment in the Twinkles project, where we need to write a set of LSST DESC Notes describing what we have done so far. I will start the authors off with a template that @heather999 and I put together here:

https://github.com/drphilmarshall/ComputingInfrastructure/blob/master/doc/LSST_DESC_Notes/template_LSST_DESC_Note.rst

and we'll follow your directory structure. This should make it easy to add in whatever yaml or json files we need for LTD later, using an upgraded cookiecutter. In the immediate term we need content to play with! :-) I'll @mention you in the relevant issue thread(s) so that you can follow along with this experiment.

On Wed, Aug 3, 2016 at 12:48 PM, Jonathan Sick notifications@github.com wrote:

@drphilmarshall https://github.com/drphilmarshall Yeah, I've been thinking about having the title go in index.rst rather than the metadata file. This touches a larger discussion I'm having about how DM should handle metadata for its repos. I've been talking to folks like @arfon https://github.com/arfon and @augustfly https://github.com/augustfly and I'm now convinced I want to migrate DM from our ad-hoc metadata.yaml to JSON-LD (e.g. http://www.arfon.org/json-ld-for-software-discovery-reuse- and-credit) and then have a metadata server that completes/generates the 'metadata.json' file in a repo with metadata extracted the repo itself (e.g., authors and date from git, title from index.rst, license info from LICENSE, etc.). Anyways, all this to say that I'll be changing the title/author metata, but it'll be part of a bigger effort.

Another thing I want to mention is that you may encounter difficulties rendering reST on GitHub. You can't use any of the role/directive syntax defined by Sphinx. In DM, I've solved that by pushing people to look primarily at the rendered HTML published by LSST the Docs. That's a big motivation for why LSST the Docs automatically publishes every branch in a Git repo.

And yeah, I think it's fine to host DESC notes inside other repos. It might require some work to get that model to work with publishing on LSST the Docs, but probably is not insurmountable. In DM we put technotes in their own repos so that they can have a well-defined git history for each document. That way when a branch is made in a technote repo, LSST the Docs knows that we mean to publish a new edition of that document.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/DarkEnergyScienceCollaboration/ComputingInfrastructure/issues/28#issuecomment-237350542, or mute the thread https://github.com/notifications/unsubscribe-auth/AArY94We2ujbn5fcDdhyPxcBZx5QqSjFks5qcPCdgaJpZM4JSoFJ .

[jonathansick]

You saw Seth asking about writing Notes in latex; we had also wondered about supporting Notes written as ipython notebooks (which do support math, on GitHub). Interested to hear whether you have thought about this channel.

I think notebooks are totally the way forward for compute-oriented documentation. I've got some ideas about integrating notebooks with Sphinx for publishing and continuous integration system for validation, but haven't personally started on this yet. But I think you could get started today by using nbconvert to convert a Jupyter notebook checked into a repo into PDF/HTML, and publish that PDF/HTML with either github pages or LSST the Docs, etc. The nbconvert stage would happen on Travis, akin to compiling a sphinx project or latex doc.

[jonathansick]

Also, we should make sure to talk and hack during the Project & Community Workshop in a couple weeks in Tucson (assuming you'll be there).

[drphilmarshall]

Yes! That's a good idea. Heather, will you be in Tucson as well?

Great to hear about notebooks! I didn't know about nbconvert. It may well be that some of our Twinkles notes are best presented as ipynb notebooks, so I'll port our template there and offer them the option, so we have some examples to test with. Thanks!

On Thu, Aug 4, 2016 at 10:16 AM, Jonathan Sick notifications@github.com wrote:

Also, we should make sure to talk and hack during the Project & Community Workshop in a couple weeks in Tucson (assuming you'll be there).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/DarkEnergyScienceCollaboration/ComputingInfrastructure/issues/28#issuecomment-237620419, or mute the thread https://github.com/notifications/unsubscribe-auth/AArY91fx9nXaiGjNY8PW1hm612yag7Tvks5qch5tgaJpZM4JSoFJ .

[heather999]

Sadly,I won't be in Tuscon, @drphilmarshall . I would be very happy to participate remotely in some of the discussion, if possible. At the very least, I'm hoping to take advantage of any opportunities to listen into the talks, particularly the one @jonathansick will present.

[drphilmarshall]

Ahh OK no worries - I think the PDF agenda will be updated with blue jeans details soon. You might ping the organizers at LSSTC if they haven't done that yet.

We could also think about jumping on skype if I can find some quality time with Jonathan.

On Thursday, August 4, 2016, Heather Kelly notifications@github.com wrote:

Sadly,I won't be in Tuscon, @drphilmarshall https://github.com/drphilmarshall . I would be very happy to participate remotely in some of the discussion, if possible. At the very least, I'm hoping to take advantage of any opportunities to listen into the talks, particularly the one @jonathansick https://github.com/jonathansick will present.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/DarkEnergyScienceCollaboration/ComputingInfrastructure/issues/28#issuecomment-237649711, or mute the thread https://github.com/notifications/unsubscribe-auth/AArY90Wg4XQY6_7t3CqXe5uX33gX8oI1ks5qcjYvgaJpZM4JSoFJ .

[jonathansick]

We could also think about jumping on skype if I can find some quality time with Jonathan.

Absolutely. I've got a fairly open schedule during the meeting so I can prioritize talking to you folks. There's a lot going on with DESC documentation so it'll be useful to have some real-time conversations and hacking to help me understanding how I can help you.

[heather999]

Ah yes, the remote connection information is online: https://project.lsst.org/meetings/lsst2016/content/remote-connections

I would be very happy to connect via Skype or whatever else works! Looking through the agenda I see the hack sessions on Mon, Tues, Wed as well as the Wednesday DM in-depth that includes Jonathan's talk at ~12:15 pm. I'm already thinking to lurk about for the Hack Sessions via BJs. I will also make a point of being logged into Skype starting on Monday.

Pinging @jchiang87 about those hack sessions, specifically the bits concerning the py3 upgrade which might be interesting.

[heather999]

See https://github.com/LSSTDESC/start_paper