What project templates (or "cookie cutters") do you use?

[gvwilson]

"Cookie cutter" projects are quite common — essentially, they are pre-built templates for project that follow some kind of convention for folder structures and basic boilerplate. Examples include cookiecutter and Project Template.

For Software Carpentry, it serves as a way for us to teach our attendees some best practices while showing them the importance of reproducibility and scripts. It also allows a low barrier of entry to actually set up a project folder structure, since re-organizing folders when a project is fully developed will most likely not happen.

If you use cookie cutter project templates of some kind, we'd like to hear about them: please add pointers to this GitHub issue or mail us directly.

[ncoghlan]

Fedora's DevAssistant is another example: http://devassistant.org/

[rmflight]

I would argue that for R, the package is a logical template for a project, as it can easily contain all of the necessary folders required, and provides a mechanism for automatic generation of reports / papers via the vignette. @hadley has a book he is writing on R packages, and I have written a couple of blog posts 1 2 on how I use packages for organizing data and generating scientific reports.

[embray]

The Astropy project tries to make it easier to set up an affiliated package by providing its own package template for developers to use to start their project, so that they don't have to think too much about details like high level source code layout, how to set up builds and test frameworks, etc. It provides most of what a project that works with Astropy will need, including support for things like Cython, though most of it is not very Astropy-specific. I feel like at the moment it's a little overcomplicated--users clone a GitHub repository and then follow instructions to tweak it for their own project. Though it has also been used successfully for a number of projects. And the use of the template makes it easy to go between development of those projects and not have to learn any new build tooling or anything.

At some point if I ever have time I want to port it to a project template framework like cookiecutter.

[jduckles]

I've developed a cookiecutter for Edition Open Access LaTeX docs. Very simple for now, but gets people over the hump to get a directory setup with the appropriate templates.

[Choens]

I maintain a project template that I use. In addition to a folder structure, it includes some boilerplate code that makes it easier for me to set up and start a new project:

Project Template: https://github.com/Choens/analytic-template-r File Templates: https://github.com/Choens/analytic-templates

I keep meaning to add more templates to the file templates, but haven't done so. I might take a look at @jduckles LaTeX templates to see if I could borrow from some of his ideas.

[sritchie73]

I had a play with ProjectTemplate, and incorporating it into a novice R lesson on projects. However, after writing the lesson, it became clear to me that:

• (a) the directory structure creates was unnecessarily complicated for a novice workshop, and
• (b) automates a lot of things (data loading, library calls), which make it hard to teach.

I ultimately ended up stripped it out, and instead used it as a guide for getting learners to populate a new project with useful subdirectories.

[hjelmj]

Don't have much experience with cookie cutters but was looking for such projects last week as I am interested in standardizing my folder structure for different projects. At the moment I have a different structure for nearly every project. It is self-inflicted but very frustrating.

I came across the following two, I find rr-init most appealing, so will try this one out for the next while.

https://github.com/Reproducible-Science-Curriculum/rr-init

https://github.com/chendaniely/computational-project-cookie-cutter

[hmgaudecker]

As a direct result of SWC, I came up with this:

[code] https://github.com/hmgaudecker/econ-project-templates [tutorial] http://hmgaudecker.github.io/econ-project-templates/

After some initial hiccups, it works well in semester-long courses, in block courses I tend to experience the same as @sritchie73 above.

[benmarwick]

+1 to @rmflight's suggestion to use R packages as a project template (for R users). There are several of us doing this, including @cboettig's template package, @pakillo's template package and @jhollist's manuscriptPackage, etc, in addition to Rob's blog posts and @hadley's book on writing packages.

The concept of an R package as a research compendium has been around for a while in academic literature (cf. Gentleman 2005, Gentleman & Temple Lang 2007), so there is some wider familiarity with R packages for a project structure already.

There are still a bunch of open questions on how best to use the R package structure (https://github.com/ropensci/unconf/issues/11), so it might be more of an intermediate/advanced lesson for people already comfortable with R.

[sritchie73]

Personally I wouldn't use R packages as a project template. R packages are specifically designed for sharing methodology, and trying to shoehorn analysis pipelines/workflows into that structure doesn't quite work (at least for me): Scientific analysis is different to software development.

I posted this in a similar SWC email thread a while back, but take a look at http://projecttemplate.net/ (another R package). I haven;t used it (yet) but it looks quite useful!

On 24 February 2015 at 06:22, Ben Marwick notifications@github.com wrote:

+1 to @rmflight https://github.com/rmflight's suggestion to use R packages as a project template (for R users). There are several of us doing this, including @cboettig https://github.com/cboettig's template package https://github.com/cboettig/template, @pakillo https://github.com/pakillo's template package https://github.com/Pakillo/template and @jhollist https://github.com/jhollist's manuscriptPackage https://github.com/jhollist/manuscriptPackage, etc, in addition to Rob's blog posts and @hadley https://github.com/hadley's book on writing packages.

The concept of an R package as a research compendium has been around for a while in academic literature (cf. Gentleman 2005 http://www.uvm.edu/%7Epdodds/files/papers/others/everything/gentleman2005a.pdf, Gentleman & Temple Lang 2007 http://www.researchgate.net/profile/Duncan_Temple_Lang/publication/40823095_Statistical_Analyses_and_Reproducible_Research/links/0deec5356feeec36e2000000.pdf), so there is some wider familiarity with R packages for a project structure already.

There are still a bunch of open questions on how best to use the R package structure (ropensci/unconf#11 https://github.com/ropensci/unconf/issues/11), so it might be more of an intermediate/advanced lesson for people already comfortable with R.

— Reply to this email directly or view it on GitHub https://github.com/swcarpentry/site/issues/806#issuecomment-75612256.

[rmflight]

@sritchie73 but R packages have all the pieces needed for a project, given functions (/R), metadata (DESCRIPTION), data (inst/data), short-form documentation (/man), and long-form documentation / analyses (/vignettes), and the ability to add whatever data / folder structure you want under /inst, that will be path agnostic after installing. I would argue that most actual analyses should be structured as vignettes or R-markdown files, because the computation should be embedded in the prose.

But, for the sake of this thread, I'll stop there.

[benmarwick]

ProjectTemplate looks interesting, but I think too complicated, labor intensive, and idiosyncratic to be a generic solution, as @rmflight details in his comparison of packages vs. ProjectTemplate. I agree that R packages are not ideal for bundling a research project, but they are probably the best current option, having the advantages of (1) being ubiquitous among R users, (2) having extensive online support and documentation, and (3) in wide use as research repositories already.

Here's another project structure template, this one is from Christopher Gandrud's book Reproducible Research with R and RStudio (very cool that the entire book can be reproduced!):

Perhaps the best thing we can do for the SWC novice learner is introduce the general principles of being organised in a research project with, at minimum, a readme plain text file and separate directories for code, data and output, then briefly survey some of the healthy diversity of options that exist for helping with this, and invite learners to find the one that suits them best.

[wking]

On Wed, Feb 25, 2015 at 10:53:23AM -0800, Ben Marwick wrote:

Perhaps the best thing we can do for the SWC novice learner is introduce the general principles of being organised in a research project with, at minimum, a readme plain text file and separate directories for code, data and output, then briefly survey some of the healthy diversity of options that exist for helping with this, and invite learners to find the one that suits them best.

For example, we could just expore the repository they downloaded for the workshop (with example filesystems, data, scripts, …). That's going to come with the usual README.md, LICENSE.md, tools/, data/, img/, …. We can hit a few of those, explain why they're useful, and then say that the exact organization chosen doesn't really matter, but that a README*, COPYING/LICENSE.md, and having some sort of organizing directory structure are almost universal conventions that are worth following.

[dfalster]

This is an important topic to cover, especially for new users who have files (input/output) and scripts all jumbled in together. @richfitz and I wrote a post on "Designing projects" on our blog, http://nicercode.github.io/blog/2013-04-05-projects/. While this stuff may seem elementary to some, it proved surprisingly popular.

[dfalster]

And here is a followup from one of student's showing how the layout of her project changed by following some simple advice, http://nicercode.github.io/blog/2013-05-17-organising-my-project/

Before:

After: