Feedback on Current Paper Outline

[smiths]

The following comes to mind while reading v3 of the outline:

• it would be nice to have references to support the statement that "[documentation] is not often prioritized on software projects."
• @JacquesCarette recently did a presentation on Drasil that you should watch (one of us will send you the link)
• @JacquesCarette and I have been working on a white paper that has some relevant material in the current draft (https://github.com/JacquesCarette/Drasil/blob/whitePaperOnDrasil/Papers/WP_GenerateEverything/GenerateEverything.tex)
• the first paragraph combines two ideas. We could split them into two paragraphs:
  • documentation is valuable, but not prioritized (see Parnas2010)
  • code and other artifacts are different views of the same information
• I like the idea of the figure that shows multiple views of the same information - if you do a good job on this figure, the text will be very easy to write for this point
• After discussing documentation generators, you could contrast this approach with literate programming. In the way you are phrasing things, we switch from just writing code (and generating the documentation) to just writing documentation (and generating the code). The problem here is that again we have multiple views of the same information that have to be manually written. Artifacts other than the code and a description of the code are still ignored. Literate programming isn't used to write requirements documentation.
• I like the "We propose a new framework ..." paragraph, but maybe we should say something more about "knowledge-centric". In particular, the reuse of knowledge in different views requires being able to capture the knowledge. It requires a well-understood domain. I think you should make the point here that building a systematic framework for generating artifacts is only possible for a well understood domain. You can then return to this point in the scope section
• Under scope, I think you should explain that scientific computing is a well-understood domain - you already say this, but you didn't previously set up the fact that a knowledge-centric/generate all things approach requires this
• Under scope, you can mention that we are aiming for the generation of artifacts to help developers with all problems, including mundane problems like renaming variables, writing README files, building traceability graphs, etc.
• I wonder if we should call the approach a "generate all things" approach, instead of knowledge-centric? We run into trouble with other domain experts when we say knowledge. It is such a loaded term. Naming our approach by what we do, as opposed to characterizing it with some generic wording, might help us avoid criticism?
• I like the term "growing Drasil" :smile:
• You are probably already planning this, but rather than have the case studies in appendices, you can use links to the existing documents
• I like the idea of summarizing the case studies, since you will be using them throughout your document
• For the case studies, you can point to the SmithJegatheesanAndKelly2016 (https://gitlab.cas.mcmaster.ca/smiths/pub) paper to explain that they are "real" projects. We took GlassBR, SSP (and NoPCM) and GamePhys from this. Projectile was added because we wanted a simple example.
• You don't have to say that Projectile didn't inform the development of Drasil, if "faking" a rational development process is an easier way to express your ideas

I stopped my review at --Breaking down artifacts--

[smiths]

Picking up the review at --Breaking down artifacts--

• The figure showing the ref section for at 3 of the case studies will be challenging to create, especially if we want to keep it legible. I think it can be done though, since you don't need all of the details. You can have a few samples and use ... (or equivalent) to hide the rest. It probably would make sense to make this figure before writing more about it, just to make sure that it is feasible. :-)
• I agree with the summary of the information provided about the reference section, but it seems to be missing the biggest selling feature - hardly any (none?) of this information is written by a human; it is automatically generated.
• I think you already know this, but the Results section is rather sparse. :-)
• I like the idea of a results section. If we write the document like there were some problems that we wanted to solve, we can then revisit how well we solved them. Maybe we need a more explicit list of the problems we are intending to address via Drasil?
• I find that after reading the summary I want to understand better how the story is going to be told. There is good information in the Outline document, but it doesn't really read like an outline. Some sections are more fleshed out than others. It would be nice to have a true outline that isn't fleshed out at all. What are the proposed pieces for the thesis? What are the chapters? What are the subsections in the chapters? I think this would help with understanding the scope and the story.