Notebook structure

[tingyuw]

Currently, the language of document in Drasil is SRS-based, the sections and theories are created for the SRS. Most of the languages and data objects are reusable, but the notebook structure is kind of ambiguous at this moment. As far as I know, the notebook is a new type of document which includes executable code. Does (or should) the notebook has a standard format? What would the future application of the notebooks look like?

The SRS is built in the order of the sections, and the contents are added separately in different modules (e.g., Assumptions.hs for assumptions and GenDefs.hs for general definitions). I think it's okay to not have a fixed structure for the notebook, but the point is knowing the order of the contents. Those sections in SRS are basically just 'names' for content units, we could simply divide the notebook into introduction, body, summary, and example (based on the projectile lesson) if we want to, although it is not the best solution.

Not quite sure if this is the first step to do, however, it seems like we need to understand the constructor of the notebook first then go into other details like theories and concepts.

[JacquesCarette]

Yes, I think your analysis is correct. We need to understand the structure of notebooks first.

My guess is some notebooks will be very structured, while others will be quite free. So the language will have to reflect, and allow for both kinds of content.

Luckily, notebooks are still linear - so a tree of content (like a Rose tree with a list of subtrees) probably works, to deal with both some nested structure and some long series of cells.

[smiths]

As a starting point, would it make sense to generate the Projectile SRS as a Notebook? @tingyuw has previously done this, but as essentially one big markdown cell. Maybe we could redo this, but with each section a separate cell?

The next step would be to add code cells. We know the inputs (they are part of the spec). For each input we could have a single code cell that defines it. The initial value can be taken from the table of typical values. The calculations cell would just be the calculation code that we already generate, just redirected to a Jupyter cell. The output would just be some cells with print statements at the end.

[tingyuw]

As we discussed earlier, we could use a general structure for now and improve it gradually. The SRS is divided into several sections which are the constructors of it. Any suggestion for how each unit/section of notebook contents should be? Maybe something like a paper structure?

@smiths Separating cells for SRS was supposed to be the next step, but I noticed that since we are not sure about how we are going to input the notebook, which might affect the way we handle the contents (even for SRS in jupyter), it might be better if we deal with it after we understand the language of the notebook in Drasil. And yes, adding code inputs would be the next thing to do after adding non-code contents.

[smiths]

@tingyuw I think we can think of the cells as units of information. They will be easier to reuse in different contexts if the blocks of information are small. We also need a way forward that is practical for you. I wonder if you can make a mapping between "units of information" and paragraphs? A well written paragraph (most of our paragraphs are fairly well written) usually starts with the first sentence as the topic sentence. Maybe you can make a connection between paragraphs as units of information and cells?

[tingyuw]

I think the mapping happens in the printer, so does separating cells. We only need to consider how to input the information in Drasil at this point. And I think we can divide the notebook application into different categories and create a language for each since we only need to make some minor changes. But we can focus on the projectile lesson for now (if it is the example that we want to start with), otherwise, the structure is confusing.

What we need now is the concepts and how big each section is to build the example.

In SRS, we have several standard sections that are the constructors, and those Doc.Introduction, Doc.prpsOfDoc, etc., are concepts.

What concepts do we need for the projectile lesson? Should we add specific concepts for the lesson or just something general? And what are the "standard sections" of the projectile lesson? I tried to add some basic data of the notebook, but still need these information to proceed.

[JacquesCarette]

A mapping happens in the printer right now. That doesn't mean that there could not be another layer of mapping that happens in between. For the rest, I agree.

As to your last 3 questions: this is part of your work to figure out the answer to exactly those questions!

[tingyuw]

I revisit this issue because I'm updating the projectile lesson example in Drasil since @smiths made some changes to it. The structure we have for notebook now is a mix of general and specific. I tried to make the main structure general but the actual sections (constructors to build the notebook) are specific for the example.

data NbSection = IntrodSec NB.IntrodSec
                | LearnObj NB.LearnObj
                | BodySec NB.BodySec
                | SmmrySec NB.SmmrySec
                | BibSec
                | ApndxSec NB.ApndxSec

One thing worth noticing about this approach is that the title of each section will be different from the actual section in the lesson, it is more general like Body. For example:

I guess my questions are 1. how general or specific we want the structure to be? It's not hard to build the section in the exact title (e.g., Rectilinear Kinematics: Continuous Motion (Recap)) but it might not be something we are looking for. And 2. if it's okay if there is difference between the generated document and the one Dr. Smith created. The situation I was thinking is the steps in the Procedure for Analysis section. Those steps (Step 1-5) are headers but I don't think we can customize the header or add one without making it a section in the current Drasil (or can we?) We can bold them but not as a header.

If it's okay to specialize the structure, we can generate the document as similar to the example, or if we want to keep it as general as possible at this point.

EDIT: Actually, it just occurred to me a few days ago that since we are generating the Jupyter notebook in a different way from the SRS documents (different structures), why not just make the notebook structure linear and keep the SRS nested? Unless we are printing the jupyter-based documents in HTML (with a mix of text and code, not sure if we are doing that) in the future.

[JacquesCarette]

Issue: Notebook Structure

[smiths]

@tingyuw we touched on your questions during #3202. We do want to keep the structure as general as possible. We don't want to tie it to the projectile example. Instead, we want to think of the Projectile example as an instance of a Lesson. We want to think of what we are doing as generating a family of lessons. The projectile lesson will have commonalities with other lessons and it will have variabilities. As much as possible we want to identify the parameters of variation for the variabilities. I mentioned during the meeting that I started an analysis along these lines a while back. You can find a brief discussion and a pointer to a longer document in an issue in the Case Studies repo.

As far as having to reproduce the exact formatting of my original Jupyter example, you definitely don't have to do that. You want to indicate structure (like a section divided into subsections) somehow, but it doesn't have to follow the specific way I chose to do it.

[tingyuw]

I've a look over the document of the lesson categories. That's a really useful information! I do believe for notebook applications we need to generalize the structure like the SRS. @smiths Do you think it makes sense if we categorize the projectile lesson chapters to the following:

• Learning Objectives
• Review - Rectilinear Kinematics: Continuous Motion (Recap)
• Case Problem - Motion of a Projectile to Procedure for Analysis (?)
• Example

[JacquesCarette]

I'm not @smiths , but this does make sense to me.

[smiths]

Yes, this makes sense to me too @tingyuw. As we discussed briefly in #3232, I'm not sure how we generalize Example. We do have an example, but every problem is going to have an example that looks differently. Let's start with learning objectives, since that is something that all lessons should have.
We can think further about the organization of an example once we see what the easier pieces look like.

[tingyuw]

The updated generated lesson can be found here: Projectile Lesson Still need improvements on how we input the receipts and the detail chapter structure. For example, I kind of "hard coded" the bolded headers because we aren't able to customize them right now. Not sure if the ability for creating customize headers will be needed in the lesson (users can create a header and decide the size which is not the header of a general chapter), I hard coded now just to make it easier to read but it won't affect the high level structure of the lesson.

Noted: The Example chapter will be updated slowly.

[smiths]

Thank you for the update @tingyuw. The lesson looks good for the most part. There are some parts that are formatted awkwardly and some of the equations didn't compile properly, but overall it looks good.

I wonder if the generalization should include a generalization of steps. Maybe you already did this? Many lessons will include step-by-step instructions.

[balacij]

@tingyuw Is there anything that needs to be done further to close this ticket?

[tingyuw]

I think we're ready to close this for now. We have the first version of the structure, and we can create new issues once we have a better understanding of the notebook structure and want to make changes.

I wonder if the generalization should include a generalization of steps.

Also, I think it is a good idea to include a generalization of steps. I've already included this in my report as a future plan, where we will design the content types for lesson plans. @smiths Do you think it's necessary to create a new issue for this now?

[smiths]

@tingyuw I agree that we can close this issue. I agree that we should generalizes the steps in the future, but we don't need to create an issue right now. We don't want to create such a big issue because it will stick around for potentially a long time. We have your MEng report to remind us of the need for further generalization. :smile: