Missing 'term' dependencies for empty SRS

[balacij]

Creating a basic, empty, SRS is based on some chunks that exist and are shared amongst each of our examples. fillCDB is used to automatically inject these dependencies into each of the SystemInformations (and ChunkDBs) of each example. We should extend the chunks it injects to include everything needed to create a basic SRS document with all fields left empty (i.e., create an SRS with all sections/subsections, but make all the 'holes' of those sections/subsections empty).

To build BeamBending, I had to do something similar to this, and we should be able to base this solution on what I did (note that we should be inspecting each one carefully however :smile: to make sure they're all actually required -- I might have been overzealous).

Contributes to #3326

[harmanpreet-sagar]

@balacij I am working on modifying the text for the Assumptions category of the SRS so that it says: "There are no assumptions." instead of the definition that it currently provides. The following is the code that I have been trying to implement:

-- Wrappers for assumpIntro. Use assumpF' if genDefs is not needed
-- | Creates an Assumptions section by prepending a general introduction to other related 'Contents'.
assumpF :: [Contents] -> Section
assumpF []            = SRS.assumpt [assumpIntroNoCont] []
  where
    assumpIntroNoCont = emptySectSentence assumption
assumpF otherContents = SRS.assumpt (assumpIntro : otherContents) []
  where
      assumpIntro = mkParagraph $ foldlSent
        [S "This", phrase section_, S "simplifies the original", phrase problem,
        S "and helps in developing the", plural thModel, S "by filling in the",
        S "missing", phrase information, S "for the" +:+. phrase physicalSystem,
        atStartNP' (the assumption), S "refine the", phrase scope,
        S "by providing more detail"]

However, when I try to implement this, it is still printing out the definition. I think that it is because that there is still an element in the OtherContents list which is of the type UnlabelledContent which has EmptyS as its input for all the categories of the RawContent within the UnlabelledContent.

I am unfortunately unable to figure out where is this input coming from. You had asked me to look into SRSDecl for that and there I found out that the Assumptions is potentially taking in input from the conceptinsTable as can be witnessed in the following code block:

https://github.com/JacquesCarette/Drasil/blob/ad9a64f442cf76f2bd52534d9eff797a8f734564/code/drasil-docLang/lib/Drasil/DocDecl.hs#L136-L149

The conceptinsTable leads me to this table:

https://github.com/JacquesCarette/Drasil/blob/ad9a64f442cf76f2bd52534d9eff797a8f734564/code/drasil-database/lib/Database/Drasil/ChunkDB.hs#L203-L209

This is somewhat indicating to me that the input incoming is from the ci part of the cdb. However, in my template code, I do not have anything for the ConceptInstance other than progName. However, the template would not compile if I remove progName giving me the error which can be viewed below:

My code block:

symbMap :: ChunkDB
symbMap = cdb ([] :: [QuantityDict]) (nw progName : nw inValue : [nw errMsg, nw program] ++ map nw doccon ++ map nw doccon' ++ [nw algorithm] ++ map nw prodtcon ++ map nw mathcon) srsDomains
  ([] :: [UnitDefn]) ([] :: [DataDefinition]) ([] :: [InstanceModel])
  ([] :: [GenDefn]) ([] :: [TheoryModel]) ([] :: [ConceptInstance])
  ([] :: [Section]) ([] :: [LabelledContent]) ([] :: [Reference])

usedDB :: ChunkDB
usedDB = cdb ([] :: [QuantityDict]) ([] :: [IdeaDict]) ([] :: [ConceptChunk])
  ([] :: [UnitDefn]) ([] :: [DataDefinition]) ([] :: [InstanceModel])
  ([] :: [GenDefn]) ([] :: [TheoryModel]) ([] :: [ConceptInstance])
  ([] :: [Section]) ([] :: [LabelledContent]) ([] :: [Reference])

Error I get when I remove progName:

template.EXE: Term: progName not found in TermMap
CallStack (from HasCallStack):
  error, called at lib\Database\Drasil\ChunkDB.hs:116:24 in drasil-database-0.1.1.0-ETrIGG9tlCdGWPiRbL6iDv:Database.Drasil.ChunkDB
make: *** [Makefile:270: template_gen] Error 1

Could you please advise as to what I should potentially do or explore in order to combat this issue?

[harmanpreet-sagar]

@balacij Another quick question. Since we are generating all the sections of the SRS in the template now, do we generate the base tables for sections such as Table of Abbreviations and Acronyms, Table of Symbols, etc. even though we do not have any content within them?

[smiths]

@harmanpreet-sagar I like the idea of having the section headings generated, even if there is no content, but the empty tables aren't necessary. Even the blurb about SI units isn't meaningful if there are no units. I think you just want the section heading and a simple statement like your "There are no symbols" statement.

[harmanpreet-sagar]

Ok perfect! Thank you for the clarification! I will do that then.

[balacij]

Great analysis, @harmanpreet-sagar ! Did you also have a look at how assumpF is used?

Using ripgrep (but grep will do just fine), I found:

λ rg "assumpF" -ths      
...
drasil-docLang/lib/Drasil/DocumentLanguage.hs
44:import qualified Drasil.Sections.SpecificSystemDescription as SSD (assumpF,
385:      SSD.assumpF $ mkEnumSimpleD $ map (`SSD.helperCI` si') ci
...

And then looking at the definition of mkEnumSimpleD, we find:

-- | Convenience function for transforming referable concepts into a bulleted list.
-- Used in drasil-docLang in making the assumptions, goals, and requirements sections.
-- Output is of the kind @Concept Name: definition of concept@.
mkEnumSimpleD :: (Referable c, HasShortName c, Definition c) => [c] -> [Contents]
mkEnumSimpleD = mkEnumSimple $ mkListTuple (\x -> Flat $ x ^. defn)

-- | Helper function for converting a list of something into a bulleted list.
-- Used in 'mkEnumSimpleD'.
mkEnumSimple :: (a -> ListTuple) -> [a] -> [Contents]
mkEnumSimple f = replicate 1 . UlC . ulcc . Enumeration . Simple . map f

So, I think it aligns what you said, almost exactly? I think we will need to make sure that mkEnumSimpleD only creates a list if a list actually exists, I think it is creating an empty list right now.

[harmanpreet-sagar]

Oh ok! Thank you for the clarification! That aspect of the issue has now been fixed.

[harmanpreet-sagar]

While solving this issue and creating the empty sections, I noticed that we have many areas where there is a list containing empty lists, which does cause issues when compiling because the code comprehends a list containing empty lists differently than an empty list. This may potentially cause further issues in other aspects as well and I believe that it should be inspected to prevent any breakages and issues that may be caused due to this. I noticed this issue mainly in the FunctionalRequirements and the Assumptions sections first. Should I create an issue regarding this?

[balacij]

Creating an issue sounds like a good idea, good catch @harmanpreet-sagar ! Is this ticket then blocked by those issues?

[harmanpreet-sagar]

Ok, I will create an issue for those. I don't think that this ticket should be blocked by those as I fixed them for those two.

[harmanpreet-sagar]

Just to clarify @balacij, before moving forward, I am supposed to create functions in the fillcdbSRS like the ones currently present for the References section or am I misunderstanding my task? The function that I am referring to can be viewed below:

https://github.com/JacquesCarette/Drasil/blob/ad9a64f442cf76f2bd52534d9eff797a8f734564/code/drasil-docLang/lib/Drasil/DocumentLanguage.hs#L80-L81

https://github.com/JacquesCarette/Drasil/blob/ad9a64f442cf76f2bd52534d9eff797a8f734564/code/drasil-docLang/lib/Drasil/DocumentLanguage.hs#L123-L221

[balacij]

Hmm, seeing that code, I might have misunderstood it the first time around. It seems to just calculate some things and add them into the ChunkDB. So, we might need to re-investigate the larger issue. The issue is that the template project does not build with non-trivial SRS document section declarations because the printer depends on specific chunks which aren't hard-required in the user-provided ChunkDB. We want those chunk requirements to be inputted into the ChunkDB before we run the transformers/printers, somehow. The question is: where does the required information come from, and how do we want them imported?

Old line of thinking (kept for the sake of it, poorly formatted due to GitHub's Markdown renderer, see "edit comment" instead)

When I was responding to your comment, I almost finished writing up my response, but then I realized my continuation after (1) was flawed. My line of thinking: 1. Should the `ChunkDB` always include the dependencies? If Drasil were strictly limited to generating SRS documents, then I would say this isn't a bad idea, but I don't think we necessarily want that. We should probably avoid this. 2. How should the chunks that the SRS printer and the code generator rely on be entered? We have some options: 1. Automatically, via `fillcdbSRS`: initially, I thought this was the solution, but I'm not sure if I'm happy with it anymore, because of 2 issues: 1) it's hinting that we are allowed to assume some chunks exist without asserting that they do (and without a traceable deference "hole"), and 2) it's still manual because `fillcdbSRS` is not hard-required by each project. 2. Manually 1. as it is: well, it's a goose chase to figure out which chunks to add and which ones to not add, so I don't think this is a good idea. 2. with a basic `ChunkDB` that each example is based on and adds to: not bad, but it's a flimsy solution because it still has to be manually done while the syntax we import (at the Haskell level) for describing the project problems is not, so there is a kind of clash between things that are forcibly required (e.g., `Expr`) and those that are not (e.g., certain chunks that are referred to by the SRS LaTeX transformer/printer).

---

I wrote up this post before thinking about this enough. I'm largely away today, but I'll try to come up with a better response tonight/tomorrow. Originally, I guess this ticket ignored the problem, pushing the solution down the road. It might be the case that having users start off with a basic chunk database (and appending to it) that already contains all the acronyms and such is a better solution than that proposed in the original post.

[harmanpreet-sagar]

@balacij I am confused as to whether I should proceed with this ticket or not and if I were to proceed, which solution should I explore, either out of the ones listed above or any new that you think might work?

[balacij]

I think 2.ii.b from my post is the best solution for now, but I've been a bit busy and haven't been able to return to this yet (sorry!). We can think of the chunks as being part of the standard library imported for each example, which should have been imported at the same time that we imported things at the Haskell level. @JacquesCarette Do you have any immediate thoughts on this?

[JacquesCarette]

Commenting starting at:

I noticed that we have many areas where there is a list containing empty lists, which does cause issues when compiling because the code comprehends a list containing empty lists differently than an empty list.

BTW, the use of 'compiling' here is misleading rather than helpful.

First and foremost: an empty list and a list containing an empty list are very different. So they should be processed differently.

More deeply: this is a symptom of abusing lists as the "good for everything" datatype. I'll comment more on that on the issue itself.

@balacij said:

I think 2.ii.b from my post is the best solution for now

I can't really give an informed opinion. I'm sure there is a real issue, but I have not been able to infer it from an (admittedly quick) reading of the (long!) thread. So I'd appreciate a concise summary of what is the blocking factor here.

[balacij]

Yep, the issue is that the SRS document abstraction/printers rely on some specific chunks. Those chunks need to be accessible by the printers, so they need to be added to the chunk database at some point.

Since they're tied to the document language or the printers, I figure we probably want them to be automatically added to the chunk database. However, since the document language and printer usage is done at the Haskell level, we can't quite have them easily & automatically added to the chunk database.

Since all the case studies are similar enough, I think our best bet is to have all the projects base their chunk database on the chunk database that merges the chunk databases necessary for the document language / printers (I don't remember exactly where the chunks appeared now, but if it was just one of them, then it's just the one).

[JacquesCarette]

Feels like there should be some kind of init process for the various processors that would load up the chunk database with their own "stuff".

A good process would have the init process return an abstract token once complete, and the processor would need to be given a value of that type to proceed. [That's easy to implement via a fresh unit type that doesn't export its constructor, and the processor being strict in the token.]

That way you don't have to init the processors you don't use, and so not pollute the database you use with extra stuff.