usfm_sb.sty - ip should not be valid in the middle of a chapter of Scripture

[tjcouch-sil]

Relevant Concepts

• You can add an ip under id to add introductory information to a book before a chapter starts
• In usfm_sb.sty, you can add ip under c (but not in usfm.sty)
  • Our understanding is that the reason to allow ip under c is that it allows you to put ip at the end of a chapter to add introductory material for the following chapter
• In usfm_sb.sty, you can add ps (and 13 other Ranked markers like m, qa, b, etc.) under ip, likely to allow adding more complex introductory material at the end of a chapter (but not in usfm.sty)
  • This is significant because p is ranked and its rank is equal to or less than ip's rank. In usfm.sty, there are no markers that have rank and can occur under ip. You can use the regex string \\OccursUnder.*ip.*\n\\Rank to investigate
• You cannot add q (and a number of other Rank 4 markers like q3, sp, d, etc.) under an ip but only under c in either usft.sty or usfm_sb.sty, maybe to indicate that you are not supposed to put an ip in the middle of a chapter as not everything that can be in a chapter is allowed under ip

Problems

• There is no way to enforce that ip only occurs after id and before a chapter or at the end of a chapter, so you can put an ip in the middle of a chapter if you want
  • Resulting issue: some ranked markers are not allowed under ip, so if those markers occur within the chapter after you put an ip, those markers are flagged invalid. However, the issue in this case is really with the ip, not with the subsequent ranked markers, so we should see the ip flagged invalid instead
• If we changed the concept that ip can only be at the end of a chapter and thought about it so an ip can occur within a chapter (since this is currently possible), there is no way to end an ip
  • Resulting issue: if you put an ip in the middle of a chapter, the subsequent Scripture text will all incorrectly be under the ip instead of outside of it

Problems Visualized

You can add p and q under c:

• c (Rank 8)
  • p (Rank 4)
  • q (Rank 4)

But the OccursUnder differences showed above indicate that, in usfm_sb.sty, you can add one or more ps under an ip under a c. Since you can add p under ip, p checks to make sure its rank is equal to or lower than the first marker under ip which is either nothing or another p. Since p's Rank 4 is equal to p's Rank 4, you can add another p under ip after a p here:

• c (Rank 8)
  • ip (Rank 6)
    • p (Rank 4)
    • p (Rank 4) < - adds successfully since it has the same rank as the first marker under its parent

However, in usfm_sb.sty, you cannot add a q under an ip like you can add a p under an ip. Since you cannot add q under ip, q tries to add itself under c. q checks to make sure its rank is equal to or lower than the first marker under c which is ip. Since ip's Rank 6 is greater than q's Rank 4, you cannot add a q under c here. This document is invalid:

• c (Rank 8)
  • ip (Rank 6)
    • p (Rank 4)
  • q (Rank 4) X - tries to add under c and fails because its Rank is lower than 6

In the following screenshot from Paratext, on the left is a base Scripture text using usfm.sty, and on the right is a test Scripture Study Bible using usfm_sb.sty containing an added ip within a chapter. Adding ip within the chapter is valid with current usfm_sb.sty rules. However, it caused the subsequent ps from the base Scripture text to be under the ip and the subsequent qs to be invalid because they cannot go under the ip since ip is not in q's OccursUnder or close the ip since they have a lower rank:

Possible solutions

• make p and other ranked markers not able to occur under ip, which improperly removes the ability to add complex introductory material for a subsequent chapter at the end of a chapter
• make q and maybe all other ranked markers less than 6 able to occur under ip, which unfortunately enables the problem of actual Scripture text being incorrectly under ip
• add some system such that q and other disallowed ranked markers would suggest looking to see if they're under an ip, which doesn't really make sense considering the problem is the ip and not ranked markers that do not opt into being able to occur under ip
• somehow restrict ip so it can't be in the middle of a chapter so that ip can be appropriately deemed invalid under certain conditions
  • We would probably need some additional stylesheet property
  • We would need some conditions to determine if the ip is not in an allowed position
• Do nothing as this may be a limitation of SFM format

[klassenjm]

Hello @tjcouch-sil ,

I've been reviewing this report. Before discussing further, I would like to point to the USFM documentation - specifically concerning USFM markup for 'Extended Study Bible Content'. You will see that \ip is intended to be supported throughout the scripture body text area for the purpose of adding division or section introductions.

The Paratext stylesheet has limitations in terms of how well it can be used to test the position and sequence of markers. The differences you see between usfm.sty and usfm_sb.sty are the effort to support the additional \e.. markers for study Bible content, and to allow more flexibility for some markers like \ip, and additional columns for tables, etc.

[tjcouch-sil]

Ok, thank you for clarifying! It is helpful to know that usfm_sb.sty does intend to allow ip anywhere in a chapter. That helps me to understand the issue at hand a little better, and that feels consistent with the overall styles design.

I do still believe we have a hierarchy problem of some sort. It seems important for complex content including various markers like p to be able to be under ips. However, this issue is still occurring where an ip is absorbing the rest of the content in the chapter after it and invalidating any ranked markers under it that are not allowed in an ip but are allowed in a chapter (see my screenshot of Paratext in the issue report where the p markers are valid but the q markers are not).

It seems like the style standard may need to have a way to designate when ip should end and the Scripture begin again. I suppose a couple of suggestions within existing properties come to mind:

• Lower the Rank of ip to 4 to match p and q and other things so that more markers can follow ip and end the ip instead of coming under it or being invalid. Then I suppose some or all current Rank 6 and 5 markers (there are 30 of them, I believe) would need to be listed in ip's OccursUnder so that we don't invalidate ip anywhere. That would mean ip would be under those markers above it, though, which would change how it works
• As suggested above, add ip in the OccursUnder list of other Ranked markers less than 6 that could be Scripture like q so that all Scripture can be under an ip. That would of course mean any Scripture can be under an ip in the hierarchy, which does not seem right

Do you have any thoughts on how to move forward or how I can understand better what the style is supposed to indicate in this situation?

Should I change the title of the issue report to describe the real issue more accurately? Maybe something like "usfm_sb.sty - ip doesn't end in the middle of a chapter once Scripture begins again" or something.

Thank you for your time!

[klassenjm]

In a study Bible, \ip can exist anywhere within a chapter, but usually would be added after a section heading.

In the usfm_sb.sty stylesheet, I propose that we remove \Rank from \ip.

[tjcouch-sil]

@klassenjm Thank you very much for looking into this and thinking carefully about these issues! Yes, it does appear that removing rank from ip works well to allow ip anywhere in a chapter. I appreciate your thoughtfulness regarding this clean and simple solution.