Support Verse Bridges/Spans for wA tool

[benjore]

User Story

As an aligner of the UST, I want the wA tool to support verse bridges so that I can complete all the alignments.

Specs

• [ ] Show verse bridges
  • [ ] in the side menu
  • [ ] show the entire target language verse span in the word bank
  • [ ] show both/all verses in the Original Language
  • [ ] Scripture Pane

To Consider

• USFM Imports
• USFM Exports
• CSV Exports
• Verse Edit
• Data Structure

Per @klappy on #4543

Importance of ContextID being fixed

Context ID comes from the checking tool itself and based on the Greek versification not the target verse. So the context ID itself wouldn't include the verse bridge information. Context ID's are like direct identifiers of where in the app, checking tool, check category/group, selected menu item that the user was in when the verse was edited.

ContextID in Tools and CheckData

Any ContextID in a menu, tW tool, wA tool, edits, bookmarks, selections, etc... uses the below path structure of the verse bridges and specific verse in the reference of the context id unchanged.

We should always open the verse bridge by knowing the specific context ID. The context ID shows the edit was verse 1, even if the target language had a verse bridge of "1-2" we could know that it is the "verse" referenced.

Verse Bridge used in Paths

Due to the target language being a verse span, we would likely store the edit in a path that uses the verse bridge such as: ./.apps/translationCore/checkData/verseEdits/tit/1/1-2/timestamp.json

Verse Edit Example (unchanged):

{
  "contextId": {
    "reference": {
      "bookId": "tit",
      "chapter": 1,
      "verse": 1
    },
    "tool": "translationWords",
    "groupId": "apostle",
    "quote": "apostle, apostles, apostleship",
    "occurrence": 1
  },
  "verseBefore": "Paul, a servant of God and an apostle of Jesus Christ, for the faith of God's chosen people and the knowledge of the truth that agrees with godliness,",
  "verseAfter": "abcabc abcabc abcabc",
  "tags": [
    "other"
  ],
  "userName": "klappy",
  "modifiedTimestamp": "2018-01-12T22:38:09.844Z"
}

Since the JSON files's payload doesn't change any, but just the path structure's "reference" that includes the verse bridge, the application needs to adjust understanding that the verse bridge is an integer range that covers the verses between and including the first and end integer in the bridge. If it were "7-9", it would include 7, 8 and 9. If it were "1-10" it would be 1, 2, 3, 4, 5. 6. 7, 8, 9 and 10 inclusively.

Target Bible Structure

The folder structure of the target language Bible data would be much similar to the above. However, since the files are at chapter level, and the keys of the object in the json file are the verse numbers. The verse number keys are strings and allow for them to be a verse bridge. This would include directly from importing and directly leveraged when exporting. I believe the USFM library already supports this. @PhotoNomad0 Can you help confirm this? I know we discussed this while addressing as many use cases as we could in the USFM.js library.

Importance of ContextID being fixed

Context ID comes from the checking tool itself and based on the Greek versification not the target verse. So the context ID itself wouldn't include the verse bridge information. Context ID's are like direct identifiers of where in the app, checking tool, check category/group, selected menu item that the user was in when the verse was edited.

ContextID in Tools and CheckData

Any ContextID in a menu, tW tool, wA tool, edits, bookmarks, selections, etc... uses the below path structure of the verse bridges and specific verse in the reference of the context id unchanged.

We should always open the verse bridge by knowing the specific context ID. The context ID shows the edit was verse 1, even if the target language had a verse bridge of "1-2" we could know that it is the "verse" referenced.

Verse Bridge used in Paths

Due to the target language being a verse span, we would likely store the edit in a path that uses the verse bridge such as: ./.apps/translationCore/checkData/verseEdits/tit/1/1-2/timestamp.json

Verse Edit Example (unchanged):

{
  "contextId": {
    "reference": {
      "bookId": "tit",
      "chapter": 1,
      "verse": 1
    },
    "tool": "translationWords",
    "groupId": "apostle",
    "quote": "apostle, apostles, apostleship",
    "occurrence": 1
  },
  "verseBefore": "Paul, a servant of God and an apostle of Jesus Christ, for the faith of God's chosen people and the knowledge of the truth that agrees with godliness,",
  "verseAfter": "abcabc abcabc abcabc",
  "tags": [
    "other"
  ],
  "userName": "klappy",
  "modifiedTimestamp": "2018-01-12T22:38:09.844Z"
}

Since the JSON files's payload doesn't change any, but just the path structure's "reference" that includes the verse bridge, the application needs to adjust understanding that the verse bridge is an integer range that covers the verses between and including the first and end integer in the bridge. If it were "7-9", it would include 7, 8 and 9. If it were "1-10" it would be 1, 2, 3, 4, 5. 6. 7, 8, 9 and 10 inclusively.

Target Bible Structure

The folder structure of the target language Bible data would be much similar to the above. However, since the files are at chapter level, and the keys of the object in the json file are the verse numbers. The verse number keys are strings and allow for them to be a verse bridge. This would include directly from importing and directly leveraged when exporting. I believe the USFM library already supports this. @PhotoNomad0 Can you help confirm this? I know we discussed this while addressing as many use cases as we could in the USFM.js library.

[benjore]

Suggested SPIKES:

• [ ] (4) Context ID
  • [ ] Impact on tW tool
  • [ ] Impact Verse Edit
  • [ ] Impact on building target Bible data structure
• [ ] (3) How to handle verse bridges in Scripture Pane
  • [ ] impact on Scripture Pane (some Bibles with bridges others without)
• [ ] (4) Data Structure for wA Tool
  • [ ] how we save things
  • [ ] checking for changes
  • [ ] Impact on USFM.js
  • [ ] impact on USFM export with alignments
  • [ ] impact on USFM import overwrite merge
  • [ ] do not create empty verses for verse spans

[benjore]

NEED: a couple projects with Verse Bridges (USFM + tStudio)

https://git.door43.org/unfoldingWord/en_ust/src/branch/master/55-1TI.usfm, https://git.door43.org/unfoldingWord/en_ust/src/branch/master/50-EPH.usfm, https://git.door43.org/unfoldingWord/en_ust/src/branch/master/43-LUK.usfm

[benjore]

Related to #3372

[da1nerd]

@klappy @benjore we need to figure how to indicate that a word comes from a verse span. @klappy you and I have discussed possibilities for words coming from a different verse, but coming from a span brings in a whole bunch of complexity.

Or could we just keep things simple and reference the first verse in the span?

[mannycolon]

SPIKE Results

• Add verse bridges support to project import
  • Add verse bridges support to code that checks for missing verses
  • We currently generate empty verses for verses within a verse-bridge (We should no longer create empty verses for verse-bridges)
  • Verse bridges should not show up as missing verses
  • Below is how we are parsing Ephesians 5 when importing it in tC

    {
    "1": "Because of what God has done for you, imitate him, just as children imitate the father who loves them dearly. ",
    "2": "Do everything in a way that shows that you love others. Be like the Messiah, who loved us so much that he willingly died for us on the cross as the offering and sacrifice to God in our place. This sacrifice pleased God very much.\n\n\\s5 ",
    "3": "But there should be no reason for anyone to even suggest that any of you are involved in sexual sin or any kind of immoral or obsessive sexual behavior. Such sins do not belong among God's people. ",
    "4": "When you speak to each other, do not tell obscene stories or say foolish things or joke about committing sins. Such things are not what people who belong to God talk about. Instead, express the things that you are thankful for.\n\n\\s5 ",
    "5": "It is quite true that these people will be excluded from the kingdom of the Messiah who is God: Everyone who is sexually immoral or indecent, or who is obsessed with sex, which is the same as worshiping an idol. ",
    "6": "Do not let anyone deceive you by telling you that we should accept these practices. It is because of these very things that God punishes the people who disobey him.\n\n\\p ",
    "7": "So do not join with those people in doing these kinds of sins.\n\\s5 ",
    "8": " Remember that before you believed in the Lord Jesus, you did not know what was true, just as people who are in a dark place do not know what is around them. But now it is as though you have come out into the light, because the Lord has shown to you what is true. So live in the way that the Lord has shown to you. ",
    "9": "Because just as people who have light will walk in the right way, as a result of knowing Jesus you can always live in a way that is good, right, and true. ",
    "10": "As you live this way, keep learning what pleases the Lord. ",
    "11": "So do not participate together with those who are doing the worthless deeds that they do in spiritual darkness. Instead, expose to everyone how worthless those deeds are. ",
    "12": "Of course, it is shameful for God's people to even talk about the evil things that people do in secret,\n\\s5 ",
    "13": "but it is necessary for us to expose them so that people can know and understand that these deeds are evil. This is like when we bring something into the light in order to reveal to everyone what it truly is. Then people can examine and judge that thing that light has exposed. ",
    "14": "This is what believers are talking about when they say,\n\\q \"You who are asleep, Wake up!\n\\q You who are dead, come out of the darkness and live!\n\\q The Messiah will show you what is true, \n\\q just as a light that shines shows people what was in the darkness.\"\n\\s5\n\\p ",
    "15": "So be very careful how you live. Do not behave as foolish people do. Instead, behave as wise people do. ",
    "16": "Do the most good that you can with the time that you have, because people are doing more and more evil things every day. ",
    "17": "So be wise, understand well what it is that the Lord Jesus wants you to do, and do it!\n\\s5\n\\p ",
    "18": "Do not become drunk by drinking alcoholic drinks, because people cannot control themselves when they are drunk. Instead, let God's Spirit control what you do at all times. ",
    "19": "Sing to each other Psalms, and songs about the Messiah, and songs that God's Spirit gives you. Even when you are not singing out loud, sing and offer music to the Lord within yourself. ",
    "20": "At all times thank God the Father for everything because of what our Lord Jesus the Messiah has done for you. ",
    "21": "Humbly submit yourselves to each other because you respect the Messiah.\n\\s5\n\\p ",
    "22": "",
    "23": "",
    "24": "As for wives, just as all believers submit themselves to the authority of the Messiah, in the same way wives must also submit themselves completely to the authority of their husbands.\n\\s5\n\\p ",
    "25": "Each of you husbands, love your wife as much as the Messiah loved all who would believe in him. He even gave up his own life for us on the cross, ",
    "26": "so that he might set us apart for himself. By speaking his message to us, Jesus purified us. He removed our sins from us, just as people make things clean by washing them with water. ",
    "27": "He did this so that he could present the group of all believers to himself as a glorious group that is completely clean and perfect, without sin or any fault, like a glorious bride ready to meet her groom. \n\\s5 ",
    "28": "In that same way each man should love his own wife as much as he loves his own body. A man who loves his wife is also, by doing that, loving himself, ",
    "29": "",
    "30": "",
    "31": "The scriptures say this about people who marry:\n\\q \"Therefore a man will leave his father and his mother and will join himself to his wife, and the two of them will become as though they were one person.\"\n\\p ",
    "32": "There is very much about this that we cannot understand, but I am telling you that this example of a husband and wife also helps us to understand some things about how much the Messiah loves his worldwide assembly of believers. ",
    "33": "However, as for you, each man must love his wife just as he loves himself, and each woman must deeply respect her husband.\n\n\\s5",
    "front": "\\p ",
    "22-23": "Wives should submit to their own husband's leadership as they do to the Lord Jesus, because the husband is the leader of the wife as also the Messiah is the leader of the worldwide assembly of believers. He is the Savior who has saved all believers from being condemned for their sins. ",
    "29-30": "because no one ever hated his own body. Instead, he feeds his own body and cares for it, just like the Messiah also cares for all us believers in his worldwide assembly. We have become one group of believers that belongs to him.\n\\s5 "
    }

• Make resources reducers support verse bridges
  • Verse edit should support verse bridges
  • Due to the target language being a verse span, we would likely store the edit in a path that uses the verse bridge such as: ./.apps/translationCore/checkData/verseEdits/tit/1/1-2/2019-04-01T18_23_11.167Z.json
• Add verse bridges support to Scripture pane
  • Verses with verse bridges should open with no problems
• Add verse bridges support to usfm-js (Is it already implemented? should we then verify it works?)
• Add verse bridges support to WA tool data structure
  • See #4545 for more info

[mannycolon]

@benjore could we also get a few tStudio projects with verse bridges?

[benjore]

@cckozie Do you have any of these ^^^?

[cckozie]

@benjore @mannycolon - I don't believe tS supports verse bridges.

[mannycolon]

@klappy @benjore we need to figure how to indicate that a word comes from a verse span. @klappy you and I have discussed possibilities for words coming from a different verse, but coming from a span brings in a whole bunch of complexity.

Or could we just keep things simple and reference the first verse in the span?

This is why this issue is blocked

[birchamp]

This issue needs to be re-considered and may need a new design. Align to unexpected verse #3739 has been de-prioritized by the stakeholders. Consider making a verse map of the imported USFM.