search

unfoldingWord / translationCore

Repository for the desktop application translationCore
https://www.translationcore.com
Other
36 stars 11 forks source link

Word Alignment tool rewrite with new architecture #4319

Closed benjore closed 6 years ago

benjore commented 6 years ago
da1nerd commented 6 years ago

@benjore what should happen if, when loading some alignment data from a file, I find a word that does not exist in the verse?

For example. I loaded this project into tC and discovered it uses the following word:

"word": "ἐν",
"lemma": "ἐν",
"morph": "Gr,P,,,,,D,,,",
"occurrence": 1,
"occurrences": 2,
"strong": "G17220"

However, in the UGNT there is only once instance of ἐν within that particular sentence. Therefore the occurrences value is wrong. It should be 1 not 2.

Should I completely reset the verse alignments in this case?

cckozie commented 6 years ago
  1. Reseting of alignments after verse edits no longer works properly. I loaded a project that was fully aligned in tW and made an edit to verse one. I was correctly informed that the alignments for the verse had been reset, but when I opened the project in Word Alignment, only two words of the verse had been moved back to the word list and the rest was still aligned as before the edit. (I wrote that up unaware that it was a new feature being worked on)
  2. The alert warning the user that the alignments for the verse have been reset are no longer being displayed in word alignment when the verse was edited outside of the wA tool (in tW).
  3. An empty alignment box is again being displayed when there is no Greek text for the verse. It looks like the code related to this from issue #3670 is no longer there or working.
da1nerd commented 6 years ago

@cckozie how can I reproduce the third issue described above?

cckozie commented 6 years ago

@neutrinog Import the attached and go to 17:21 in the alignment tool. 41-MAT.usfm.zip

cckozie commented 6 years ago

0.10.0 (e6fe53e) A. Footnotes being displayed in word list en_ult_mat_text.zip image

B. The check mark for completed alignments is not being displayed or reset until the user clicks on another verse or exits the tool.

mannycolon commented 6 years ago

build 0.10.0 (af7b11b) (Copied from #3562 written by @cckozie ) There is still a problem here. To recreate: 

9) Align verse 1:10
10) Switch over to tW and change the word 'people' to 'folks in verse 1:10
11) BUG: The reset alignment warning is not displayed when the Save button is clicked
12) Go the tools page and see that the wA card does show 1 reset alignment
13) Load the project in wA and navigate to verse 1:10
14) The reset alignment alert is correctly displayed
da1nerd commented 6 years ago

@cckozie @benjore @jag3773 Should wA be expected to parse footnote USFM or should tC clean all of that out first?

cckozie commented 6 years ago

The user is no longer warned when a verse is edited in wA that invalidates a check in tW.

cckozie commented 6 years ago

0.10.0 (a48d474)

  1. The broken link icon is not being displayed on verses where alignments have been reset.
  2. The % complete progress circles are not being updated as alignments are done.
da1nerd commented 6 years ago

The alignment warning is now fixed in https://github.com/unfoldingWord-dev/translationCore/pull/4621 and https://github.com/translationCoreApps/wordAlignment/pull/43.

The progress circles also appear to be working. I think this was fixed by something @PhotoNomad0 was working on.

da1nerd commented 6 years ago

The warning about the missing bible has been added back in https://github.com/translationCoreApps/wordAlignment/pull/43

cckozie commented 6 years ago

A. When moving an aligned word to a different Greek word, the user is warned that there have been changes to the verse and the word being moved is moved back to its original place. FIXED IN 0.10.0 (4add745)

  1. Open a project in wA.
  2. Drag the first word in the word list to align with the first Greek word.
  3. Drag that word to align with the second Greek word. The change warning is displayed and the word is moved back.

B. s, p, and q usfm markers show as words in the word list image

da1nerd commented 6 years ago

@cckozie I've fixed the dragging issue in https://github.com/translationCoreApps/wordAlignment/pull/47.

Those extra tokens are handled in this PR from @mannycolon https://github.com/translationCoreApps/wordAlignment/pull/49

cckozie commented 6 years ago

wA locks up when dragging word between alignment boxes. I made alignments in 1 Timothy 3:16 as shown here:

image

I tried dragging 'taken' from ἔθνεσιν to ἀνελήμφθη and the app froze other than I can still scroll.

image

da1nerd commented 6 years ago

@cckozie can you upload the project? I cannot reproduce this error with the projects I have.

cckozie commented 6 years ago

@neutrinog Here is the project, but I have been unable to reproduce the problem in the current build 0.10.0 (0ef3ec4). en_ult_1ti_book.zip

It was imported from 55-1TI.usfm.zip

cckozie commented 6 years ago

0.10.1 (f43aa52) en_ult_1th_book.zip

When I open this project in wA I get the 'There have been changes ...' alert, but none of the alignments get reset. Seems like I should either not get the message or I should see words returned to the word list.

da1nerd commented 6 years ago

@cckozie the changes may have been in another verse. wA will now perform alignment validation on the entire book not just the current verse.

cckozie commented 6 years ago

A. The 'some alignments have been reset' message is displayed on the wrong verse.

  1. Make some alignments on a verse in Titus
  2. Click on the previous verse in the verse menu
  3. Exit wA and load the same project in tW
  4. Make an edit to a word that was aligned in 1. above
  5. Exit tW and load the same project in wA
  6. The 'some alignments have been reset' message is displayed even though the current verse did not have any edits
  7. Click on the next verse (the one that was aligned in step 1. The 'some alignments have been reset' message is not displayed but should be.

B. The 'some alignments have been reset' message is displayed when no resets occurred. Edit a word in the current verse that has not been aligned. The message is displayed but should not be.

da1nerd commented 6 years ago

@cckozie when a verse is edited wA is evaluating the entire book. It doesn't matter which verse you are currently on. So what you have described above is exactly how it should work. This ensures the user can always see the correct icons in the groups menu.

If you think it would help, I can expand on the message by including the references that was reset. Or if multiple verses were reset I can just include something like "several verses have been reset".

cckozie commented 6 years ago

@neutrinog That is not the way it worked before the refactor. Then the message was displayed when the user navigated to the verse that had alignments reset.

da1nerd commented 6 years ago

That is correct. We had to change it to avoid stale state.

da1nerd commented 6 years ago

waiting on feedback from @benjore

benjore commented 6 years ago

Here's what we need to do:

  1. For any verse that has had something reset/invalidated, the invalidated icon needs to show in the side menu. (The invalidated icon remains until some action is taken on that verse (e.g. click or drag a card or click a button), then it disappears.)

  2. New warning dialogs:

    • Changes have been detected that invalidate work previously done in the translationWords Tool.
    • Changes have been detected that invalidate work previously done in the Word Alignment Tool.
da1nerd commented 6 years ago

@benjore is this invalid icon meant to be functionally separate from the the invalid icon seen in tW? That is to say, a verse that is invalid due to reset alignments will show the invalid icon while in wA but not in tW. This pattern is seen currently in tW where an invalid icon there is not shown in wA.

However, if each tool displays it's own invalidation icons what happens when someone edits a verse in tW and is directed to look at the invalid icons. They won't actually see the new invalid icons.

I suspect we will need to change the invalidation (I think this is what is being called checkData in the project) to be a generic feature of the tc tool so that there is consistency between the tools.

cckozie commented 6 years ago

@neutrinog The invalid icon needs to be displayed in wA for each verse that has alignments that have been invalidated regardless of whether the edit was performed in tW or wA. The icon is displayed on the verse until the alignment is fixed. (This should be the same behavior as before the rewrite) Also, per @benjore, the warning message should be displayed only when an edit causes one or more words to be moved back to the word bank. Currently the message is displayed in this scenario for edits that do not affect alignments: Partially align a verse Make an edit to that verse that affects the alignment (message is displayed) Make another edit to the same verse that does not affect alignment (message is again displayed)

da1nerd commented 6 years ago

ok. Then fixing the text and fixing when the dialog will be displayed is going to be easy. However, the icon will be tricky, and I think we'll need to discuss this in person to make sure everyone understands what's happening.

da1nerd commented 6 years ago

@benjore @cckozie what would you think if the invalidation icon disappeared when the user visited the verse instead of just "when they change it"?

There are a few benefits to defining it that way

cckozie commented 6 years ago

@neutrinog @benjore I think ^^^ is a less desirable solution for the user.

cckozie commented 6 years ago

0.10.0 (26126c1) A. [#4766] Broken links not displayed by verses that have reset alignments THIS ISSUES WAS MOVED TO #4768

https://git.door43.org/tc02/es_fail_tit_book

~~1. Load this project in wA and note the alignments in 1:1

  1. Switch over to tW and go 1:1 in the group 'apostle...'
  2. Select words 'a servant'
  3. Click Save
  4. Click Save & Continue
  5. Click on the group 'blameless'
  6. Expand the scripture pane and open verse 1:1 for editing
  7. Change the words 'a servant' to 'an apostle'
  8. Click Next, then choose a reason and click Save
  9. Click OK on the 'alignments reset' alert (I'm writing up the multiple alerts problem elsewhere)
  10. Close the expanded scripture pane
  11. Load the project in wA and notice that the broken link displayed by verse 1:1 briefly appears and then disappears~~

B. [#4765] THIS ISSUES WAS MOVED TO #4769 The tW 'some selections are no longer valid' alert should be displayed when edits performed in wA affect tW selections. (Currently when edits are made in wA that affect tW selections, the user is not warned that selections have been cleared.)

C. [#4767] en_tstf_tit_book.zip

  1. Project shows 100% complete alignment on wA tool card. This is inconsistent with the verses with check marks in the projects.
  2. When loaded in wA the 'some alignments reset' alert is displayed and a few broken links are shown by the verse number. However, when those verses are clicked on, the verse alignments appear to all be complete.
  3. Only 1:1 and 1:2 have the check mark even though many more, if not all, of the verses do have complete alignments.
  4. Only 1:1 and 1:2 have the Alignment Complete switch set even though many more, if not all, of the verses do have complete alignments. (Probably same issue as 2.)
  5. Clicking on a verse with the broken link icon causes the icon to be cleared. However, after exiting wA and then reloading the project, the broken link icon reappears.
  6. After clicking on all of the verses with broken links to clear the icon, when the project is again loaded to wA the 'alignments reset' message is still displayed.
mannycolon commented 6 years ago

@cckozie Hey could you please give me more details about B. The tW 'some selections are no longer valid' alert should be displayed when edits performed in wA affect tW selections.

Is the some selections are no longer valid alert supposed to open while working in wA ?

benjore commented 6 years ago

For the 0.10 release, let's just do letter C.

cckozie commented 6 years ago

0.10.0 (6ea8367) This is an aligned project from a repo the aligners are using. https://git.door43.org/ult-pre-alignment/en_tit

When loaded in wA the 'invalidated alignments' alert is displayed. A broken link is displayed at 1:7 and 2:5. Clicked on both verses to clear the broken link. Switched to tW. 'Invalidated alignments' alert is displayed. Switched back to wA. 'Invalidated alignments' alert is displayed. Broken link is no longer displayed on 2:5, but it is still displayed on 1:7. Clicked on 1:7 to clear the broken link. Switched to tW. 'Invalidated alignments' alert is displayed. Switched back to wA. 'Invalidated alignments' alert is displayed. Broken link is no longer displayed on 1:7, but now it is still displayed on 2:5.

It appears that whatever verse was selected in wA when the switch was made to tW will keep the broken link cleared, but the broken link will come back on the other verse(s). If a completely different verse is selected in wA when switching to tW, then both of the broken links come back. It works the same way if I load a different project with wA and then return to the original one.

cckozie commented 6 years ago

https://git.door43.org/ult-pre-alignment/en_1th

  1. Import and load project in wA
  2. Click on 2:7 to clear the broken link
  3. Load the project in tW
  4. Load the project in wA Notice that all of the other verses in chapter 2 now have the broken link icon
cckozie commented 6 years ago

There are still a lot of remaining problems, but they have all been written up in new issues so that we can get 0.10.0 out.