Batch edit

[Woseseltops]

• Somewhere in the search results there is a button 'batch edit'
• This button brings you to the BatchEditListView, where you see each search result as a row, and as columns:
  • The small video
  • The gloss annotation ID
  • Several editable gloss fields, which @uklomp will provide below
• Important: the BatchEditListView has only one Apply button (unlike the detail view, where you have to click OK for every change you make)

[susanodd]

I'm working on the BatchEditForm now. For Senses, only existing senses can be edited (translations filled in, as discussed).

(At the moment, new senses cannot be created, and senses cannot be reordered. Gloss Edit will need to be used for that. Otherwise too much interface code is needed.)

[susanodd]

@uklomp do all glosses have senses?

If the gloss does not have senses, do you want an empty field to appear? Or only to appear if a language has not been filled in? (Like a touch up edit.)

To avoid yet another method to create senses if possible?

At the moment, there is Gloss Edit, Keywords Mapping Edit (this is also batch edit but just for senses), and API create gloss, and API update gloss.

The Analysis -> Keywords Mapping was meant to facilitate the change from the Keywords system to the Senses system for already existing glosses, since there were only keywords to begin with. These can then be split up into senses.

[susanodd]

I have some of the fields done already. The toggles are implemented and work. The language fields can be edited, but a Save button is needed for these. (Hence my question above. As you can see there are no fields for the senses for this specific gloss -- a copy from NGT to tstMH, the Copy button only copies phonology.)

[susanodd]

The plan is to show videos for similar glosses below when say Handedness, Strong Hand and Weak Hand have been filled in.

(Enough fields need to be filled in or there can be thousands, if you only have Handedness filled in.)

Sufficient fields need to be available in the Toggles area in order to narrow down the focus.

Perhaps a button when you think the gloss phonology is "done" to generate the similar gloss videos? (There needs to be phonology for the other glosses obviously.) (A new search would be needed if the similar glosses are not in the list of batch edit glosses, so you could work on them all if needed to disambiguate if more phonology is needed.) The facility to generate a search based on the single gloss phonology is available via the Query button in Gloss Detail View.

[susanodd]

Here is what happens with displaying the senses this way. This is one of the test glosses that we used when Senses were being developed. The keywords are split into different senses. (Although as non-signers, we have no idea if this is nonsense.) This looks a bit much. Although this is what it looks like. We implemented the functionality for already existing glosses to split the keywords in different senses.

[susanodd]

I can make it so the language fields also collapse to hide them. Suggestions are welcome! Think about what you need to be able to do in batch edit.

[susanodd]

Here you can see one with the EN senses with empty fields. Only two phonology toggles are implemented so far, so the rest don't show yet for the glosses.

[susanodd]

More toggles added, showing highlighted sense translation field:

[uklomp]

@uklomp do all glosses have senses?

If the gloss does not have senses, do you want an empty field to appear? Or only to appear if a language has not been filled in? (Like a touch up edit.)

To avoid yet another method to create senses if possible?

At the moment, there is Gloss Edit, Keywords Mapping Edit (this is also batch edit but just for senses), and API create gloss, and API update gloss.

The Analysis -> Keywords Mapping was meant to facilitate the change from the Keywords system to the Senses system for already existing glosses, since there were only keywords to begin with. These can then be split up into senses.

Not all glosses have senses, I'd like to be able to add them then yes, so a field would be good. Would it be possible to have all English stuff in a column next to the Dutch stuff? At least for the senses, but preferably also for the glosses and lemmas. So not all underneath each other, but English next to Dutch.

[susanodd]

Not all glosses have senses, I'd like to be able to add them then yes, so a field would be good. Would it be possible to have all English stuff in a column next to the Dutch stuff? At least for the senses, but preferably also for the glosses and lemmas. So not all underneath each other, but English next to Dutch.

I'll experiment with that next. I have the phonology fields working as shown above. Just need to keep adding more.

The query results has been done so it starts with the query results in the list.

I'm working on the language fields now. I added a panel "language fields" with a button "save language fields" button in the toggles area.

I'll modify the display to show the languages in columns.

Do you want more things in the Language Fields panel? (The Save button is there so it's more like the phonology toggle buttons.)

A create button for senses? An "assign lemma" to replace the lemma with an existing lemma? Another option is to put the "new contents" in the languages panel and on update shove it into the fields to the left. Then the fields on the left could look prettier.

I'll do the columns next and post a screenshot.

[susanodd]

Here's with columns (the display scrolls to the right):

[uklomp]

I remember that we talked about making phonology work this way, but I didn't remember putting them in this BatchEdit as well. I'm not sure yet if it's a benefit, but it's also not a disadvantage I suppose.

Senses in two columns: looks way better, thanks.

What do you mean with language fields, I have no idea what they are?

[susanodd]

I remember that we talked about making phonology work this way, but I didn't remember putting them in this BatchEdit as well. I'm not sure yet if it's a benefit, but it's also not a disadvantage I suppose.

Senses in two columns: looks way better, thanks.

What do you mean with language fields, I have no idea what they are?

The language fields (what I call them, code wise) are dependent on the translation languages of the dataset. Here the user enters text But they need to be generated dynamically for the input form, because of the languages/dataset (the code is more complicated because it includes a language code, e.g., lemma_NNNN_nl (for gloss NNNN lemma in Dutch) Those are the internal labels attached to the text fields of the input form.

Because batch edit is a giant input area, the form fields have the gloss id attached, the buttons have the id attached,...

The phonology is super easy to update this way, no pulldowns.

What about calling them "Text Translation Fields" ? (There needs to be some way to identify them on the page.)

It would be possible to just hit Enter after updating some text. But that could go wrong if the user forgets to type enter. The fields will looked changed on the screen after editing the text. But there needs to be a signal to the computer that it is ready to save. (Hence the OK buttons all over the place in Gloss Edit.)

The text fields are handled differently because for each text field, there are some other objects attached. In contrast, the phonology fields are fields of the gloss itself. But the translated text fields are stored in other tables. So different implementation code is needed.

[susanodd]

I remember that we talked about making phonology work this way, but I didn't remember putting them in this BatchEdit as well. I'm not sure yet if it's a benefit, but it's also not a disadvantage I suppose.

We talked about having the similar glosses show up while you are entering the phonology. This will work in this view when you are filling in phonology for a gloss that does not have phonology yet.

(For glosses that have the phonology filled in already, you can see homonyms and minimal pairs, or use the Query button to search on the glosses's phonology fields.) (With the toggles, you are gradually filling in more phonology, so it's possible to show glosses that have the same as has been entered. But it ought to be more than handedness, because there are thousands of similar glosses if only handedness is filled in.)

This would be weird to do in Gloss Edit, since you are in Edit mode and all the cells are red, waiting for input.

With Batch Edit, you don't need to keep reloading the page nor go to a different list view. You can just keep editing the glosses on the page, in-line.

[susanodd]

[NOTE TO SELF SKIP]

This is implemented now, except for the senses.

There needs to be some kind of indicator so the user will know when the text fields have been updated in the database. In gloss detail edit, you see this because you are inside a form field with the OK button, then leave it and go to the next field.

With the batch edit form, you type in text and the fields appear changed in the display, but nothing is saved until you hit the save button for the gloss. After clicking save, error message feedback appears under the save button. But what to do if the save is okay? And what to do if the user changes the text again?

We could use separate entry fields for the "new" text versus the "gloss" text. Then you would see the fields update. (As is done for the toggles.)

Or I can make it show success feedback or a green checkmark.

Django can keep track of the gloss field values. But if you change this in the form, then you need to do something (user needs to click on something) in order for Django to fetch the gloss data for the field again. (When you update the text, you are changing what is on the screen.) This works really well for the toggles, they are WYSIWYG. But for text, this is not apparent to the user.

I'll try adding a green checkmark after save. Or when the text matches the contents of the gloss. Or a different checkmark if the on-screen text has been changed. (But this cannot be know, because the real values are in the database.)

That's probably confusing.

This could be solved by putting the "entry text fields" in the panel to the right and not type directly into the fields. Then clicking on save, you would see the fields on the left updated.

[susanodd]

FEEDBACK SCREENSHOTS updating text fields / language fields

[susanodd]

[NOTE TO SELF] There might need to be a clear form button after an unsuccessful attempt to update the text fields. (At the moment, the proposed changes still remain visible. Need to investigate.)

[susanodd]

@uklomp this is on signbank-test now. UNDER THE ANALYSIS MENU

[susanodd]

• Somewhere in the search results there is a button 'batch edit'

• This button brings you to the BatchEditListView, where you see each search result as a row, and as columns:

  • The small video
  • The gloss annotation ID
  • Several editable gloss fields, which @uklomp will provide below
• Important: the BatchEditListView has only one Apply button (unlike the detail view, where you have to click OK for every change you make)

One comment here, there are no small videos. The creation of small videos has been broken for a long time. There is a bug in the CNGT scripts. The videos are merely displayed smaller. @Jetske determined that the bug relates to fps, that a different frame rates is used by some video software and the error occurs because when resized to a smaller video the length (in number of frames and frames per second) no longer matches that of the input video and can't be mp4.

[uklomp]

I remember that we talked about making phonology work this way, but I didn't remember putting them in this BatchEdit as well. I'm not sure yet if it's a benefit, but it's also not a disadvantage I suppose. Senses in two columns: looks way better, thanks. What do you mean with language fields, I have no idea what they are?

The language fields (what I call them, code wise) are dependent on the translation languages of the dataset. Here the user enters text But they need to be generated dynamically for the input form, because of the languages/dataset (the code is more complicated because it includes a language code, e.g., lemma_NNNN_nl (for gloss NNNN lemma in Dutch) Those are the internal labels attached to the text fields of the input form.

Because batch edit is a giant input area, the form fields have the gloss id attached, the buttons have the id attached,...

The phonology is super easy to update this way, no pulldowns.

What about calling them "Text Translation Fields" ? (There needs to be some way to identify them on the page.)

It would be possible to just hit Enter after updating some text. But that could go wrong if the user forgets to type enter. The fields will looked changed on the screen after editing the text. But there needs to be a signal to the computer that it is ready to save. (Hence the OK buttons all over the place in Gloss Edit.)

The text fields are handled differently because for each text field, there are some other objects attached. In contrast, the phonology fields are fields of the gloss itself. But the translated text fields are stored in other tables. So different implementation code is needed.

Ok, so if I understand correctly, you're asking me if we need more buttons and if it's OK to have one 'save' button? I think the 'create sense' would be a good option, like you suggested, and one 'save' button for all the language fields as well. Btw I think text fields would be a more common name yes.

[susanodd]

First version of similar glosses. Of the available fields, you can turn them on or off to search by them for that gloss. (There aren't enough fields yet for this.) It shows the number of similar glosses. And if the number is 18 or less, it shows them. (6 rows of 3 when shown with videos)

Here is the first version (without the videos yet, just the annotations of the similar glosses) This is basically a search on Handedness 1, Location Head, Movement Shape Bow (In the screenshot, the Weak hand is empty, so it's not using that. The Strong hand is turned off.)

[susanodd]

I was trying to find the horns example at the live meeting. I need to add a few more phonology fields for that....

[uklomp]

Shall we separate the 'batch edit phonology' and 'see similar glosses' from the current issue of making batch edit possible for lemma, glosses and senses? I'm getting confused now.

[susanodd]

Shall we separate the 'batch edit phonology' and 'see similar glosses' from the current issue of making batch edit possible for lemma, glosses and senses? I'm getting confused now.

Do you want separate pages?

I thought the idea was to do all the editing in-line instead of using csv. :)

[susanodd]

I fixed a bug related to the page links at the bottom. (It was doing some weird stuff on the pages after 1,showing glosses that had not been queried.)

I haven't done the senses yet.

If you have a chance please try the Keywords Mapping (#1273). (That is batch edit functionality to edit senses.)

[uklomp]

Shall we separate the 'batch edit phonology' and 'see similar glosses' from the current issue of making batch edit possible for lemma, glosses and senses? I'm getting confused now.

Do you want separate pages?

I thought the idea was to do all the editing in-line instead of using csv. :)

I want separate issues here on Github :)

[uklomp]

I was trying to find the horns example at the live meeting. I need to add a few more phonology fields for that....

I don't see this. Where can I find these overviews? I went to signbank-dev and I tried several datasets but I don't see these phonology fields.

[uklomp]

As for 'horns', I meant the sign that is glossed as koe-a

[susanodd]

Shall we separate the 'batch edit phonology' and 'see similar glosses' from the current issue of making batch edit possible for lemma, glosses and senses? I'm getting confused now.

Do you want separate pages? I thought the idea was to do all the editing in-line instead of using csv. :)

I want separate issues here on Github :)

Can it still be on one page, or would you prefer "Batch Edit Text" and "Batch Edit Phonology" ?

[susanodd]

I was trying to find the horns example at the live meeting. I need to add a few more phonology fields for that....

I don't see this. Where can I find these overviews? I went to signbank-dev and I tried several datasets but I don't see these phonology fields.

I meant that I was trying to simulate your example where you made the horns sign (in real) and I was trying to figure out what phonology comprised the sign you were making. (I do not know sign language, so I was trying to figure out what phonology I would need to put in the Gloss Search form in order to find signs with horns. (I needed to add Movement Shape.)

If I already know the sign KOE and I type in all the fields and search for this, I only find KOE.

To find more than one sign, I needed to ignore some of the phonology of KOE.

It worked to make a Boog near the Hoofd. (Search on Boog and Hoofd in the appropriate fields.)

[susanodd]

I revised the display as per feedback, fewer Sense labels It looks a bit more user friendly now.

[susanodd]

This is live on signbank-test

Analysis -> Batch Edit
Analyse -> Batchbewerking

[susanodd]

Updating Senses and Create Sense are implemented.

This is on signbank-test.

[susanodd]

The hyperlink annotation shown under the video is updated to show an update to the dataset default annotation.

[susanodd]

Most recent code is on signbank-test.

Here, the panels are now collapsible and there are videos for the similar glosses. The values of the phonology fields are shown in the buttons now. You can turn the buttons on and off to refine the query.

[susanodd]

Text Fields panel OPEN on viewing. (User can explicitly close them.)

[susanodd]

• Somewhere in the search results there is a button 'batch edit'

• This button brings you to the BatchEditListView, where you see each search result as a row, and as columns:

  • The small video
  • The gloss annotation ID
  • Several editable gloss fields, which @uklomp will provide below
• Important: the BatchEditListView has only one Apply button (unlike the detail view, where you have to click OK for every change you make)

For each gloss, there will be a SAVE TEXT button. For the purpose of "atomic transaction" and "checking constraints" this is necessary. Otherwise, work by other users of signbank during the time a user is "busy" changing text fields, may interfere and prevent work from being saved.

Updates to the "text fields" of EACH gloss involves updates to the following TABLES / related objects to the gloss:

• Gloss
• LemmaIdglossTranslation
• AnnotationIdglossTranslation
• Keyword
• Translation
• SenseTranslation
• Sense
• GlossSense
• GlossRevision

Additionally, video files may be moved if lemma translations or gloss annotations of one gloss have been updated.

[Woseseltops]

Hey @uklomp (when you're back), Susan and I disagree about whether there should be 1 generic 'save/apply' button instead of a separate one for each gloss. I don't remember your viewpoint here? Susan does correctly note that if there's only one button, you might get a whole list of objections, similar to what you get when uploading a CSV.

[susanodd]

There are sufficient phonology fields now to show the similar glosses to KOE-A (example given at the live meeting) (Observe that some "similar gloss fields" shown have been toggled off so to be ignored in the Query.)

[susanodd]

All but one phonology field added. Here's what the quick search on similar glosses looks like for KOE-A now.

[susanodd]

On initial view, all the phonology area are closed and only the text fields are visible.

I designed it this way because this allows the users to conveniently check and correct the phonology here, if necessary, without needing to go to a different page. This page does not need to be reloaded.

[uklomp]

Edit: I'm confused. To answer @Woseseltops question I'd like to try it again. Should I go to signbank-dev or signbank-test? And then perform a search query and then go to Analyse > Batchbewerking, right? It used to work but I don't think it's working on either of these test sites right now.

[susanodd]

@uklomp Yes, signbank-test.

(It was moved to signbank-test because signbank-dev is being used by @vanlummelhuizen to work on the API permissions and updating the view dataset problem. We have four development servers now: signbank-dev, signbank-susan, signbank-jetske, signbank-test

[uklomp]

Ok! A few feedback points:

1. The textfield where the pointer is, in the screenshot, is too small. I can't add any more characters, while I do need them.

3. I'd like 'enter' to mean 'save / finished' and not mean to go to the next line in a text field. I know this is the functionality when editing senses on the gloss detail page and that's fine, but in the batch edit when you're typing, you just want to continue to the next one and finish it with 'enter'. Adding multiple words to one number of senses can still be done with comma's.
4. As for the question of saving: right now, I'm under the impression that everything that I've typed so far, is saved. So I don't see the need for a save button at all. Is this the wrong impression?

[susanodd]

Ok! A few feedback points:

1. The textfield where the pointer is, in the screenshot, is too small. I can't add any more characters, while I do need them.

Okay, I can fix this. On a small screen there isn't enough space, although the entire page is really wide and the user can "pan" to the right. (I'm describing what happens on a small macbook 11 inch, it's not scrolling but you can keep moving to the right. I don't know what that's called, this is the only view that works like that for me, so I'm confused by the markup. But yes, I'll make it wider.)

3. I'd like 'enter' to mean 'save / finished' and not mean to go to the next line in a text field. I know this is the functionality when editing senses on the gloss detail page and that's fine, but in the batch edit when you're typing, you just want to continue to the next one and finish it with 'enter'. Adding multiple words to one number of senses can still be done with comma's.

Okay, I need to figure that out. I guess it will just be a save for that field then.

4. As for the question of saving: right now, I'm under the impression that everything that I've typed so far, is saved. So I don't see the need for a save button at all. Is this the wrong impression?

No, they are shown as updated on the screen but (as it is now, before doing request 3) they are only "saved" when you click the save. And if there are problems with the constraints, feedback is shown below the save button. For example, if you change it to something that already exists. There needs to be feedback and it's shown there. It can't be shown "inside" of the area where you type the new text because that is all generated dynamically. It would be a mess to start showing feedback between the rows, say. But there needs to be a place to show the feedback because the updates to the lemma and annotation fields can fail.

I need to figure out what to do for implementing (3) because there can also be failure feedback.

[susanodd]

Do you already want to start using it once the text area is made wider? And make the updates to the save later?

[IMPLEMENTATION CAVEATS] For the implementation of the "save" (per gloss) as it is now (for the text fields), "all" the constraints are checked at the same time. I need to check how this would happen if the save is "per text field".

(In computer science, there is a well-known "dining philosophers problem" for mutual exclusion when checking constraints on transactions. The philosophers are dining with chopsticks, but there is only one chopstick between each, so they need to grab both chopsticks in order to eat. So they need to wait if another philosopher is using one (or both) of the chopsticks. I have no idea if that is relevant, but it's the same idea. The constraints ought to be applied to "all" the lemma fields, and likewise to "all" the annotation fields. Say for example if you want to change from Lemma-A to LEMMA-B.)

(Just ignore that or browse it, but that's in the back of my head. I need to check whether anything like that could happen That there is a database underneath means that there are transactions, so they should end up "all or nothing" when you update.There are mini transactions now per "gloss field" with the toggles. But for the text fields there are constraints.)

[uklomp]

No, the updates to the save are more important.

If we do need a save button, I'd like one very clear save button (maybe even red or another bright color) on the top of the screen and on the bottom of the screen. But not a save button for every gloss.

[susanodd]

No, the updates to the save are more important.

If we do need a save button, I'd like one very clear save button (maybe even red or another bright color) on the top of the screen and on the bottom of the screen. But not a save button for every gloss.

Okay, I need to think about this. I revised the answer above at the same time as you were typing. There is a difficulty with "transactions" if it's "one" button. You would basically need to do something like the way "Edit" is done in Gloss Edit to make sure nobody else is editing at the same time.

If there is only one button, there is a big chance that work will be lost for example, if any of the constraints fail. There would need to be a clear semantics for what happens if any of the lemma or annotation text field "update attempts" fail. It would need to be that "none" of the updates are done. (Because one of the updates could not be done),

@Woseseltops:

"how to keep ONE button update to ALL glosses LEMMA and ANNOTATION fields atomic?"

This becomes a problem if you are attempting to SWAP different lemmas or annotations at the same time. For example, make gloss 1 go from ANNOTATION-A to ANNOTATION-B and at the same time make gloss 2 go from ANNOTATION-B to ANNOTATION-A.

(That is a transaction failure. Normally, you would need to sequence them. But then that is the same as doing it per gloss...)

[susanodd]

@uklomp I can make it so the "senses text" can be updated without using the "Save" button, but the Lemma and Annotation text fields have the problems described above (because there are database constraints).

The "senses" text as they are now are done with the "save" button because:

when updating senses, the "old" senses (objects) are "replaced with" the new senses (objects) when they are updated. This is ALSO how the API works. It replaces the old ones with the new ones. (That is the solution that @Jetske came up with for updating the senses in the API.)

So I need to figure out if using an "update save per sense text field" has any adverse effects. This probably needs to be completely re-implemented since this would be an "update of the sense itself" rather than an "update of the glosses senses".

(The other issue for moving around keywords to different senses -- #1273 -- does it that way; it is updating the individual senses, not all the senses, and not replacing the senses. The internal sense ID and sense translation ID and sense # are used internally inside the template and Python code to do this. The code is updating the objects that comprise the senses.)

Probably need to discuss this with @Woseseltops because one approach (as it is now) is from the perspective of the Gloss object and its senses, but the other approach would be from the perspective of the Sense object itself. At the moment, the user has no access to the sense objects, there are no "Sense IDs" visible. (Likewise, the Senses code freely creates new and deletes old sense objects because they are only accessible via the gloss they are associated with.So if we make things work at the Sense level, then we should not be deleting them and replacing them as is done now. Then the approach of #1273 would need to be used instead.)

[susanodd]

No, the updates to the save are more important.

If we do need a save button, I'd like one very clear save button (maybe even red or another bright color) on the top of the screen and on the bottom of the screen. But not a save button for every gloss.

Okay, I can phrase the problem differently: if there can't be a save button per gloss, then there needs to be a "check consistency" button and an area of the layout for reporting inconsistency (for all the glosses on the page, similar to what CSV update does with feedback.)