Increasing efficiency of adding to and editing a dialect's grammatical data

[GeoffreyKhan]

The NENA team have been discussing possible ways of increasing the efficiency of uploading grammatical data. The 'bulk edit' facility is helpful for the entry of data in individual paragraphs. However, it seems to us that the idea would be to have some kind of table with all the 'grammatical slots', ideally in Word (or written in Word and copied to Excel), in which we can write all the data, then upload it in one go to the database. Can this be done?

[jamespstrachan]

This reminds me of a similar conversation about having a word template file and an importer for transcriptions and translations. As I tried to express at that time, it only sounds simpler to do it like that. It actually gives us two problems to solve simultaneously:

• How to deal with the myriad formatting, character and alignment issues that Word causes, or at least can't robustly prevent
• How and if to accept bad data, then how to present the issues back to the user for fixing

Extending the scope of the existing bulk editor, or even making a more replete alternative would avoid these tricky interface and integration issues. These are not trivial development challenges, and to do the second one properly basically means implementing a new, interactive bulk editor in-browser anyway!

I previously claimed that with time and a clear specification we could build all of the features, keyboard-shortcuts and conveniences you liked about Word into a more focussed web interface. I still believe that's true; that it's actually less work than the hidden-costs of building a loader/checker/reviewer; and that we've yet to scratch the surface of what's possible.

Let's discuss what good and great look like in the context of bulk editing. I'm assuming that - unlike the corpus texts - you don't already have folders full of documents containing new grammatical data, right?

[GeoffreyKhan]

Understood. Expansion of the bulk editing facility would probably, therefore, be the best way forward, for example, a bulk editing window that would allow bulk entry of a whole section together with the lines of the subsections, e.g. a bulk entry of lines of data for the whole of section 6.14.1 that would include the subsections of this section.

[GeoffreyKhan]

If the extension of the bulk editing facility such as I have just described seems feasible, please could you prioritize this, since we are about to embark on a lot of data entry and it would greatly increase efficiency.

[GeoffreyKhan]

Bulk entry at, e.g. 6.14.1. at the moment only allows for entry of items one level down:

Ideally the bulk entry at level 6.14.1. should allow the entry of items arranged one under the other, which would slot into each of the slots of the lower levels, thus: a > 6.14.1.1.1. b > 6.14.1.1.2. c > 6.14.1.1.3. d >6.14.1.2.1.
e > 6.14.1.2.2. f > 6.14.1.2.3. etc

[jamespstrachan]

That's a very clear spec, with clear pictures and text - thank you!

I have bodged together an update to the bulk editor which removes the 1-depth restriction. It seems to work from my quick testing so I've pushed it to staging so you can have a go, eg: https://nena-staging.ames.cam.ac.uk/dialects/49/grammar/6.14.?edit=1

Warning: the power this gives you to do lots of good changes all at once is also the power to corrupt/delete loads of data in one fell swoop, particularly when editing long lists of text which disappear off the bottom of the screen. Be careful. Any suggestions of sensible guard-rails we could put in place would be worth talking about.

[GeoffreyKhan]

This seems to be working fine, so long as you leave an empty line in the bulk editing that corresponds to the subparagraph title, thus:

[jamespstrachan]

Good feedback once again. In fact the picture let me spot an issue that I hadn't noticed in my testing - namely that there's a subtle misalignment appearing by the bottom of the screen shot (notice how the riser of the "b" is getting closer and closer to the underline of the previous section each time). I'm guessing that this screenshot was taken on a browser that was zoomed in, as I managed to get some pretty severe misalignments by the end of longer edit pages by zooming to over 130%. I think I've balanced out the way the two columns zoom to keep them aligned. I'd encourage you to test this too!

empty line in the bulk editing that corresponds to the subparagraph title

That feature groups are themselves features has been a "feature" of the software since before I was involved. In fact many dialects contain data within their heading sections (eg 4.3.[2,3,6]. on https://nena-staging.ames.cam.ac.uk/dialects/49/grammar/4.3.) Whether or not we want this to remain how the system works in the longer term, it's how it works currently and so each numbered section will have a row in the bulk edit, even if it contains subsections.

[GeoffreyKhan]

Thanks, James. I think for the time being please keep the 'feature' that feature groups are features, since several entries are in the slot of these subtitles. The new bulk extended editing system seems to be working well, if you've fixed the alignment issue.

[jamespstrachan]

I think I've fixed the alignment issue so that it works on long pages and at high zoom levels. However part of User Acceptance Testing is that you also confirm it's fixed by having a good go at breaking it! Please push the boundaries and see if you can still make stuff fall out of sensible alignment.

Copying in @Paul32N as this is relevant to part of the conversation we had on Friday. Paul, could you test the above and if you're happy that it's non-breaking apply the ready for production label?

[jamespstrachan]

Closing as this has been rolled out to production.