search

ualbertalib / avalon

University of Alberta's Media Repository based on Avalon
Apache License 2.0
2 stars 2 forks source link

Waveform editor not working correctly #680

Closed seanluyk closed 3 years ago

seanluyk commented 3 years ago

To replicate

The window closes and no edits are allowed. May be item-specific so I'm wondering if this one of the items where the backfill script failed?

jefferya commented 3 years ago

Tested on Ubuntu 18.04 / Firefox 82.0.3. Able to reproduce.

According to Firefox console, there is a JavaScript exception causing the window to close.

Todo: try to reproduce in a dev environment.

Hypothesis, there might be an invalid number in one of the time spans, will check. Uncaught TypeError: peaks.segments.add(): startTime should be a valid number

Screenshot from 2020-11-12 15-10-41

jefferya commented 3 years ago

@seanluyk

Summary of what I've found: if the start time and end time are the same exact time then the structural editor with the waveform fails. For example <Span label="Sparshott reads “Untitled”" begin="00:00:10.00" end="00:00:10.00"/>.

Questions:

Details:

I'm able to reproduce in the following manner.

The window closes and no edits are allowed.

seanluyk commented 3 years ago

Thanks for your investigation @jefferya, I believe these are data entry errors, and that they explain why the waveform editor is not working properly. I'll explain this to the users and let you know if any other issues come up!

seanluyk commented 3 years ago

Just took a look at the XML and something weird is going on. If you compare the XML that appears in the source file used for batch ingest of this item with what is now appearing in the advanced structural editor, you'll see some differences:

Original Source XML in Dropbox <Div label="Francis Sparshott"> <Span label="Birney introduces Francis Sparshott" end="0:09:14" begin="0:09:06"/> <Span label="Sparshott introduces and reads “Wasp Winter”" end="0:09:56" begin="0:09:15"/> <Span label="Sparshott reads “Dream Poem”" end="0:10:24" begin="0:09:59"/> <Span label="Sparshott reads “Untitled”" end="0:10:52" begin="0:10:25"/></Div>

What appears in the advanced editor

<Div label="Francis Sparshott">
        <Span label="Birney introduces Francis Sparshott" begin="00:00:09.00" end="00:00:09.00"/>
        <Span label="Sparshott introduces and reads “Wasp Winter”" begin="00:00:09.00" end="00:00:09.00"/>
        <Span label="Sparshott reads “Dream Poem”" begin="00:00:09.00" end="00:00:10.00"/>
        <Span label="Sparshott reads “Untitled”" begin="00:00:10.00" end="00:00:10.00"/>
    </Div>

The spans seem to have been transformed during the ingest process

seanluyk commented 3 years ago

@jefferya just to add to this discussion, we're getting these kinds of errors not just with legacy items (i.e. those created prior to the Avalon upgrade) but also with newly created ones, e.g. https://era-av.library.ualberta.ca/media_objects/c821gm09w

In this case, it's not possible to edit the final timestamp properly and add more

jefferya commented 3 years ago

@seanluyk I've done a superficial investigation.My hypothesis is the input validation for the structural components, in the past, may have missed certain scenarios (e.g., Dec 1st comment) and allowed the input but today, with the current v6.5 Avalon, the input triggers failures. I need to test this more fully. The 6t053g96s example needs a bit more investigation on my end (a hypothesis below)

Details:

Regarding https://era-av.library.ualberta.ca/media_objects/c821gm09w, there's something weird. The last span has an end point of 01:64:0-3586.54 and the Avalon may be having a problem with the time format (I don't recognize the format). The JSON input to the waveform structure editor is the following https://era-av.library.ualberta.ca/master_files/x059c857c/structure.json:

{"type":"span","label":"Jones performs \"At Ease in Order or In Ruins…”","begin":"00:59:52.40","end":"NaN:NaN:NaN"}

Regarding 6t053g96s (above), could the the missing 00 in the input (e.g., 0:09:06) be the problem?

seanluyk commented 3 years ago

@jefferya that weird last span seems to be what Avalon is adding. Even if you try to create an end span that's 1 ms after the begin span, it doesn't let you update it. We've noticed that spans in the new editor tend to be problematic at the end of a recording, so I'm not sure if that's significant. Also of note in c821gm09w is that no XML appears in the "advanced editor". With other problems like these, we've been able to fix them by changing the spans there. Not sure if the XML is supposed to populate from entry in the waveform editor or not?

jefferya commented 3 years ago

@seanluyk clarification, when the structural metadata is added, is it added via the UI (i.e., the waveform editor or the advanced editor)? Or can the structural metadata be added via a batch ingest? Also, is the "upload" UI element used to attach structural metadata? The questions aim to help understand which approaches are used to add structural metadata and which I should focus on when testing.

Second question, is there some preexisting documentation for ERA A+V that outlines the steps to add structural metadata? I'd like my tests to as much as possible follow what that of others. However, if the nothing written exists, I'll follow the direction the UI intuitively takes be based on examples.

c821gm09w: is there an urgent need? There might be way to fix the time via the api as I suspect the problematic time is causing the UI problems (assuming the structural metadata wasn't added via the upload button).

anayram commented 3 years ago

Structural metadata can be added via the UI and also via batch ingest.

anayram commented 3 years ago

Sorry, I sent before completing the message. Structural metadata can be added via the UI and also via batch ingest.

We don't have specific instructions for adding structure metadata for ERA A+V but here are the documentation pages we use:

  1. Adding Structure to Files Using the Graphical XML Editor

  2. Visual Structural Metadata Editor

  3. Adding structure via batch

jefferya commented 3 years ago

@anayram very helpful, thank you.

For testing batch ingest with structural metadata, is there an item in dropbox or google drive that you can point me to as I'd like to grab the batch ingest plus structural metadata file to use for local testing. No rush.

anayram commented 3 years ago

@jefferya no problem, here are the test files I use: https://drive.google.com/drive/u/0/folders/1_knq9iHuJqitilTOCnhHFRxEVcUSuMOS

Use the files highlighted here: image

seanluyk commented 3 years ago

@jefferya we've experienced issues with both structural metadata added via batch and newly created metadata in the UI using the waveform editor. I don't believe we've had many issues when using the advanced editor

jefferya commented 3 years ago

Note:

seanluyk commented 3 years ago

Here are two items where the waveform editor is causing issues:

SW039a1 - (it is listed in ERA-AV as SW039a) https://era-av.library.ualberta.ca/media_objects/c821gm09w SW050b2 - (it is listed in ERA-AV as SW050a) https://era-av.library.ualberta.ca/media_objects/j9602160s

"once I get up to about the 59 minute mark, the autofilled end times and start times begin corrupting to 01:64:0-3586.54"

jefferya commented 3 years ago

Able to reproduce

Tried with upstream version v7.1.1 (test if updated structural metadata editor (SME) JS code fixes problem)

Try update v7.2 (test if updated structural metadata editor JS code fixes problem)

seanluyk commented 3 years ago

That's excellent news, Jeff. I'm hoping that this fixes the issue! Let me know if I can help with any testing

jefferya commented 3 years ago

Diagnostics:

Problematic code:

https://github.com/avalonmediasystem/react-structural-metadata-editor/blame/82a4b6b36406ae117fcf00f70ce9660df07fa2a1/src/services/StructuralMetadataUtils.js#L829

Upstream fixed by:

https://github.com/avalonmediasystem/react-structural-metadata-editor/blame/master/src/services/StructuralMetadataUtils.js#L838

Tried using the following commit:

https://github.com/avalonmediasystem/react-structural-metadata-editor#9e9527eb4c5ae2f17a69441e61955743cb0d5117

Uncaught TypeError: nextProps.structureIsSaved is not a function

Next steps:

jefferya commented 3 years ago

End time bug (3643.35 seconds not properly converted to HH:MM:SS.ss):

Screenshot from 2021-01-26 16-36-46

After React Structural Metadata Editor update with properly converted end time

Screenshot from 2021-01-26 16-26-11

jefferya commented 3 years ago

@seanluyk @anayram To repair the problematic structural metadata produced by the Structural Metadata Editor (SME) in upstream v6.5 once this is in production:

Notes:

seanluyk commented 3 years ago

@jefferya I just did some testing and I'm not sure if this is completely fixed? I was able to add a new timespan to the SpokenWeb example in the object you shared with us, however, the changes don't appear in the advanced (XML) view

jefferya commented 3 years ago

Sean and I met and verified the changes fix the problem.