search

LD4P / sinopia_editor

Sinopia Linked Data Editor
https://sinopia.io/
Apache License 2.0
35 stars 10 forks source link

Investigate Failed MARC conversion #2530

Closed dezel002 closed 4 years ago

dezel002 commented 4 years ago

When I try to use the MARC conversion feature on this instance record: https://api.stage.sinopia.io/resource/57c50ae7-a5e6-4116-8422-f8eb9de61cb1 I get an error message saying "Server error: Ooops, something went wrong: Not a URI".

I am using Sinopia 3.2.1 in Stage on Firefox

image

jermnelson commented 4 years ago

Confirm that using converting this Instance should display an error. Create any follow-up tickets if the rdf2marc needs fixing or improve error reporting.

justinlittman commented 4 years ago

The Primary Contributor for the Work is a literal, not a URI:

{
      "@id": "_:b24",
      "@type": [
        "http://id.loc.gov/ontologies/bibframe/Person"
      ],
      "http://www.w3.org/1999/02/22-rdf-syntax-ns#value": [
        {
          "@value": " dicamillo, kate"
        }
      ]
    },
dezel002 commented 4 years ago

I replaced the literal with the URI from the LCNAF (I am unable to get the authority for Kate DiCamillo to come up in the first 8 hits in QA, so I have to copy and paste it from the NAF). I'm still getting the error, I'm guessing because there is a contributor listed as a translator that does not have an authority record (he does now, but didn't at the time I was cataloging). Since Sinopia allows entering a literal if a URI does not exist for the Agent, should this case be accounted for in the conversion?

michelleif commented 4 years ago

@justinlittman the Sinopia help tries to outline what is required for a conversion to succeed. currently I have the following requirements listed:

So it sounds like we need to add:

Are there other requirements? Do the non-primary Contributors in the Work have to be URIs? The Works' Subjects?

justinlittman commented 4 years ago

Any field that is a URI (either a lookup or URI input), must be a URI, not a literal.

From: michelleif notifications@github.com Sent: Tuesday, September 29, 2020 3:59 PM To: LD4P/sinopia_editor sinopia_editor@noreply.github.com Cc: Justin Littman justinlittman@stanford.edu; Mention mention@noreply.github.com Subject: Re: [LD4P/sinopia_editor] Investigate Failed MARC conversion (#2530)

@justinlittmanhttps://github.com/justinlittman the Sinopia help tries to outline what is required for a conversion to succeed. currently I have the following requirements listed:

So it sounds like we need to add:

Are there other requirements? Do the non-primary Contributors in the Work have to be URIs? The Works' Subjects?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/LD4P/sinopia_editor/issues/2530#issuecomment-700953128, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAEPUL5N7C6YBDPOINW7GZ3SII4C3ANCNFSM4RYORGRQ.

michelleif commented 4 years ago

@dezel002 in the case above where you had to copy and paste a URI from LCNAF rather than finding it in QA (I agree, not ideal), did you get the option to enter it as a new URI, like this?

Screen Shot 2020-09-29 at 1 40 04 PM
dezel002 commented 4 years ago

Yes

michelleif commented 4 years ago

ah ok sorry, didn't read the part about the literal for the translator who didn't have a URI.

back to your question "Since Sinopia allows entering a literal if a URI does not exist for the Agent, should this case be accounted for in the conversion?" there is a ticket here proposing to create a URI out of the user's literal in these cases: https://github.com/LD4P/sinopia_editor/issues/2339

@justinlittman would that meet the converter's requirements?

justinlittman commented 4 years ago

To correctly generate the MARC record, additional metadata about the entity is required. Which metadata depends on the field. Rdf2marc gets that metadata by querying the authority. However, it individually needs to know how to query each authority (right now, it only handles id.loc.gov). So minting a URI does not help.

One possibility is to disallow entering literals (yeah!). A cataloger could create the new entity in Sinopia, e.g., a new Agent record, and use that URI.

dezel002 commented 4 years ago

For the sake of having a successful(ish) MARC conversion, I'll take out the URI-less contributor and see if it works. I may not have time to do this until later today, but will report back.

I have more to say about a potential workflow where an Agent record can be created within Sinopia, but again, I'll wait until I have more time.

dezel002 commented 4 years ago

Did you take out the ability to add a URI? Because now when I go in I can't add the URI for Kate DiCamillo. The option to add a literal or URI is completely gone. I don't see this as a workable solution, as undoubtedly people are going to want to enter Agents that aren't in id.loc.gov (and often even if they are QA doesn't return it) so we need the option to add a URI as a substitute.

michelleif commented 4 years ago

yes that inadvertently disappeared. working on a fix. it was not intentional.

dezel002 commented 4 years ago

I tried again with the new release and a completely different record. I still get the same error message, and I can't figure out where there is a URI missing where it should be.

Record is: https://api.stage.sinopia.io/resource/c45808bc-91d6-4847-96d0-a85ec55edd06

michelleif commented 4 years ago

Manufacture Name is not a URI. @justinlittman could that cause a conversion error?

justinlittman commented 4 years ago

Give it a try now. I've made a few changes:

dezel002 commented 4 years ago

Initially, it did not identify which was the problem (probably because there was more than one?) The issues appear to have been a literal instead of a URI for institution in the Admin Metadata, and a literal subject string. I removed them and got it to work.

But this leaves me with a couple of additional questions. I know subject are being worked on, but should a literal subject string be accepted (since it's accepted on input)? And the 650 fields that were URIs (there were 3 in the record) did not convert, though the 655s (Genre/form terms) did. I guess the second one is more of a comment than a question :)

The literal in Manufacture name was fine, and converted correctly. The publication place (New York, N.Y.) did not convert, even though it was a URI.

Nevertheless, it's a start!

justinlittman commented 4 years ago

On literal subject strings: Unfortunately, from just a string, it cannot be determined which MARC subject field to use. (This is one of many cases in which there is not a clear mapping from allowable Bibframe to MARC.)

justinlittman commented 4 years ago

On 650 fields: They are not currently supported. You can see which fields are supported here: https://github.com/LD4P/rdf2marc/blob/master/lib/rdf2marc/model2marc/record.rb

justinlittman commented 4 years ago

On publication place: The URI is https://lccn.loc.gov/n79007751. Currently, only https://id.loc.gov URIs are supported.

dezel002 commented 4 years ago

Okay, thanks. Subject strings are not ideal in a linked data environment, anyway, but I thought I would try it out. Using FAST or other faceted vocabularies (or single-element LSCH) is probably a better option.

Is there a reason 650 was omitted, when the other 6XX fields were included in the conversion? Is it because of the complications with LCSH strings, or was it an oversight? It's a core MARC field that catalogers would expect to be converted.

dezel002 commented 4 years ago

I will try to remember to get my alternate URIs from id.loc.gov.

justinlittman commented 4 years ago

Our initial focus was on an "operational" record. I don't believe 650 was in that specification. However, this is really a question for @NancyL or @vitustang.

This is a bit muddied because I did some of this work before the operation record specification was prepared, so includes some fields that go beyond the spec.

dezel002 commented 4 years ago

Okay, thank you. I will close this ticket since I was able to successfully convert to MARC. And I learned a lot! Thanks for your help.

vitustang commented 4 years ago

650 is in the Operational record's specificationshttps://docs.google.com/spreadsheets/d/1Cg0aXgh4fqFTLkxw0D9xeiHrYFqHPI3xgt666CqZ-lY/edit#gid=556150058:

6xx -- Convert all that have id.loc.gov URI. 2nd indicator all "0"

-- Vitus

From: Justin Littman notifications@github.com Reply-To: LD4P/sinopia_editor reply@reply.github.com Date: Wednesday, October 7, 2020 at 5:20 AM To: LD4P/sinopia_editor sinopia_editor@noreply.github.com Cc: Vitus Tang vtang@stanford.edu, Mention mention@noreply.github.com Subject: Re: [LD4P/sinopia_editor] Investigate Failed MARC conversion (#2530)

Our initial focus was on an "operational" record. I don't believe 650 was in that specification. However, this is really a question for @NancyLhttps://github.com/NancyL or @vitustanghttps://github.com/vitustang.

This is a bit muddied because I did some of this work before the operation record specification was prepared, so includes some fields that go beyond the spec.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/LD4P/sinopia_editor/issues/2530#issuecomment-704897337, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AG4YADJMPGLQH7FORJF6ONLSJRMKHANCNFSM4RYORGRQ.

justinlittman commented 4 years ago

@vitustang @dezel002 Can you provide some examples of an id.loc.gov URIs that should get mapped to a 650?

dezel002 commented 4 years ago

On Stage, https://api.stage.sinopia.io/resource/2fb50552-91b3-43a9-ab8e-8a2fa71adb91 has http://id.loc.gov/authorities/subjects/sh85073889

justinlittman commented 4 years ago

FYI @vitustang @dezel002 650s (including complex subjects) are now handled. This has been deployed to stage.