parse new fields from Submission's metadata blob to put into NIHMS package

[htpvu]

This is related to this issue in the pass-ember repos https://github.com/OA-PASS/pass-ember/issues/637

@jabrah will implement necessary API calls to get information about nlmta and issn+pubtype and put them in the metadata-blob on submission.

Please implement functionality to parse out these fields from the blob and put them into the NIHMS packages according to the attached spec doc:

Bulk submission specs_Funding Agency_July 2017.pdf

[emetsger]

Documenting here, the new metadata format to parse:

[
   {
      "id":"JScholarship",
      "data":{
         "embargo":"NON-EXCLUSIVE LICENSE FOR USE OF MATERIALS This non-exclusive license defines the terms for the deposit of Materials in all formats into the digital repository of materials collected, preserved and made available through the Johns Hopkins Digital Repository, JScholarship. The Contributor hereby grants to Johns Hopkins a royalty free, non-exclusive worldwide license to use, re-use, display, distribute, transmit, publish, re-publish or copy the Materials, either digitally or in print, or in any other medium, now or hereafter known, for the purpose of including the Materials hereby licensed in the collection of materials in the Johns Hopkins Digital Repository for educational use worldwide. In some cases, access to content may be restricted according to provisions established in negotiation with the copyright holder. This license shall not authorize the commercial use of the Materials by Johns Hopkins or any other person or organization, but such Materials shall be restricted to non-profit educational use. Persons may apply for commercial use by contacting the copyright holder. Copyright and any other intellectual property right in or to the Materials shall not be transferred by this agreement and shall remain with the Contributor, or the Copyright holder if different from the Contributor. Other than this limited license, the Contributor or Copyright holder retains all rights, title, copyright and other interest in the images licensed. If the submission contains material for which the Contributor does not hold copyright, the Contributor represents that s/he has obtained the permission of the Copyright owner to grant Johns Hopkins the rights required by this license, and that such third-party owned material is clearly identified and acknowledged within the text or content of the submission. If the submission is based upon work that has been sponsored or supported by an agency or organization other than Johns Hopkins, the Contributor represents that s/he has fulfilled any right of review or other obligations required by such contract or agreement. Johns Hopkins will not make any alteration, other than as allowed by this license, to your submission. This agreement embodies the entire agreement of the parties. No modification of this agreement shall be of any effect unless it is made in writing and signed by all of the parties to the agreement.",
         "agreement-to-deposit":"true"
      }
   },
   {
      "id":"common",
      "data":{
         "title":"Specific protein supplementation using soya, casein or whey differentially affects regional gut growth and luminal growth factor bioactivity in rats; implications for the treatment of gut injury and stimulating repair",
         "journal-title":"Food & Function",
         "volume":"9",
         "issue":"1",
         "ISSN":"2042-6496,2042-650X",
         "abstract":"Differential enhancement of luminal growth factor bioactivity and targeted regional gut growth occurs dependent on dietary protein supplement.",
         "authors":[
            {
               "author":"Tania Marchbank",
               "orcid":"http://orcid.org/0000-0003-2076-9098"
            },
            {
               "author":"Nikki Mandir"
            },
            {
               "author":"Denis Calnan"
            },
            {
               "author":"Robert A. Goodlad"
            },
            {
               "author":"Theo Podas"
            },
            {
               "author":"Raymond J. Playford",
               "orcid":"http://orcid.org/0000-0003-1235-8504"
            }
         ],
         "issn-map":{
            "2042-6496":{
               "pub-type":[
                  "Print"
               ]
            },
            "2042-650X":{
               "pub-type":[
                  "Electronic"
               ]
            }
         }
      }
   },
   {
      "id":"crossref",
      "data":{
         "doi":"10.1039/c7fo01251a",
         "publisher":"Royal Society of Chemistry (RSC)",
         "journal-title-short":"Food Funct."
      }
   },
   {
      "id":"pmc",
      "data":{
         "nlmta":"Food Funct"
      }
   }
]

[emetsger]

See also #119, as we have to now reconcile the metadata from #119 with what is in here (e.g. embargo metadata).

[emetsger]

Unified blob documented on #119, used for all the tests:

[
  {
    "id":"common",
    "data":{
      "title":"Electrogenerated chemiluminescence of Ru(bpy)32+ at a black phosphorus quantum dot modified electrode and its sensing application",
      "journal-title":"The Analyst",
      "volume":"143",
      "issue":"1",
      "ISSN":"0003-2654,1364-5528",
      "abstract":"Strong anodic electrogenerated chemiluminescence (ECL) of Ru(bpy)32+ can be obtained under neutral conditions at a black phosphorus quantum dot (BPQD) modified electrode due to the catalytic effect of BPQDs. Dopamine exhibits an apparent inhibiting effect on the ECL signal, and as a result, can be sensitively detected.",
      "authors":[
        {
          "author":"Lei Zhang"
        },
        {
          "author":"KaiJin Tian"
        },
        {
          "author":"YongPing Dong",
          "orcid":"http://orcid.org/0000-0002-0455-9982"
        },
        {
          "author":"HouCheng Ding"
        },
        {
          "author":"ChengMing Wang"
        }
      ],
      "Embargo-end-date":"2018-06-30"
    }
  },
  {
    "id":"crossref",
    "data":{
      "doi":"10.1039/c7an01617d",
      "publisher":"Royal Society of Chemistry (RSC)",
      "journal-title-short":"Analyst"
    }
  },
  {
    "id":"JScholarship",
    "data":{
      "embargo":"NON-EXCLUSIVE LICENSE FOR USE OF MATERIALS This non-exclusive license defines the terms for the deposit of Materials in all formats into the digital repository of materials collected, preserved and made available through the Johns Hopkins Digital Repository, JScholarship. The Contributor hereby grants to Johns Hopkins a royalty free, non-exclusive worldwide license to use, re-use, display, distribute, transmit, publish, re-publish or copy the Materials, either digitally or in print, or in any other medium, now or hereafter known, for the purpose of including the Materials hereby licensed in the collection of materials in the Johns Hopkins Digital Repository for educational use worldwide. In some cases, access to content may be restricted according to provisions established in negotiation with the copyright holder. This license shall not authorize the commercial use of the Materials by Johns Hopkins or any other person or organization, but such Materials shall be restricted to non-profit educational use. Persons may apply for commercial use by contacting the copyright holder. Copyright and any other intellectual property right in or to the Materials shall not be transferred by this agreement and shall remain with the Contributor, or the Copyright holder if different from the Contributor. Other than this limited license, the Contributor or Copyright holder retains all rights, title, copyright and other interest in the images licensed. If the submission contains material for which the Contributor does not hold copyright, the Contributor represents that s/he has obtained the permission of the Copyright owner to grant Johns Hopkins the rights required by this license, and that such third-party owned material is clearly identified and acknowledged within the text or content of the submission. If the submission is based upon work that has been sponsored or supported by an agency or organization other than Johns Hopkins, the Contributor represents that s/he has fulfilled any right of review or other obligations required by such contract or agreement. Johns Hopkins will not make any alteration, other than as allowed by this license, to your submission. This agreement embodies the entire agreement of the parties. No modification of this agreement shall be of any effect unless it is made in writing and signed by all of the parties to the agreement.",
      "agreement-to-deposit":"true"
    }
  }
]

[emetsger]

Unifying the blobs from this issue and #119 gives this blob, which will be parsable by this PR:

[
   {
      "id":"JScholarship",
      "data":{
         "embargo":"NON-EXCLUSIVE LICENSE FOR USE OF MATERIALS This non-exclusive license defines the terms for the deposit of Materials in all formats into the digital repository of materials collected, preserved and made available through the Johns Hopkins Digital Repository, JScholarship. The Contributor hereby grants to Johns Hopkins a royalty free, non-exclusive worldwide license to use, re-use, display, distribute, transmit, publish, re-publish or copy the Materials, either digitally or in print, or in any other medium, now or hereafter known, for the purpose of including the Materials hereby licensed in the collection of materials in the Johns Hopkins Digital Repository for educational use worldwide. In some cases, access to content may be restricted according to provisions established in negotiation with the copyright holder. This license shall not authorize the commercial use of the Materials by Johns Hopkins or any other person or organization, but such Materials shall be restricted to non-profit educational use. Persons may apply for commercial use by contacting the copyright holder. Copyright and any other intellectual property right in or to the Materials shall not be transferred by this agreement and shall remain with the Contributor, or the Copyright holder if different from the Contributor. Other than this limited license, the Contributor or Copyright holder retains all rights, title, copyright and other interest in the images licensed. If the submission contains material for which the Contributor does not hold copyright, the Contributor represents that s/he has obtained the permission of the Copyright owner to grant Johns Hopkins the rights required by this license, and that such third-party owned material is clearly identified and acknowledged within the text or content of the submission. If the submission is based upon work that has been sponsored or supported by an agency or organization other than Johns Hopkins, the Contributor represents that s/he has fulfilled any right of review or other obligations required by such contract or agreement. Johns Hopkins will not make any alteration, other than as allowed by this license, to your submission. This agreement embodies the entire agreement of the parties. No modification of this agreement shall be of any effect unless it is made in writing and signed by all of the parties to the agreement.",
         "agreement-to-deposit":"true"
      }
   },
   {
      "id":"common",
      "data":{
         "title":"Specific protein supplementation using soya, casein or whey differentially affects regional gut growth and luminal growth factor bioactivity in rats; implications for the treatment of gut injury and stimulating repair",
         "journal-title":"Food & Function",
         "volume":"9",
         "issue":"1",
         "ISSN":"2042-6496,2042-650X",
         "abstract":"Differential enhancement of luminal growth factor bioactivity and targeted regional gut growth occurs dependent on dietary protein supplement.",
         "authors":[
            {
               "author":"Tania Marchbank",
               "orcid":"http://orcid.org/0000-0003-2076-9098"
            },
            {
               "author":"Nikki Mandir"
            },
            {
               "author":"Denis Calnan"
            },
            {
               "author":"Robert A. Goodlad"
            },
            {
               "author":"Theo Podas"
            },
            {
               "author":"Raymond J. Playford",
               "orcid":"http://orcid.org/0000-0003-1235-8504"
            }
         ],
         "Embargo-end-date":"2018-06-30",
         "issn-map":{
            "2042-6496":{
               "pub-type":[
                  "Print"
               ]
            },
            "2042-650X":{
               "pub-type":[
                  "Electronic"
               ]
            }
         }
      }
   },
   {
      "id":"crossref",
      "data":{
         "doi":"10.1039/c7fo01251a",
         "publisher":"Royal Society of Chemistry (RSC)",
         "journal-title-short":"Food Funct."
      }
   },
   {
      "id":"pmc",
      "data":{
         "nlmta":"Food Funct"
      }
   }
]

[jabrah]

Given our discussion about the issn-map and the Linking pub-type, the UI will need to do some fairly trivial changes. Does it matter how some of these cases are represented in the metadata blob? None of the options are really too different, just want to make sure we pick the easiest one to parse

Possible changed scenarios

Most importantly, the overall structure will remain basically untouched. We do plan on including all ISSNs that this external service provides, except the ones with pub-type=Linking. It is possible that the journal info service returns data about a given journal and it contains an ISSN that has only Linking as it's pubtype. There are a few ways I can represent this data:

Say a given dataset has two ISSNs, 2451-9030 with pub-type=Electronic and 2451-9022 with pub-type=Linking. Does it matter how the front end hands this data to deposit services?

1. Simply not include the 2nd ISSN
2. Include the 2nd ISSN as it would be in the current code: as an empty array "issn": { "pub-type": []}
3. Include the 2nd ISSN with pub-type Linking "issn": { "pub-type": ["Linking"]}. I don't think this one would be a great idea, as it would mean some ISSNs would have multiple values in the pub-type array.

"issn-map": {
  "2451-9030": { "pub-type":["Electronic"] },
  // 1
  "2451-9022": { "pub-type":[] }, // 2
  "2451-9022": { "pub-type":["Linking"] }, // 3
}

[emetsger]

@jabrah Ok, let me see if I understand.

Given a journal with issn 2451-9030, pub-type=Electronic and issn 2451-9022 with pub-type=Linking, the question you are asking is: "How to represent this in the ISSN map"?

Would this work?

 "issn-map":{
            "2451-9030":{
               "pub-type":[
                  "Electronic"
               ]
            },
            "2451-9022":{
               "pub-type":[
                  "Linking"
               ]
            }
         }

Can an ISSN have multiple pub-types?

[emetsger]

@jabrah if we think it is important to include the Linking pub-type for completeness in the blob, Deposit Services can be configured to only process Electronic and Print types (and ignore any other types). Especially now that we have some breathing room.

[jabrah]

@emetsger

• Can ISSN have multiple pub-types? Yes - if we include Linking. There are cases, for example, where a single ISSN can have pub-type: [ "Print", "Linking" ]
• Is it important to include Linking in the blob for completeness? I don't think so. If we need to make this change in the future, ideally we would have more time to think about and work on it. Including Linking would mean DS would have to know to ignore it, and DS would have to support multiple pub-types for ISSNs

I would personally lean towards simply dropping any Linking in the UI - not including it in the blob

[emetsger]

I would personally lean towards simply dropping any Linking in the UI - not including it in the blob

👍Sounds great to me! The simplest thing.

Given a journal with issn 2451-9030, pub-type=Electronic and issn 2451-9022 with pub-type=Linking, the issn-map would be the following, correct?

 "issn-map":{
            "2451-9030":{
               "pub-type":[
                  "Electronic"
               ]
            }
         }

[jabrah]

Exactly

[emetsger]

Closed, implemented in #121.