Spike on exclusion of certain collections from DOI generation

[pbinkley]

• Investigate in code and whiteboard for the dev team
• Implement, if the decision is to go ahead

Note: _export field allows us to say yes or no to export and indexing of a DOI. We may use this to suppress dissemination of DOIs for private/ualberta/dark items. In that case, we would need to address the change of status of items: e.g. to turn on _export when an item moves from private to public.

[pgwillia]

Took a quick look at https://github.com/mlibrary/umrdr -- implemented DOIs using ezid-client

• DOIMintingJob calls DOIMintingService
• DOIMintingJob is scheduled by GenericWorks controller when #identifiers is called

[pgwillia]

Comments on unfactored ark code: https://github.com/ualbertalib/HydraNorth/commit/31352953f85d7e71ecf9fd998f530c0063785856

• using ezid-client 1.2.0 -- 1.5.0 is available
• EzidService is called by Hydranorth::GenericFile::Actor, BatchUpdateJob and solr reindex tasks
• expects ark_id
  • RedirectController
  • Hydranorth::GenericFile::Export
  • will show if it exists in views/generic_files/show actions.html.erb

[pgwillia]

Assertion from stakeholders

• all works need an ARK for preservation
• most works having a DOI would be valuable for citation

Assertions from EZid

• DOI create shadow ARK
• identifiers status can change
  • shadow ark shares the same status
  • => if one deletes a reserved DOI the shadow ARK is gone too
  • if the status of the shadow ARK is changed, so is the DOI
  • status can be
  • reserved (only at creation time)
  • public (default)
  • unavailable (displays citation metadata on tombstone page + user given reason for unavailability)

Questions

• does a reserved DOI create a shadow ark that can be used for preservation?
• do all works need a 'public' status identifier for preservation?
• is hiding DOI in the public interface + setting _export: no sufficient?
  • OAI
  • endnote export
• where/when does an admin set exclusion?
  • Collection edit form?
  • on batch migration?
  • on individual items?
• can anyone that's not admin set exclusion?
  • => cancan ability
• when/where are DOI's minted?
  • async job+service like umich?
  • with metadata creation of the object
  • when file is uploaded?
• does DOI minting or setting collection happen first for new items?
• can we enumerate in advance which collections need to be excluded?
  • is this likely to change?

# helper method that DOI code might call to decide if a DOI needs minting
def needs_doi?
end

• DOIs for all with hiding
  • would allow for retroactive suppression
• DOIs for some ARKs for others
  • could be inconsistent
• public DOIs for some reserved DOIs for others
  • reserved DOIs can be deleted +/-

[pgwillia]

Some ezid trial and error https://gist.github.com/pgwillia/70ceca2f31ae72ba51dc7e3686e347ec

[pgwillia]

So I've got questions (see above) that probably need answering in order to do this right. Some for DOI implementers @mbarnett & @murny some for preservation experts @pbinkley & @umarqasim, some for service managers @leahvanderjagt and some for metadata @johnhuck et al. Can we break out for a quick conversation after standup tomorrow?

[johnhuck]

@pgwillia, yes, absolutely, although isn't tomorrow sprint review? Regardless, I will make myself available.

[pgwillia]

https://docs.google.com/a/ualberta.ca/document/d/1KYkFjz7qqoGthslsiILVcDkULUDvPKmSusy7E-2C3z8/edit?usp=sharing

[pgwillia]

Thanks for talking this through with me! Here's what I heard decided:

We will:

• not use identifiers with the reserved status
• unavailable or public DOI (and shadow ARK) is sufficient for local preservation
• in general we will give everything a EZid registered DOI (and shadow ARK)
• exclude Dataverse items by not assigning DOIs
• exclude Private items by assigning identifiers with unavailable status
  • use :unal (unallowed) datacite metadata values to mask the private metadata (Metadata team to confirm)
  • set _export to no so that harvesters of EZid will not see these items
• manage exclusion of individual items through the rails console or rake task/migration
• display on item view
  • private DOI (which would only be viewable by the depositor or admin)
  • public DOI
  • Dataverse DOI (in a different location)
• display on endnote export
  • public DOI
  • otherwise current Rails.application.routes.url_helpers.generic_file_url(id)
• display in OAI
  • public DOI
  • Dataverse DOI
  • private items are irrrelevant

We still need to think about:

• whether metadata only records need identifiers and their special cases
• any TDR implications
• private because individual requests, copyright investigation and licensed materials policy
• next phase(s) content

[johnhuck]

@pgwillia The final decision about what :erc code to use to suppress metadata for private items has not been made. :unal is likely, but I would like a chance to consider the options.

[pgwillia]

@johnhuck -- apologies. I jumped the gun. Probably makes sense to communicate your decision on #1283

[mbarnett]

Yeah, #1283 is probably a good place for that. @murny is knee-deep in implementing this so I imagine he'll need a decision by the end of this sprint at the latest.

[johnhuck]

No worries. I will aim to have a decision within the next day or so. I'll add the information to #1283.

Another thought: we have the opportunity to supply a short explanatory message when setting the unavailable status for private items. Is that something we should try to decide now with the stakeholders, or leave for the moment?

[mbarnett]

Good question. We could probably hammer that out now.

[johnhuck]

I'll send an email to Sonya, Leah, and Sharon to get their input on the wording.