mbjones
commented
3 years ago I worked on a proof of concept for a cite()-like method. For DataONE-related identifiers to understand a bit better what is involved. The challenges were:
1) Identifying which datasets came from DataONE based on the data access URI alone returned from query_sourcea()
2) Determining which metadata document is the most appropriate for a given contentid
3) walking the version chain of each of those efficiently
4) getting bibliographic metadata in the right form for constructing a citation
5) Flexible formatting of citations following CSL and bibtex, for example
The following function works partially for DataONE. It would be better implemented as an adapter for each known registry that knows how to create the citation given the repository-specific APIs available.
library(dataone)
library(solrium)
# Function to return the text of a citation for a given contentid
get_citation <- function(contentid) {
# Set up for SOLR queries
solr <- SolrClient$new(host = "cn.dataone.org", path = "/cn/v2/query/solr/", scheme = "https", port=443)
# Use query_sources to determine if a copy is on DataONE or amenable repository (based on the URL pattern)
d1_locations <- query_sources(contentid, cols=c("identifier", "source", "date", "status", "sha1", "sha256", "md5")) %>%
filter(grepl('cn.dataone.org|v2/object|v2/resolve', source))
if (nrow(d1_locations) > 0) {
# Query that network API to determine which Datasets the content identifier is associated with
# If more than one is found, reduce the list to the most recent version of each Dataset
# (eliminating duplicate older versions, in favor of citing the most recent)
# Look up the metadata for this object, including which metadata documents describe it in DataONE
pids <- d1_locations$source %>% basename() %>% unique()
subquery_pids <- stringr::str_replace_all(stringr::str_flatten(pids, collapse=" OR "), ":", "\\\\:")
subquery <- paste0("id:(", subquery_pids, ") AND -obsoletedBy:*")
fields <- 'identifier,checksum,checksumAlgorithm,datasource,isDocumentedBy,resourceMap'
metadata <- solr$search(params = list(q=subquery, rows=100, fl=fields))
# Retrieve the bibliographic metadata for each of those datasets by searching on the PID for each
documented_by <- stringr::str_split(metadata$isDocumentedBy, ",")
subquery_ids <- stringr::str_replace_all(stringr::str_flatten(documented_by[[1]], collapse=" OR "), ":", "\\\\:")
subquery <- paste0("id:(", subquery_ids, ")")
fields <- paste(sep=",", 'origin,identifier,formatId,checksum,checksumAlgorithm,title,datePublished,pubDate',
'datasource,obsoletes, obsoletedBy, isDocumentedBy,resourceMap')
datasets <- solr$search(params = list(q=subquery, rows=100, fl=fields))
# Determine the repository name and URI
repos <- listNodes(CNode())
repo_list <- repos[sapply(repos, function(repo) { repo@identifier==datasets$datasource[[1]]}, simplify = TRUE ) ]
# Return a list of citations, one for each matching Dataset (possibly providing different formats
# for returning the citation information (text string, bibtex, CSL-formatted string))
# Currently this only returns the first result as a POC, needs work
citation_text <- paste0(datasets$origin[[1]], ". ", format(as.Date(datasets$pubDate[[1]]), "%Y"), ". ",
datasets$title[[1]], ". ", repo_list[[1]]@name, ". ", datasets$identifier, " ",
paste0("https://search.dataone.org/view/", datasets$identifier[[1]]))
return(citation_text)
} else {
return(list())
}
}
jhpoelen
cboettig
Now that we can resolve to content in data repositories such as DataONE network or Zenodo, it might make sense to also be able to report citation information from those repositories.
With Zenodo, queries always match hash to the record containing the matching file, and the DOI is readily available (
doi_urlfield), as is remaining citation metadata. With DataONE, not all objects have DOI and tracking from PID to related DOI is a bit less obvious, but all information is still present there somewhere.Because the same file can be part of multiple packages (very common in 'versioned series', but also other cases), an object may have more than one DOI. In a versioned series, it would make sense to cite the most recent version in the series containing the exact match. In other cases, it may make more sense to merely return all possible citations and leave resolution to the user.
Unclear what citation should be returned for content that only resolves to other sources (URLs from hash-archive records, software heritage archives, etc). Some sources (e.g. Software Heritage) are arguably still 'citable', though ideally would resolve to the parent DOI if also found on an archive. For content resolvable by hash-archive alone, we could arguably then check the other algo ids to see if content is also on DataONE. (i.e. the sha256 id resolves only to a URL from hash-archive, but from that we get the md5 and discover that one resolves on DataONE. also, many URLs on hash-archive will already point to download URLs from DataONE or Zenodo).
Concerns:
Sometimes content will fail to resolve to a scientific archive even when it is available there, because DataONE and Zenodo recognize only some hashes (e.g. md5). Need to avoid user confusion that can arise here. (This deviates from other
contentidfunctions that are location-agnostic, forcite()to work, suddenly we need to resolve to a specific location....)Counter-argument: Perhaps the use-case for generating a citation given hash-identifier is poorly motivated, in that it seems unlikely that a user would somehow know the content-identifier for content without already knowing other information about it. e.g. a typical workflow might be: read paper or search DataONE catalog to discover data, copy content identifier down, use it to embed in code. Generating the bibtex might be convenient, but is arguably out-of-scope. (However, we can 'resolve' citation information for a DOI, so the analogy begs being able to do so for a content id?)
Idea for
cite()suggested originally by @mbjones.