CRCNS metadata fetching needs fixups

An old API we used is now gone, and we should use the new datacite API

https://api.datacite.org/works?data-center-id=cdl.ucbcrcns&page[size]=1000 is the url to fetch
['data'] contains the list of json records
records seems still to contain 'xml' field, although it seems that json now gets more of metadata fields "natively"

need to figure out exactly how to identify datasets since some times .url could point to it directly but not always:

$> jq ".data[].attributes.url" dump-fresh 
"https://ezid.cdlib.org/id/doi:10.6080/K0PZ5712"
"https://crcns.org/data-sets/ofc/ofc-3"
...
null

so it seems not all have a url pointing to the crcns.org and I do not see "AlternativeTitle" field among json records, so we would be doomed to still parse that xml (need bleeding edge with a patch from a PR to build jq to get @base64d, but then look how lovely it is ;)) :

$> wget -O- -q 'https://api.datacite.org/works?data-center-id=cdl.ucbcrcns&page[size]=1000' | /home/yoh/proj/misc/jq/jq '.data[].attributes| { url:.url, xml:.xml|@base64d|capture(".*AlternativeTitle\">CRCNS.org (?<ds>.*)</title.*") } '                                                                                 
{
"url": "https://ezid.cdlib.org/id/doi:10.6080/K0PZ5712",
"xml": {
"ds": "pfc-7"
}
}
...

so, from above and a brief look at jq '.data[].attributes | .xml |= @base64d' it feels that we just need to keep storing that xml record for now and forget about the json record which seems to contain almost (but not) everything for us

datalad / datalad-crawler

CRCNS metadata fetching needs fixups #15