error with new version and DSA-110 application

[caseyjlaw]

I am updating my archiving tools based on our conversation about the DSA-110 application. I’ve updated to the latest version of caltechdata_api on github (looks like a commit on Oct 25). After the update, I can see the “publish” argument for the caltechdata_write/edit functions. However, when using the production system to create an unpublished event, I get this error:

> caltechdata_write(metadata, caltechdata.token, production=True, schema="43", publish=False)
Unknown Type mapping GRID
{"metadata": {"identifiers": [], "alternateIdentifiers": [{"alternateIdentifier": "210810aaaa", "alternateIdentifierType": "DSA-100 internal id"}], "dates": [{"date": "2022-10-28", "type": {"id": "created"}}], "formats": ["zip", "png"], "publisher": "Caltech Data", "version": "0", "mjds": 59436.59969748689, "snr": 9.03739, "dm": 29.0454, "ra": 0.0, "dec": 0.0, "internalname": "210810aaaa", "width": 1.048576, "radecerr": 1, "resource_type": {"id": "dataset"}, "creators": [{"affiliations": [{"id": "05dxps055", "name": "Caltech"}], "role": {"id": "contactperson"}, "person_or_org": {"name": "Casey Law", "type": "personal", "given_name": "Casey", "family_name": "Law", "identifiers": [{"schemeUri": "https://orcid.org", "identifier": "0000-0002-4119-9963", "scheme": "orcid"}]}}], "title": "DSA-110 Data for Candidate Fast Radio Burst 210810aaaa", "description": "Data associated with DSA-110 candidate transient. Zipped file contains (1) a filterbank file with burst dynamic spectrum and (2) a FITS file with localization region. Second file is a png image showing summary views of data used to identify the candidate.", "publication_date": "2022", "subjects": [{"subject": "OVRO"}, {"subject": "Astrophysics"}, {"subject": "Fast Radio Bursts"}], "languages": [{"id": "en"}], "related_identifiers": [{"identifier": "http://deepsynoptic.org", "scheme": "url", "relation_type": {"id": "isdocumentedby"}}], "rights": [{"title": {"en": "CC By 4.0"}}], "locations": {"features": [{"geometry": {"type": "Point", "coordinates": ["37.2339", "-118.282"]}, "place": "OVRO"}]}, "funding": [{"funder": {"name": "National Science Foundation"}}]}, "pids": {"doi": {"identifier": "setDOI", "provider": "external"}}, "parent": {}, "files": {"enabled": false}}
{
  "message": "Referer checking failed - no Referer.",
  "status": 400
}

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
Input In [11], in <module>
----> 1 caltechdata_write(
      2     metadata, caltechdata.token, production=True, schema="43", publish=False
      3 )

File ~/anaconda3arm/envs/py38/lib/python3.8/site-packages/caltechdata_api/caltechdata_write.py:132, in caltechdata_write(metadata, token, files, production, schema, publish, file_links, s3, community)
    130 if result.status_code != 201:
    131     print(result.text)
--> 132     exit()
    133 idv = result.json()["id"]
    134 print(f"record {idv} created")

NameError: name 'exit' is not defined

I can't find documentation of the message from the API. Is my metadata somehow incorrect or incomplete? Also, when I try the same with production=False, I get a timeout on the API call. How can I check that I have an account on the test system?

[tmorrell]

Interesting- I've used that exit syntax for awhile and haven't run into problems before. I've switched everything to Exceptions, which should be more standard. Please grab the latest version of main and see if that fixes the problem.

We're still working on re-launching the test version of CaltechDATA, so it's expected that production=False will timeout. I'll let you know when the test version is back online.

[caseyjlaw]

Well, I think it is two issues right? The exit doesn't work, but that's called because the API query doesn't have a status_code=201. Incidentally, your latest change has Exceltion rather than Exception.

[tmorrell]

Thanks for the catch, fixed. Yeah, there very well might be an error in your metadata. But we need to see what the API returns to debug the problem.

[tmorrell]

Oh...other potential reason for the 400. Have you gotten a new token for data.caltech.edu? Ones from the old system won't work.

[caseyjlaw]

Ah, that is probably it. I'll try again with a new token.

[caseyjlaw]

Ok, a new token is working for me. One last issue was that my metadata had a field "language" and it was set to "en". That produced a 201 status from the API request. Removing it for now.

[tmorrell]

Awesome! The new version of CaltechDATA uses three letter codes for languages ('eng'), but it's also fine if you leave that field off. Let me know if you run into any more issues.