Closed attwad closed 3 years ago
Here's an actual document schema error:
{
"name": "BulkWriteError",
"driver": true,
"code": 121,
"writeErrors": [
{
"code": 121,
"index": 0,
"errmsg": "Document failed validation",
"op": {
"q": {
"caseReference.sourceId": "5f5b5ae6def27b58cfb60471",
"caseReference.sourceEntryId": "1"
},
"u": {
"$set": {
"revisionMetadata": {
"revisionNumber": 0,
"creationMetadata": {
"curator": "ingestion@covid-19-map-277002.iam.gserviceaccount.com",
"date": "2020-09-11T12:24:08.719Z"
}
},
"symptoms": {
"values": [],
"status": "Symptomatic"
},
"notes": "Imported case",
"events": [
{
"name": "confirmed",
"dateRange": {
"start": "2020-01-23T00:00:00.000Z",
"end": "2020-01-23T00:00:00.000Z"
}
},
{
"name": "onsetSymptoms",
"dateRange": {
"start": "2020-01-21T00:00:00.000Z",
"end": "2020-01-21T00:00:00.000Z"
}
},
{
"name": "outcome",
"value": "Recovered"
}
],
"demographics": {
"nationalities": [],
"gender": "Male",
"ageRange": {
"start": 39,
"end": 39
}
},
"location": {
"country": "China",
"administrativeAreaLevel1": "Hong Kong",
"geoResolution": "Admin1",
"name": "Hong Kong",
"geometry": {
"longitude": 114.15861,
"latitude": 22.27833
}
},
"caseReference": {
"uploadIds": [
"5f5b6c51d5b11efa26bae2c5"
],
"verificationStatus": "UNVERIFIED",
"sourceId": "5f5b5ae6def27b58cfb60471",
"sourceEntryId": "1",
"sourceUrl": "http://www.chp.gov.hk/files/misc/enhanced_sur_covid_19_eng.csv",
"additionalSources": []
}
}
},
"multi": false,
"upsert": true
}
}
]
}
I can't figure out what's wrong with this document tbh...
The document is valid, we just need to push the new schema to prod, @axmb will revive a script we had to do that and push whenever convenient.
Also as for the original issue that I reported, the individual errors are logged by the lambda (this is how I got the example above), this is good enough, I don't think we want to keep those in mongo.
Fixed by #997
Looking at prod logs, I see:
We're not looking at writeErrors so all these details end up being swallowed :(
FYI @axmb