Missing procedures from docdb

[mekhlakapoor]

Describe the bug There are ~25 data assets in docdb that have procedures, but when those subject_ids are input into the metadata-service, they have no procedures (or some of the procedures are missing). Look into what could be causing this.

To Reproduce Steps to reproduce the behavior:

1. Look up and compare 614608 in docdb vs prcoedures endpoint

Expected behavior A clear and concise description of what you expected to happen.

Screenshots

Desktop (please complete the following information):

• OS: [e.g. iOS]
• Browser [e.g. chrome, safari]
• Version [e.g. 22]

Smartphone (please complete the following information):

• Device: [e.g. iPhone6]
• OS: [e.g. iOS8.1]
• Browser [e.g. stock browser, safari]
• Version [e.g. 22]

Additional context Speak with Saskia and Mae about this to get list of subject_ids. I believe there may have been a breaking change in NSB, so check for updates in the NSB schema model

[jtyoung84]

Let's figure out what the root issue is and we can scope out the fix

[Sun-flow]

referenced asset has been giving an internal server error for ~2-3 weeks now when looking it up in metadata service. looks fine in docdb.

[Sun-flow]

Issue assets were originally identified because they contained procedures that lacked a procedure_type field. This occurred at the general Procedures level (which usually contains surgeries), not at the specific procedure level.

[Sun-flow]

A number of assets turn up with the filter query: {$and: [{"procedures.subject_procedures.procedure_type": {"$exists": false}}, {"procedures.subject_procedures": {"$ne": []}}, {"procedures.subject_procedures": {"$ne": null}}]} in mongoDB.

When looking up the corresponding subject IDs in metadata service, a number of assets return different procedures. Usually, the docdb asset contains a single incomplete subject procedure from NSB, while the data asset in the metadata service contains multiple surgeries at varying levels of completeness, such as containing multiple surgeries with no listed procedures.

[Sun-flow]

List of subjects found using this method: ['689383', '675481', '665840', '665840', '665847', '688267', '689383', '675481', '677330', '665840', '665840', '686314', '688267', '690487', '669705', '669702', '669702', '689383', '669705', '677330', '688267', '690487', '665847', '688267', '669702', '669702', '686314']