search

populationgenomics / metamist

Sample level metadata system
MIT License
1 stars 1 forks source link

Fixes to seqr sync #772

Closed EddieLF closed 2 months ago

EddieLF commented 2 months ago

Attempt to resolve https://github.com/populationgenomics/metamist/issues/760

The API endpoint "Get individual metadata template for Seqr" has not been working properly, since the headers value returned by the db participant layer get_seqr_individual_template is returning an empty list. This is because the set_headers contains the snake_case header names, like 'family_id' and 'individual_id', while the values from SeqrMetadataKeys.get_ordered_headers() are the human readable header names, like 'Family ID' and 'Individual ID'.

This change utilises the lheader_to_json dictionary constructed earlier in the function to get the snake_case header names from the human readable header names. This way, the headers returned will not be empty, but a list of the snake_case headers that are keys in the dicts in rows.

As mentioned in the issue linked above, it's unclear why this has become a problem now, seeing as the code looks to be unchanged for years. I may be totally misunderstanding something but it definitely looks like the headers should be a list of snake case headers as they appear in rows.

There is a second change I'm sneaking in here, which is adding @backoff to the seqr sync API call to update saved variants. This call often succeeds and the Seqr VM logs show a 200 response, however Metamist frequently receives a 502 error response and outputs an error when reporting the sync results. By adding the backoff and retry, these transient 502 errors should mostly go away.

I'm seeing that the @backoff library is not found and causing the unit tests / linting to fail. Why is this? It also gets imported and used here without issue.

codecov-commenter commented 2 months ago

Codecov Report

Attention: Patch coverage is 33.33333% with 4 lines in your changes are missing coverage. Please review.

Project coverage is 77.13%. Comparing base (f1b9811) to head (d9fff56).

Files Patch % Lines
db/python/layers/participant.py 0.00% 4 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## dev #772 +/- ## ========================================== - Coverage 77.15% 77.13% -0.03% ========================================== Files 157 157 Lines 12950 12956 +6 ========================================== + Hits 9991 9993 +2 - Misses 2959 2963 +4 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.