Fix for correctly retrieving filtered out variants, update unit tests

[jethror1]

• Adds fix for bug in bcftools isec returning wrong variant for those excluded 1701
  • switch to using bcftools filter with --soft-filter to tag the variants to exclude, then read these into a dataframe and split on the tag
  • changes in vcf.py to accommodate new filtering
• updated htslib 1.14.0 -> 1.15.0
• updated and fixed unit tests:

jethro@jethro-T490:~/Projects/eggd_vcf2xls_nirvana/resources/home/dnanexus/generate_workbook/tests$ python3 -m pytest -v .
================================================================================================================================= test session starts =================================================================================================================================
platform linux -- Python 3.8.10, pytest-7.0.1, pluggy-1.0.0 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: /home/jethro/Projects/eggd_vcf2xls_nirvana/resources/home/dnanexus/generate_workbook/tests
collected 18 items                                                                                                                                                                                                                                                                    

test_columns.py::TestInfoColumn::test_parsed_correct_columns_from_info_records PASSED                                                                                                                                                                                           [  5%]
test_columns.py::TestInfoColumn::test_parsed_correct_gnomAD_AF_values PASSED                                                                                                                                                                                                    [ 11%]
test_columns.py::TestFormatSample::test_format_sample_values_are_correct PASSED                                                                                                                                                                                                 [ 16%]
test_filters.py::TestModifyingFieldTypes::test_type_correctly_modified PASSED                                                                                                                                                                                                   [ 22%]
test_filters.py::TestModifyingFieldTypes::test_header_overwritten_correctly PASSED                                                                                                                                                                                              [ 27%]
test_filters.py::TestFilters::test_filter_with_include_eq PASSED                                                                                                                                                                                                                [ 33%]
test_filters.py::TestFilters::test_filter_with_exclude_eq PASSED                                                                                                                                                                                                                [ 38%]
test_filters.py::TestFilters::test_filter_with_exclude_gt PASSED                                                                                                                                                                                                                [ 44%]
test_filters.py::TestFilters::test_combined_exclude_float_and_string PASSED                                                                                                                                                                                                     [ 50%]
test_vcf.py::TestHeader::test_column_names PASSED                                                                                                                                                                                                                               [ 55%]
test_vcf.py::TestHeader::test_parse_reference PASSED                                                                                                                                                                                                                            [ 61%]
test_vcf.py::TestHeader::test_only_header_parsed PASSED                                                                                                                                                                                                                         [ 66%]
test_vcf.py::TestDataFrameActions::test_drop_columns_exclude PASSED                                                                                                                                                                                                             [ 72%]
test_vcf.py::TestDataFrameActions::test_drop_columns_include PASSED                                                                                                                                                                                                             [ 77%]
test_vcf.py::TestDataFrameActions::test_reorder_columns_correct_order PASSED                                                                                                                                                                                                    [ 83%]
test_vcf.py::TestDataFrameActions::test_reorder_columns_no_dropped_columns PASSED                                                                                                                                                                                               [ 88%]
test_vcf.py::TestDataFrameActions::test_non_rename_columns_unaffacted PASSED                                                                                                                                                                                                    [ 94%]
test_vcf.py::TestDataFrameActions::test_renamed_correctly PASSED                                                                                                                                                                                                                [100%]

================================================================================================================================= 18 passed in 19.85s =================================================================================================================================

---

This change is 

[pep8speaks]

Hello @jethror1! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

• In the file resources/home/dnanexus/generate_workbook/tests/test_filters.py:

Line 140:5: E303 too many blank lines (2) Line 173:80: E501 line too long (85 > 79 characters) Line 189:80: E501 line too long (81 > 79 characters) Line 194:5: E303 too many blank lines (2) Line 214:5: E303 too many blank lines (2) Line 235:5: E303 too many blank lines (2) Line 262:5: E303 too many blank lines (2) Line 289:9: E303 too many blank lines (2) Line 291:17: E127 continuation line over-indented for visual indent Line 292:21: E127 continuation line over-indented for visual indent Line 300:56: E502 the backslash is redundant between brackets Line 301:47: E502 the backslash is redundant between brackets Line 302:65: E502 the backslash is redundant between brackets Line 310:1: W293 blank line contains whitespace

• In the file resources/home/dnanexus/generate_workbook/tests/test_vcf.py:

Line 106:80: E501 line too long (80 > 79 characters) Line 117:5: E303 too many blank lines (2)

• In the file resources/home/dnanexus/generate_workbook/utils/filters.py:

Line 31:5: E303 too many blank lines (2) Line 80:5: E303 too many blank lines (2) Line 80:80: E501 line too long (85 > 79 characters) Line 97:80: E501 line too long (89 > 79 characters) Line 98:80: E501 line too long (89 > 79 characters)

• In the file resources/home/dnanexus/generate_workbook/utils/vcf.py:

Line 89:80: E501 line too long (80 > 79 characters)

Comment last updated at 2022-04-22 12:16:32 UTC