search

PoonLab / OpenRDP

An open-source re-implementation of the RDP4 recombination detection program
GNU General Public License v3.0
45 stars 9 forks source link

Update unit tests #48

Closed ArtPoon closed 1 year ago

ArtPoon commented 1 year ago

I've probably broken most of them in refactoring the interface and package structure

ArtPoon commented 1 year ago

I'm working on this in the dev branch. @GopiGugan has also been updating unit tests in another branch, mostly for Bootscan. I am going to push my local changes to a new branch (iss48) to make it easier for Gopi to push his updates to dev.

ArtPoon commented 1 year ago

See commit 8a2a8cde99def7944f9a02108468ad09e8243b2b

(venv) art@orolo:~/git/OpenRDP$ python -m unittest openrdp/tests/test_common.py 
.................
----------------------------------------------------------------------
Ran 17 tests in 1.220s

OK
(venv) art@orolo:~/git/OpenRDP$ python -m unittest openrdp/tests/test_bootscan.py 
E
/usr/local/lib/python3.9/unittest/case.py:614: ResourceWarning: unclosed file <_io.TextIOWrapper name='/home/art/git/OpenRDP/openrdp/tests/long.fasta' mode='r' encoding='UTF-8'>
  outcome.errors.clear()
ResourceWarning: Enable tracemalloc to get the object allocation traceback
EEE
======================================================================
ERROR: test_execute_hiv (openrdp.tests.test_bootscan.TestBootscan)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/configparser.py", line 846, in items
    d.update(self._sections[section])
KeyError: 'Bootscan'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/art/git/OpenRDP/openrdp/tests/test_bootscan.py", line 58, in setUp
    test_settings = dict(config.items('Bootscan'))
  File "/usr/local/lib/python3.9/configparser.py", line 849, in items
    raise NoSectionError(section)
configparser.NoSectionError: No section: 'Bootscan'

======================================================================
ERROR: test_execute_long (openrdp.tests.test_bootscan.TestBootscan)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/configparser.py", line 846, in items
    d.update(self._sections[section])
KeyError: 'Bootscan'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/art/git/OpenRDP/openrdp/tests/test_bootscan.py", line 58, in setUp
    test_settings = dict(config.items('Bootscan'))
  File "/usr/local/lib/python3.9/configparser.py", line 849, in items
    raise NoSectionError(section)
configparser.NoSectionError: No section: 'Bootscan'

======================================================================
ERROR: test_execute_short (openrdp.tests.test_bootscan.TestBootscan)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/configparser.py", line 846, in items
    d.update(self._sections[section])
KeyError: 'Bootscan'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/art/git/OpenRDP/openrdp/tests/test_bootscan.py", line 58, in setUp
    test_settings = dict(config.items('Bootscan'))
  File "/usr/local/lib/python3.9/configparser.py", line 849, in items
    raise NoSectionError(section)
configparser.NoSectionError: No section: 'Bootscan'

======================================================================
ERROR: test_set_and_validate_options (openrdp.tests.test_bootscan.TestBootscan)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/configparser.py", line 846, in items
    d.update(self._sections[section])
KeyError: 'Bootscan'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/art/git/OpenRDP/openrdp/tests/test_bootscan.py", line 58, in setUp
    test_settings = dict(config.items('Bootscan'))
  File "/usr/local/lib/python3.9/configparser.py", line 849, in items
    raise NoSectionError(section)
configparser.NoSectionError: No section: 'Bootscan'

----------------------------------------------------------------------
Ran 4 tests in 8.727s

FAILED (errors=4)
ArtPoon commented 1 year ago

I resolved the problem with configparser by hard-coding the configuration dicts into the test scripts, which is apparently the preferred approach anyhow. As of commit e493f824ca9e6f96b2f97f2f02764c09a3938761, all unit tests pass except for those in test_bootscan.py - just waiting on @GopiGugan to update

GopiGugan commented 1 year ago

test_bootscan.py unit tests have been updated in dev. I'm looking into why one of the unit tests failed.

The expected output in test_execute_hiv did not match the result:

[('07_BC', ('B', 'C'), 580, 1260, 1.0158213242512536e-09),
+ ('07_BC', ('B', 'C'), 1280, 1340, 0.024030305239701163),
 ('07_BC', ('B', 'C'), 2060, 2560, 1.7427448207965451e-06),
+ ('07_BC', ('B', 'C'), 2620, 2900, 2.1518293256796478e-07),
 ('07_BC', ('B', 'C'), 3000, 3160, 0.003489818124092514),
+ ('07_BC', ('B', 'C'), 3260, 5680, 1.5236266249248996e-40),
 ('07_BC', ('B', 'C'), 8960, 9140, 0.0010069350767726085),
 ('B', ('07_BC', 'C'), 100, 560, 3.53955031313494e-19),
+ ('B', ('07_BC', 'C'), 1280, 1340, 0.022012231965328217),
 ('B', ('07_BC', 'C'), 2060, 2560, 1.6836162768997676e-11),
+ ('B', ('07_BC', 'C'), 3000, 3160, 8.361164579231644e-08),
 ('B', ('07_BC', 'C'), 8960, 9140, 1.5107519378439202e-05),
+ ('C', ('07_BC', 'B'), 580, 1260, 0.04109525585896175),
 ('C', ('07_BC', 'B'), 2620, 2900, 0.00022445087998712194),
+ ('C', ('07_BC', 'B'), 3260, 5680, 9.28749065950454e-34)]
GopiGugan commented 1 year ago

Traced the issue to changes done in the merge_breakpoints function (https://github.com/PoonLab/OpenRDP/commit/6c7261dd1d75f1a2cd1ceca25c749bcd321009b7). However, these changes make sense.

The result_dict is the same in the previous and current versions of this function:

{
    ('07_BC', ('B', 'C')): [
        (580, 1260, 1.0158213242512536e-09), 
        (1280, 1340, 0.024030305239701163), 
        (2060, 2560, 1.7427448207965451e-06), 
        (2620, 2900, 2.1518293256796478e-07), 
        (3000, 3160, 0.003489818124092514), 
        (3260, 5680, 1.5236266249248996e-40), 
        (8960, 9140, 0.0010069350767726085)
      ], 
    ('B', ('07_BC', 'C')): [
        (100, 560, 3.53955031313494e-19), 
        (1280, 1340, 0.022012231965328217),  
        (2060, 2560, 1.6836162768997676e-11), 
        (3000, 3160, 8.361164579231644e-08), 
        (8960, 9140, 1.5107519378439202e-05)
     ], 
    ('C', ('07_BC', 'B')): [
        (100, 560, 0.11570386064994018), 
        (580, 1260, 0.04109525585896175), 
        (2620, 2900, 0.00022445087998712194), 
        (3260, 5680, 9.28749065950454e-34)
      ]
}

In the previous version, records from the results_dict values were being removed while iterating through the list: https://github.com/PoonLab/OpenRDP/blob/3a59bc1ae09c7fed4af4dd077c02d5abe9e097ad/openrdp/scripts/bootscan.py#L241-L255

This caused a few of the records to be skipped over.

This issue was fixed in https://github.com/PoonLab/OpenRDP/commit/6c7261dd1d75f1a2cd1ceca25c749bcd321009b7:

https://github.com/PoonLab/OpenRDP/blob/d5cb63962586120b65f22ba7d5d2345969e24517/openrdp/bootscan.py#L300-L316

The purpose of this function is to merge overlapping breakpoint locations. However, none of the following regions overlap:

[('07_BC', ('B', 'C'), 580, 1260, 1.0158213242512536e-09),
+ ('07_BC', ('B', 'C'), 1280, 1340, 0.024030305239701163),
 ('07_BC', ('B', 'C'), 2060, 2560, 1.7427448207965451e-06),
+ ('07_BC', ('B', 'C'), 2620, 2900, 2.1518293256796478e-07),
 ('07_BC', ('B', 'C'), 3000, 3160, 0.003489818124092514),
+ ('07_BC', ('B', 'C'), 3260, 5680, 1.5236266249248996e-40),
 ('07_BC', ('B', 'C'), 8960, 9140, 0.0010069350767726085),
 ('B', ('07_BC', 'C'), 100, 560, 3.53955031313494e-19),
+ ('B', ('07_BC', 'C'), 1280, 1340, 0.022012231965328217),
 ('B', ('07_BC', 'C'), 2060, 2560, 1.6836162768997676e-11),
+ ('B', ('07_BC', 'C'), 3000, 3160, 8.361164579231644e-08),
 ('B', ('07_BC', 'C'), 8960, 9140, 1.5107519378439202e-05),
+ ('C', ('07_BC', 'B'), 580, 1260, 0.04109525585896175),
 ('C', ('07_BC', 'B'), 2620, 2900, 0.00022445087998712194),
+ ('C', ('07_BC', 'B'), 3260, 5680, 9.28749065950454e-34)]

I have modified the expected result in the unit test that failed in test_bootscan.py (in the dev branch). Can you please confirm whether the expected result for this unit test should include these additional breakpoints?

ArtPoon commented 1 year ago

@ArtPoon to review in PR

ArtPoon commented 1 year ago

@GopiGugan can you run this test case using Windows RDP to generate the expected results, and then compare to the above output?

ArtPoon commented 1 year ago

OpenRDP develop was based on RDP verison 5.5.1 according to the rdp5.vbp file:

MajorVer=5
MinorVer=5
RevisionVer=1