search

SwissTPH / openmalaria

A simulator of malaria epidemiology and control
GNU General Public License v2.0
76 stars 30 forks source link

unittest for PkPD conversion model fails #53

Closed tph-thuering closed 9 years ago

tph-thuering commented 9 years ago

Metabolite concentration values are negative.

relevant unittests are called testAR and testAS, both use the conversion model with DHA-metabolites

see Unittest output

For a more detailed unittest with intermediate values for g, h, i and qtyM, qtyP output, see the first revision

Hint - if you'd like to reproduce the verbose output: remove comment from this line and compile OM.

tph-thuering commented 9 years ago

Monolix PKPDlibrary reference: http://www.lixoft.eu/wp-content/resources/docs/PKPDlibrary.pdf For three compartment model see page 44. (Not sure if conversion model is related or not)

tph-thuering commented 9 years ago

After applying this patch https://gist.github.com/tph-thuering/288474a8ae514b0d9d55#ian.patch

And running the unittests, we get the following results:

Running 58 tests................................................. In PkPdComplianceSuite::testAR:

Artemether conversion unittest (AR):

Type day difference found expected
DF 0 0.00000012266375776 0.00002453275155652 0.00000017000000002
CO 0 0.00000100949828987 -0.00020189965787820 0.00020131259995317
DF 1 0.00000000000300366 0.00000000060073299 0.00000000000002840
CO 1 0.00000100949195449 -0.00020189839099318 0.00020131390006428
DF 2 0.00000000000000007 0.00000000000001471 0.00000000000000000
CO 2 0.00000100949195445 -0.00020189839098524 0.00020131390006428
DF 3 0.00000000000000007 0.00000000000001471 0.00000000000000000
CO 3 0.00000000000635000 0.00000000126063351 0.00000000127000000
DF 4 0.00000000000000007 0.00000000000001471 0.00000000000000000

. In PkPdComplianceSuite::testAS:

Arthesunate conversion unittest(AS):

Type day difference found expected
DF 0 0.00002387377793806 0.00477475558105623 0.00053229079937151
CO 0 0.00000000000057577 0.00000000011515423 0.00000000011400000
DF 1 0.00000011399145468 0.00002279829092163 0.00000028300000004
CO 1 0.00000000000057577 0.00000000011515423 0.00000000011400000
DF 2 0.00000000054428133 0.00000010885626694 0.00000000015100000
CO 2 0.00000000000057577 0.00000000011515423 0.00000000011400000
DF 3 0.00000000054428133 0.00000010885626694 0.00000000015100000
CO 3 0.00000000000000000 0.00000000000000000 0.00000000000000000
DF 4 0.00000000054428133 0.00000010885626694 0.00000000015100000

...... Failed 2 of 58 tests Success rate: 96%

tph-thuering commented 9 years ago

Running 58 tests................................................

Testing AR1:

day factor f_error f_rel_error concentration c_error c_rel_error concentration2 c2_error
0 1 0 1 0 0 -nan
1 1.03393e-12 -4.03897e-27 1 0.015352 -3.65369e-11 1
2 1.06825e-24 -6.19157e-28 0.999421 0.0156447 1.06909e-11 1
3 1.1037e-36 4.06951e-40 1.00037 0.0156502 4.77728e-05 1.00306
4 1.7314e-42 -2.99925e-42 0.365996 0.000298342 4.56162e-10 1
5 1.73054e-42 -3.0001e-42 0.365815 5.68734e-06 -3.66015e-12 0.999999

.

Testing AR:

day factor f_error f_rel_error concentration c_error c_rel_error concentration2 c2_error
0 1 0 1 0 0 -nan 0 0
1 0.00148275 0.00148258 8746.55 0.000182523 -1.39572e-11 1 0.000201313 -2.97398e-11
2 2.15304e-06 2.15304e-06 7.58607e+07 0.000182524 3.29943e-11 1 0.000201314 -6.28492e-11
3 3.12634e-09 3.12634e-09 6.59562e+11 0.000182524 3.30015e-11 1 0.000201314 -6.28413e-11
4 3.12634e-09 3.12634e-09 6.57971e+11 1.14696e-09 -3.04129e-12 0.997355 1.2669e-09 -3.10151e-12
5 3.12634e-09 3.12634e-09 6.57971e+11 7.18952e-15 -4.80014e-19 0.999933 7.94134e-15 1.34264e-18

.

Testing AS1:

day factor f_error f_rel_error concentration c_error c_rel_error concentration2 c2_error
0 1 0 1 0 0 -nan
1 1.20467e-05 4.67463e-08 1.0039 8.98336e-08 3.36194e-11 1.00037
2 1.45124e-10 1.24082e-13 1.00086 8.98336e-08 3.3625e-11 1.00037
3 1.74827e-15 -1.72717e-18 0.999013 8.98336e-08 3.3625e-11 1.00037
4 1.74827e-15 -1.72717e-18 0.999013 5.54818e-15 -1.82022e-18 0.999672
5 1.74827e-15 -1.72717e-18 0.999013 0 -3.43e-22 0

.

Testing AS:

day factor f_error f_rel_error concentration c_error c_rel_error concentration2 c2_error
0 1 0 1 0 0 -nan 0 0
1 0.00477476 0.00424246 8.9702 2.3013e-14 1.30484e-17 1.00057 -1.15154e-10 -2.29154e-10
2 2.27983e-05 2.2515e-05 80.4645 2.3013e-14 1.30484e-17 1.00057 -1.15154e-10 -2.29154e-10
3 1.08856e-07 1.08705e-07 721.782 2.3013e-14 1.30484e-17 1.00057 -1.15154e-10 -2.29154e-10
4 1.08856e-07 1.08705e-07 721.782 8.2455e-28 -4.50008e-31 0.999455 -1.06782e-21 -2.13782e-21
5 1.08856e-07 1.08856e-07 inf 2.95434e-41 4.33563e-44 1.00147 -9.94054e-33 -1.98805e-32

.

Testing DHA:

day factor f_error f_rel_error concentration c_error c_rel_error concentration2 c2_error
0 1 0 1 0 0 -nan
1 0.000355234 -3.76304e-10 0.999999 6.75839e-09 -1.61418e-12 0.999761
2 1.26191e-07 1.90927e-10 1.00152 6.75839e-09 -1.61416e-12 0.999761
3 4.48273e-11 2.72601e-14 1.00061 6.75839e-09 -1.61416e-12 0.999761
4 4.48273e-11 2.72601e-14 1.00061 1.70142e-17 1.42277e-20 1.00084
5 4.48273e-11 2.72601e-14 1.00061 0 -4.28e-26 0

.

Testing LF:

day factor f_error f_rel_error concentration c_error c_rel_error concentration2 c2_error
0 1 0 1 0 0 -nan
1 0.0317463 4.73527e-10 1 1.01443 5.87699e-11 1
2 0.00100781 -1.4843e-09 0.999999 1.87888 -1.46805e-10 1
3 3.19935e-05 -6.50612e-09 0.999797 2.61551 -2.37216e-10 1
4 1.01565e-06 -4.34607e-09 0.995739 2.22879 -3.57779e-10 1
5 3.22426e-08 4.25871e-11 1.00132 1.89925 1.77621e-10 1

.

Testing MQ:

day factor f_error f_rel_error concentration c_error c_rel_error concentration2 c2_error
0 1 0 1 0 0 -nan
1 0.0317458 2.32157e-10 1 0.37844 -3.81273e-07 0.999999
2 0.00100779 2.76226e-10 1 0.737344 -1.10487e-06 0.999999
3 3.1993e-05 -7.02364e-09 0.999781 1.07772 -2.13423e-06 0.999998
4 1.01564e-06 -4.36257e-09 0.995723 1.02209 -3.05396e-06 0.999997
5 3.22421e-08 4.20583e-11 1.00131 0.969327 -3.87312e-06 0.999996

.

Testing PPQ:

day factor f_error f_rel_error concentration c_error c_rel_error concentration2 c2_error
0 1 0 1 0 0 -nan
1 0.0524515 4.20915e-10 1 0.116453 2.5821e-11 1
2 0.00168186 -5.37307e-12 1 0.229465 -4.40691e-11 1
3 5.34433e-05 2.06221e-10 1 0.339137 -4.97115e-08 1
4 1.69855e-06 1.34833e-14 1 0.329114 -3.12838e-11 1
5 5.39961e-08 -3.93245e-12 0.999927 0.319387 9.61325e-12 1

.

Testing PPQ2:

day factor f_error f_rel_error concentration c_error c_rel_error concentration2 c2_error
0 1 0 1 0 0 -nan
1 0.0342259 -2.82181e-08 0.999999 0.0724459 3.31059e-11 1
2 0.00108659 -4.07372e-10 1 0.121802 5.33078e-12 1
3 3.4495e-05 -5.04796e-09 0.999854 0.156117 -5.19698e-12 1
4 1.09514e-06 -4.8571e-09 0.995584 0.108163 -1.41887e-11 1
5 3.47865e-08 -1.35093e-11 0.999612 0.076857 4.55472e-11 1

.

Testing PPQ3:

day factor f_error f_rel_error concentration c_error c_rel_error concentration2 c2_error
0 1 0 1 0 0 -nan
1 0.0342176 5.96403e-09 1 0.0768788 1.41506e-11 1
2 0.00108631 2.00802e-10 1 0.120169 2.83906e-11 1
3 3.44858e-05 4.04864e-10 1.00001 0.152677 -1.6656e-11 1
4 1.09489e-06 1.91858e-13 1 0.101699 4.14293e-11 1
5 3.47787e-08 -4.33748e-12 0.999875 0.0798269 1.51654e-12 1
tph-thuering commented 9 years ago

Not fixed, will supply a fix now.

tph-thuering commented 9 years ago

commit 5c89b03 fixes AR unittest. -> https://travis-ci.org/SwissTPH/openmalaria/jobs/75018663#L645

AS needs probably more precise expected drug factor values from @KatherineKay

Testing AS:

day factor f_error f_rel_error concentration c_error c_rel_error concentration2 c2_error
0 1 0 1 0 0 -nan 0 0
1 0.000515278 -1.70125e-05 0.968039 2.3013e-14 1.30484e-17 1.00057 1.14249e-10 2.49099e-13
2 2.65512e-07 -1.78218e-08 0.9371 2.3013e-14 1.30484e-17 1.00057 1.14249e-10 2.49099e-13
3 1.36812e-10 -1.40036e-11 0.907148 2.3013e-14 1.30484e-17 1.00057 1.14249e-10 2.49099e-13
4 1.36812e-10 -1.40036e-11 0.907148 8.2455e-28 -4.50008e-31 0.999455 -1.06782e-21 -2.13782e-21
5 1.36812e-10 -1.40036e-11 0.907148 2.95434e-41 4.33563e-44 1.00147 9.94054e-33 5.40864e-37
tph-thuering commented 9 years ago

After verification our expected values we use to compare seem to be correct.

Output of the AR and AS conversion unittests are now as follows:

Testing AR:

day factor f_error f_rel_error concentration c_error c_rel_error concentration2 c2_error
0 1 0 0% 0 0 -nan% 0 0
1 1.69524e-07 4.51805e-14 0% 0.000182523 -1.39572e-11 0.01% 0.000201313 -4.94356e-12
2 2.83815e-14 7.47577e-21 0% 0.000182524 3.29943e-11 0% 0.000201314 -3.8053e-11
3 4.75159e-21 1.15717e-23 -0.24% 0.000182524 3.30015e-11 0% 0.000201314 -3.80451e-11
4 4.75148e-21 -4.23176e-29 0.01% 1.14696e-09 -3.04129e-12 0.27% 1.2669e-09 -3.10151e-12
5 4.75148e-21 -4.23176e-29 0.01% 7.18952e-15 -4.80014e-19 0.01% 7.94134e-15 1.34264e-18

.

Testing AS:

day factor f_error f_rel_error concentration c_error c_rel_error concentration2 c2_error
0 1 0 0% 0 0 -nan% 0 0
1 0.000515278 -1.70125e-05 3.2% 2.3013e-14 1.30484e-17 -0.05% 1.14249e-10 2.49099e-13
2 2.65512e-07 -1.78218e-08 6.3% 2.3013e-14 1.30484e-17 -0.05% 1.14249e-10 2.49099e-13
3 1.36812e-10 -1.40036e-11 9.29% 2.3013e-14 1.30484e-17 -0.05% 1.14249e-10 2.49099e-13
4 1.36812e-10 -1.40036e-11 9.29% 8.2455e-28 -4.50008e-31 0.06% 0 -1.07e-21
5 1.36812e-10 -1.40036e-11 9.29% 2.95434e-41 4.33563e-44 -0.14% 9.94054e-33 5.40864e-37

between day 3&4: ERROR! concentration is lower than zero, setting it to zero for now: -1.06782e-21 drug_index: 4

tph-thuering commented 9 years ago

Testing AR - DHA_AR Conversion

AR

day factor f_abs_error f_rel_err % conc c_abs_error c_rel_err % type
0 1 +0 +0 0 +0 -nan P
0 1 +0 +0 0 +0 -nan M
1 1.69524e-07 +4.51805e-14 +0 0.000182523 -1.39572e-11 -0.0001 P
1 1.69524e-07 +4.51805e-14 +0 0.000201313 -4.83963e-12 -0.0001 M
2 2.83815e-14 +7.47577e-21 +0 0.000182524 +3.29943e-11 +0 P
2 2.83815e-14 +7.47577e-21 +0 0.000201314 -3.79488e-11 -0.0001 M
3 4.75159e-21 +1.15717e-23 +0.2441 0.000182524 +3.30015e-11 +0 P
3 4.75159e-21 +1.15717e-23 +0.2441 0.000201314 -3.79409e-11 -0.0001 M
4 4.75148e-21 -4.23176e-29 -0.0001 1.14696e-09 -3.04129e-12 -0.2645 P
4 4.75148e-21 -4.23176e-29 -0.0001 1.2669e-09 -3.10151e-12 -0.2443 M
5 4.75148e-21 -4.23176e-29 -0.0001 7.18952e-15 -4.80014e-19 -0.0067 P
5 4.75148e-21 -4.23176e-29 -0.0001 7.94134e-15 +1.34264e-18 +0.0169 M

. ...

Testing AS - DHA_AS Conversion

AS

day factor f_abs_error f_rel_err % conc c_abs_error c_rel_err % type
0 1 +0 +0 0 +0 -nan P
0 1 +0 +0 0 +0 -nan M
1 0.000532291 +2.41417e-10 +0 2.3013e-14 +1.30484e-17 +0.0567 P
1 0.000532291 +2.41417e-10 +0 1.14249e-10 +2.49099e-13 +0.2185 M
2 2.83334e-07 +2.52769e-13 +0 2.3013e-14 +1.30484e-17 +0.0567 P
2 2.83334e-07 +2.52769e-13 +0 1.14249e-10 +2.49099e-13 +0.2185 M
3 1.50816e-10 +1.83276e-17 +0 2.3013e-14 +1.30484e-17 +0.0567 P
3 1.50816e-10 +1.83276e-17 +0 1.14249e-10 +2.49099e-13 +0.2185 M
4 1.50816e-10 +1.83276e-17 +0 8.2455e-28 -4.50008e-31 -0.0546 P
4 1.50816e-10 +1.83276e-17 +0 1.05936e-21 -1.06425e-23 -0.9947 M
5 1.50816e-10 +1.83276e-17 +0 2.95434e-41 +4.33563e-44 +0.1469 P
5 1.50816e-10 +1.83276e-17 +0 9.8618e-33 -7.82036e-35 -0.7868 M