Failing Tests in Python 2.7

eric-tramel commented 7 years ago

I am having some discrepancies when running tests in either Python 3 or in Python 2.7. I have included the full test logs at the end. Specifically, Python 2.7 seems to have issues with the following tests:

estim.linear.lin_test_mult -- "Posterior estimate Gaussian error does not match value"
estim.linear_two.lin_two_test-mult -- "Error in second order terms"
solver.vamp.vamp_test_mult -- "Variance does not match"
vamp_spare_test.sparse_inv -- "Final MSE = 0.000000" (seems okay but sends a Fail signal.)

These tests pass on Python 3. If we want to support Python 2.7 going forward, we'll need to look at the discrepancies that are causing these errors.

Python 3 Test Output

~/src/vampyre   master  python3 setup.py test
/usr/local/Cellar/python3/3.5.2_2/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/dist.py:261: UserWarning: Unknown distribution option: 'test_require'
  warnings.warn(msg)
running test
running egg_info
writing requirements to vampyre.egg-info/requires.txt
writing dependency_links to vampyre.egg-info/dependency_links.txt
writing vampyre.egg-info/PKG-INFO
writing top-level names to vampyre.egg-info/top_level.txt
reading manifest file 'vampyre.egg-info/SOURCES.txt'
writing manifest file 'vampyre.egg-info/SOURCES.txt'
running build_ext
vampyre.test.test_version ... vampyre version 0.0
total diff= 0.000000
common.utils.repeat Passed
ok
vampyre.test.test_suite.test_version ... vampyre version 0.0
total diff= 0.000000
common.utils.repeat Passed
ok
test_suite.test_version ... vampyre version 0.0
common.utils.repeat_test                 Pass
estim.gaussian.gauss_test                Pass
estim.mixture.mix_test                   Pass
trans.matrix.matrix_test                 Pass
estim.linear.lin_test_mult               Pass
estim.discrete.discrete_test             Pass
estim.linear_two.lin_two_test_mult       Pass
estim.interval.gauss_integral_test       Pass
estim.interval.hard_thresh_test          Pass
estim.relu.relu_test                     Pass
/Users/eric/src/vampyre/vampyre/estim/msg.py:163: DeprecationWarning: elementwise != comparison failed; this will raise an error in the future.
  if (r1_prev != []) and (rvar1_prev != []) and (self.damp < 1):
solver.vamp.vamp_test_mult               Pass
solver.mlvamp.mlvamp_probit_test         Pass
vamp_sparse_test.sparse_inv              Pass
13 out of 13 passed
ok

----------------------------------------------------------------------
Ran 3 tests in 1.771s

OK

Python 2.7 Test Output

 ~/src/vampyre   master  python setup.py test
/usr/local/Cellar/python/2.7.12_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'test_require'
  warnings.warn(msg)
running test
running egg_info
writing pbr to vampyre.egg-info/pbr.json
writing requirements to vampyre.egg-info/requires.txt
writing vampyre.egg-info/PKG-INFO
writing top-level names to vampyre.egg-info/top_level.txt
writing dependency_links to vampyre.egg-info/dependency_links.txt
reading manifest file 'vampyre.egg-info/SOURCES.txt'
writing manifest file 'vampyre.egg-info/SOURCES.txt'
running build_ext
vampyre.test.test_version ... vampyre version 0.0
total diff= 0.000000
common.utils.repeat Passed
ok
vampyre.test.test_suite.test_version ... vampyre version 0.0
total diff= 0.000000
common.utils.repeat Passed
ok
test_suite.test_version ... vampyre version 0.0
common.utils.repeat_test                 Pass
estim.gaussian.gauss_test                Pass
estim.mixture.mix_test                   Pass
trans.matrix.matrix_test                 Pass
estim.linear.lin_test_mult               Fail
   Posterior estimate Gaussian error  does not match predicted value
estim.discrete.discrete_test             Pass
estim.linear_two.lin_two_test_mult       Fail
   Error in second order terms
estim.interval.gauss_integral_test       Pass
estim.interval.hard_thresh_test          Pass
estim.relu.relu_test                     Pass
solver.vamp.vamp_test_mult               Fail
   Variance does not match
solver.mlvamp.mlvamp_probit_test         MSE 0 actual:   -0.28 pred:   -0.28
MSE 1 actual:   -0.28 pred:   -0.28
Pass
vamp_sparse_test.sparse_inv              Final MSE = 0.000000
Fail
   MSE exceeded expected value
9 out of 13 passed
ok

----------------------------------------------------------------------
Ran 3 tests in 1.341s

OK

eric-tramel commented 7 years ago

@mborgerding Since you've already had a hand at 2.7 compatibility, could you take a look at this discrepancy we're observing between 3.5 & 2.7 in the unit tests?

sdrangan commented 7 years ago

Mark, Let me make a check in first. I realized I didn't add the from future import division everywhere.

Sundeep

On Thu, Jan 26, 2017 at 5:49 PM, Eric W. Tramel notifications@github.com wrote:

@mborgerding https://github.com/mborgerding Since you've already had a hand at 2.7 compatibility, could you take a look at this discrepancy we're observing between 3.5 & 2.7 in the unit tests?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/GAMPTeam/vampyre/issues/3#issuecomment-275538895, or mute the thread https://github.com/notifications/unsubscribe-auth/ANO8I2yZg0YAnVc8ha1RuNwSjgymjxFWks5rWSLygaJpZM4LupIt .

sdrangan commented 7 years ago

The problems appear to not arise after I added from future import division. All test pass now.

GAMPTeam / vampyre

Failing Tests in Python 2.7 #3

Python 3 Test Output

Python 2.7 Test Output