dbogdanov
commented
8 years ago Tests failing on OSX:
- AudioWriter seems to be broken with the new LibAv
- We need to decide what to do with the issue of consistency of MD5 values in AudioLoader
- tests for outdated RhythmExtractor fail, perhaps we should not support it anymore
- some regression tests fail because we need to review and improve them
======================================================================
ERROR: testOgg (io.test_audiowriter_streaming.TestAudioWriter_Streaming)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/dbogdanov/mtg/essentia/test/src/unittest/io/test_audiowriter_streaming.py", line 223, in testOgg
self.encoderTest('audiowritertest.ogg', precision=5e-6);
File "/Users/dbogdanov/mtg/essentia/test/src/unittest/io/test_audiowriter_streaming.py", line 183, in encoderTest
run(gen)
File "test/src/unittest/all_tests.py", line 190, in runResetRun
File "/Users/dbogdanov/mtg/essentia/build/python/essentia/__init__.py", line 148, in run
return _essentia.run(gen)
RuntimeError: Error while writing delayed audio frame
======================================================================
FAIL: testRegression (io.test_audiowriter_streaming.TestAudioWriter_Streaming)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/dbogdanov/mtg/essentia/test/src/unittest/io/test_audiowriter_streaming.py", line 69, in testRegression
self.compare(pool['audio'], signal)
File "/Users/dbogdanov/mtg/essentia/test/src/unittest/io/test_audiowriter_streaming.py", line 34, in compare
self.assertAlmostEqual(result[i][1] - expected[i][1], 0, 0.5/32767)
File "/Users/dbogdanov/mtg/essentia/test/src/unittest/essentia_test.py", line 116, in assertAlmostEqual
self.assert_(diff <= precision, 'Difference is %e while allowed deviation from the expected value is %e %%' % (diff, precision*100))
AssertionError: Difference is 3.051758e-05 while allowed deviation from the expected value is 1.525925e-03 %
======================================================================
FAIL: testSaturation (io.test_audiowriter_streaming.TestAudioWriter_Streaming)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/dbogdanov/mtg/essentia/test/src/unittest/io/test_audiowriter_streaming.py", line 140, in testSaturation
self.compare(pool['audio'], expected)
File "/Users/dbogdanov/mtg/essentia/test/src/unittest/io/test_audiowriter_streaming.py", line 33, in compare
self.assertAlmostEqual(result[i][0] - expected[i][0], 0, 0.5/32767)
File "/Users/dbogdanov/mtg/essentia/test/src/unittest/essentia_test.py", line 116, in assertAlmostEqual
self.assert_(diff <= precision, 'Difference is %e while allowed deviation from the expected value is %e %%' % (diff, precision*100))
AssertionError: Difference is 3.051758e-05 while allowed deviation from the expected value is 1.525925e-03 %
======================================================================
FAIL: testMD5 (io.test_audioloader_streaming.TestAudioLoader_Streaming)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/dbogdanov/mtg/essentia/test/src/unittest/io/test_audioloader_streaming.py", line 241, in testMD5
self.assertEqual(md5_ogg, "a87dad40fea0966cc5b967d5412e8868")
AssertionError: 'd90386f75405eb12c7555d2ea8fd0f16' != 'a87dad40fea0966cc5b967d5412e8868'
======================================================================
FAIL: testImpulseTrain (rhythm.test_rhythmextractor.TestRhythmExtractor)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/dbogdanov/mtg/essentia/test/src/unittest/rhythm/test_rhythmextractor.py", line 308, in testImpulseTrain
self.assertVectorWithinVector(result[1], expectedTicks, 0.03)
File "/Users/dbogdanov/mtg/essentia/test/src/unittest/rhythm/test_rhythmextractor.py", line 110, in assertVectorWithinVector
self.assertAlmostEqual(found[i], expected[j-1], precision)
File "/Users/dbogdanov/mtg/essentia/test/src/unittest/essentia_test.py", line 116, in assertAlmostEqual
self.assert_(diff <= precision, 'Difference is %e while allowed deviation from the expected value is %e %%' % (diff, precision*100))
AssertionError: Difference is 3.711977e-02 while allowed deviation from the expected value is 3.000000e+00 %
======================================================================
FAIL: testImpulse (rhythm.test_superfluxextractor.TestSuperFluxExtractor)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/dbogdanov/mtg/essentia/test/src/unittest/rhythm/test_superfluxextractor.py", line 48, in testImpulse
self.assertAlmostEqualVectorAbs(result, expected, 256./44100)
File "/Users/dbogdanov/mtg/essentia/test/src/unittest/essentia_test.py", line 159, in assertAlmostEqualVectorAbs
self.assertAlmostEqualAbs(val1, val2, precision)
File "/Users/dbogdanov/mtg/essentia/test/src/unittest/essentia_test.py", line 154, in assertAlmostEqualAbs
self.assert_(diff <= precision, 'Difference is %e while allowed absolute error is %e' % (diff, precision))
AssertionError: Difference is 2.979591e-02 while allowed absolute error is 5.804989e-03
======================================================================
FAIL: testRealCase (spectral.test_mfcc.TestMFCC)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/dbogdanov/mtg/essentia/test/src/unittest/spectral/test_mfcc.py", line 97, in testRealCase
self.assertAlmostEqualVector(mean(pool['mfcc'], 0), expected, 1.0e-5)
File "/Users/dbogdanov/mtg/essentia/test/src/unittest/essentia_test.py", line 129, in assertAlmostEqualVector
self.assertAlmostEqual(val1, val2, precision)
File "/Users/dbogdanov/mtg/essentia/test/src/unittest/essentia_test.py", line 116, in assertAlmostEqual
self.assert_(diff <= precision, 'Difference is %e while allowed deviation from the expected value is %e %%' % (diff, precision*100))
AssertionError: Difference is 1.266321e-05 while allowed deviation from the expected value is 1.000000e-03 %
======================================================================
FAIL: testRegression (standard.test_panning.TestPanning)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/dbogdanov/mtg/essentia/test/src/unittest/standard/test_panning.py", line 83, in testRegression
self.assertAlmostEqualVectorFixedPrecision(expected[i], output[0], 2)
File "/Users/dbogdanov/mtg/essentia/test/src/unittest/essentia_test.py", line 107, in assertAlmostEqualVectorFixedPrecision
self.assertAlmostEqualFixedPrecision(val1, val2, digits)
File "/Users/dbogdanov/mtg/essentia/test/src/unittest/essentia_test.py", line 102, in assertAlmostEqualFixedPrecision
BaseTestCase.assertAlmostEqual(self, found, expected, digits)
AssertionError: -149.452 != -149.87411 within 2 places
======================================================================
FAIL: testRegression (tonal.test_dissonance.TestDissonance)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/dbogdanov/mtg/essentia/test/src/unittest/tonal/test_dissonance.py", line 168, in testRegression
self.assertAlmostEqual(dissAvg, 0.475740219278, 1e-6)
File "/Users/dbogdanov/mtg/essentia/test/src/unittest/essentia_test.py", line 116, in assertAlmostEqual
self.assert_(diff <= precision, 'Difference is %e while allowed deviation from the expected value is %e %%' % (diff, precision*100))
AssertionError: Difference is 1.619066e-06 while allowed deviation from the expected value is 1.000000e-04 %
======================================================================
FAIL: testARealCase (tonal.test_pitchyin.TestPitchYin)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/dbogdanov/mtg/essentia/test/src/unittest/tonal/test_pitchyin.py", line 131, in testARealCase
self.assertAlmostEqualVector(pitch, expected_pitch)
File "/Users/dbogdanov/mtg/essentia/test/src/unittest/essentia_test.py", line 129, in assertAlmostEqualVector
self.assertAlmostEqual(val1, val2, precision)
File "/Users/dbogdanov/mtg/essentia/test/src/unittest/essentia_test.py", line 116, in assertAlmostEqual
self.assert_(diff <= precision, 'Difference is %e while allowed deviation from the expected value is %e %%' % (diff, precision*100))
AssertionError: Difference is 6.245615e-01 while allowed deviation from the expected value is 1.000000e-05 %
======================================================================
FAIL: testARealCase (tonal.test_pitchyinfft.TestPitchYinFFT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/dbogdanov/mtg/essentia/test/src/unittest/tonal/test_pitchyinfft.py", line 131, in testARealCase
self.assertAlmostEqualVector(pitch, expected_pitch)
File "/Users/dbogdanov/mtg/essentia/test/src/unittest/essentia_test.py", line 129, in assertAlmostEqualVector
self.assertAlmostEqual(val1, val2, precision)
File "/Users/dbogdanov/mtg/essentia/test/src/unittest/essentia_test.py", line 116, in assertAlmostEqual
self.assert_(diff <= precision, 'Difference is %e while allowed deviation from the expected value is %e %%' % (diff, precision*100))
AssertionError: Difference is 1.141042e-02 while allowed deviation from the expected value is 1.000000e-05 %
palonso
Some regression tests are failing for some algorithms. Review and update the ground truth annotation.
PitchYinFFT
Panning
RhythmExtractor (outdated algorithm, #3 )