pandas-dev / pandas

Flexible and powerful data analysis / manipulation library for Python, providing labeled data structures similar to R data.frame objects, statistical functions, and much more
https://pandas.pydata.org
BSD 3-Clause "New" or "Revised" License
43.77k stars 17.97k forks source link

`nan != 'KORD'` error in test_multiple_date_cols_chunked #7288

Closed matthew-brett closed 10 years ago

matthew-brett commented 10 years ago

Running pandas tests as part of scipy-stack in multiple OSX / Python install variations:

https://travis-ci.org/matthew-brett/scipy-stack-osx-testing/builds/26419147

For at least Python.org 3.3.5, no virtualenv, I see this error:

======================================================================
FAIL: test_multiple_date_cols_chunked (pandas.io.tests.test_parsers.TestCParserLowMemory)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/pandas/io/tests/test_parsers.py", line 1740, in test_multiple_date_cols_chunked
    tm.assert_frame_equal(chunks[2], df[4:])
File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/pandas/util/testing.py", line 585, in assert_frame_equal
    check_exact=check_exact)
File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/pandas/util/testing.py", line 532, in assert_series_equal
    assert_almost_equal(left.values, right.values, check_less_precise)
File "testing.pyx", line 58, in pandas._testing.assert_almost_equal (pandas/src/testing.c:2536)
File "testing.pyx", line 93, in pandas._testing.assert_almost_equal (pandas/src/testing.c:1859)
File "testing.pyx", line 69, in pandas._testing.assert_almost_equal (pandas/src/testing.c:1556)
AssertionError: nan != 'KORD'

See: https://travis-ci.org/matthew-brett/scipy-stack-osx-testing/jobs/26419150

For the exact same set of packages / Python version but in a virtualenv, I don't see the error:

https://travis-ci.org/matthew-brett/scipy-stack-osx-testing/jobs/26419153

Is there something random in the test or the code that could explain this?

jreback commented 10 years ago

nothing random in there; does this recur?

matthew-brett commented 10 years ago

I've only seen it for that entry in the test grid (Python.org 3.3.5, OSX 10.9, no virtualenv).

It's passing on the same Python in a virtualenv, on Python.org 3.4, on macports 3.3.5 and on Python.org 3.3.5 on OSX 10.6:

https://travis-ci.org/matthew-brett/scipy-stack-osx-testing/jobs/26419153 https://travis-ci.org/matthew-brett/scipy-stack-osx-testing/jobs/26419151 https://travis-ci.org/matthew-brett/scipy-stack-osx-testing/jobs/26419162 http://nipy.bic.berkeley.edu/builders/scipy-stack-3.3.5-wheel-requires/builds/20

matthew-brett commented 10 years ago

And passes on a rerun in (as far as I can tell) the exact same circumstances:

https://travis-ci.org/matthew-brett/scipy-stack-osx-testing/jobs/26429160

jreback commented 10 years ago

I suppose it's possible that a disk fault that has it reading only a part of the file could cause a comparison error. (this test is a chunked read vs the entire file)

I have seen similar random occurrences with the disk i/o on HDFStore once in a while; but which don't recur

closing - but pls reopen / make a new issue if u find other issues