Doctest failures with new helpers

[bsipocz]

This came up when testing against the new helpers, and only in this setup: py2.7, dev astropy, np 1.9.3, astropy-helpers v1.0.5

I can't see at the moment which package is the weak reason for the fail.

platform darwin -- Python 2.7.10 -- py-1.4.30 -- pytest-2.7.3
rootdir: /private/var/folders/dc/hsm7tqpx2d57n7vb3k1l81xw0000gq/T/photutils-test-1qsinj, inifile: setup.cfg
plugins: cov, html, mpl, selenium, selenium, variables

Running tests with photutils version 0.2.dev1410.
Running tests in lib.macosx-10.10-x86_64-2.7/photutils docs.

Date: 2015-10-07T23:24:09

Platform: Darwin-14.5.0-x86_64-i386-64bit

Executable: /Users/bsipocz/.virtualenvs/astropy-dev/bin/python

Full Python Version: 
2.7.10 (default, May 26 2015, 20:29:34) 
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)]

encodings: sys: ascii, locale: UTF-8, filesystem: utf-8, unicode bits: 15
byteorder: little
float info: dig: 15, mant_dig: 15

Numpy: 1.9.3
Scipy: 0.16.0
Matplotlib: 1.4.3
scikit-image: 0.12dev
Astropy: 1.1.dev13864

collected 955 items 

photutils/segmentation.py .......
photutils/data/README.rst .
photutils/datasets/load.py .
photutils/datasets/data/README.rst .
photutils/datasets/tests/test_load.py ..s
photutils/datasets/tests/test_make.py .................
photutils/detection/tests/test_core.py ....................................
photutils/detection/tests/test_findstars.py .......................
photutils/geometry/tests/test_circular_overlap_grid.py ......................................................
photutils/geometry/tests/test_elliptical_overlap_grid.py ......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
photutils/geometry/tests/test_rectangular_overlap_grid.py .................................................................................
photutils/tests/test_aperture_photometry.py .............................................................................s........
photutils/tests/test_background.py ............................
photutils/tests/test_morphology.py ...........................
photutils/tests/test_psf_photometry.py ...........
photutils/tests/test_psfs.py .........
photutils/tests/test_segmentation.py ...............................................
photutils/utils/interpolation.py .
photutils/utils/tests/test_interpolation.py .........
photutils/utils/tests/test_prepare_data.py ......
photutils/utils/tests/test_random_state.py ......
../docs/changelog.rst .
../docs/index.rst .
../docs/photutils/aperture.rst .
../docs/photutils/background.rst F
../docs/photutils/datasets.rst .
../docs/photutils/detection.rst .
../docs/photutils/geometry.rst .
../docs/photutils/getting_started.rst .
../docs/photutils/high-level_API.rst .
../docs/photutils/install.rst .
../docs/photutils/morphology.rst F
../docs/photutils/overview.rst .
../docs/photutils/psf.rst .
../docs/photutils/segmentation.rst .
../docs/photutils/utils.rst .

==================================================================================== FAILURES =====================================================================================
_____________________________________________________________________ [doctest] docs/photutils/background.rst _____________________________________________________________________
100 statistics is called sigma clipping.  Briefly, pixels that are above
101 or below a specified sigma level from the median are discarded and the
102 statistics are recalculated.  The procedure is typically repeated over
103 a number of iterations or until convergence is reached.  This method
104 provides a better estimate of the background and background noise
105 levels::
106 
107     >>> from astropy.stats import sigma_clipped_stats
108     >>> mean, median, std = sigma_clipped_stats(data, sigma=3.0)
109     >>> print(mean, median, std)    # doctest: +FLOAT_CMP
Expected:
    (5.19888175541, 5.15557244947, 2.09393539966)
Got:
    (5.1991386516217908, 5.1555874333582912, 2.0942752121329691)

/private/var/folders/dc/hsm7tqpx2d57n7vb3k1l81xw0000gq/T/photutils-test-1qsinj/docs/photutils/background.rst:109: DocTestFailure
--------------------------------------------------------------------------------- pytest-selenium ---------------------------------------------------------------------------------

_____________________________________________________________________ [doctest] docs/photutils/morphology.rst _____________________________________________________________________
139     >>> from astropy.stats import sigma_clipped_stats
140     >>> from photutils.morphology import data_properties
141     >>> from photutils import properties_table
142     >>> mean, median, std = sigma_clipped_stats(data, sigma=3.0)
143     >>> data -= median    # subtract background
144     >>> props = data_properties(data)
145     >>> columns = ['id', 'xcentroid', 'ycentroid', 'semimajor_axis_sigma',
146     ...            'semiminor_axis_sigma', 'orientation']
147     >>> tbl = properties_table(props, columns=columns)
148     >>> print(tbl)
Differences (unified diff with -expected +actual):
    @@ -1,4 +1,4 @@
    - id   xcentroid     ycentroid   ... semiminor_axis_sigma  orientation
    -         pix           pix      ...         pix               rad
    + id   xcentroid     ycentroid   ... semiminor_axis_sigma  orientation
    +         pix           pix      ...         pix               rad
     --- ------------- ------------- ... -------------------- -------------
    -  1 14.0192015504 16.9885140744 ...         3.7763953673 1.05053207277
    +  1 14.0225090502 16.9901801466 ...        3.69777618702 1.04943689372

/private/var/folders/dc/hsm7tqpx2d57n7vb3k1l81xw0000gq/T/photutils-test-1qsinj/docs/photutils/morphology.rst:148: DocTestFailure
--------------------------------------------------------------------------------- pytest-selenium ---------------------------------------------------------------------------------

[bsipocz]

It seems that astropy-helpers triggers it, rebuilding the current master with the latest astropy-dev and helpers v1.0.3 works fine for the whole travis matrix. https://travis-ci.org/bsipocz/photutils/builds/84202936

cc @embray as he may know the trick, and also this may effect other affiliated packages.

[embray]

Except it looks like you're actually getting some numerical differences in some tests...

[larrybradley]

See https://github.com/astropy/photutils/pull/286 for the fix.

[bsipocz]

Ohh, OK, now I see. Somehow it's a bit confusing why we didn't see these fails with the old helper even though using the same astropy dev master. Also the fact that the rst files are not picked up by the py3 test run added to my confusion, as these apparently fails with py3, too, travis just simply fails to notice it.

[larrybradley]

The doctests don't run with python 3 yet. And we don't test python 2.6 with astropy dev. So, the only failing test was python 2.7 with astropy dev.

Did the old astropy-helpers actually run the doctests?

[bsipocz]

Yes, you're right it didn't run the rst tests due to the same testing issue that was raised on the mailing list by Tom A. Should have identified it, sorry for the extra work.

[bsipocz]

Closing as this got fixed in #286