STScI-Citizen-Science / MTPipeline

Pipeline to produce CR rejected, astrodrizzled, png's of HST WFPC2 solar system data.
6 stars 1 forks source link

Merging travis-fix with master #108

Closed ktfhale closed 10 years ago

ktfhale commented 10 years ago

As of Build#37, run on July 7th 2014, the pipeline installs, and its tests pass, on the Travis virtual environment. Three major things have been done to make this happen:

  1. The CPLUS_INCLUDE_PATH and LIBRARY_PATH have been manually set in .travis.yml to /sw/include and /sw/lib, respectively, where we set the cfitsio library to build. This is terribly kludgy, as it overwrites anything that might have already been in these paths. At the moment, they are undefined prior to our setting them. Curiously, attempting to merely append the /sw directories doesn't work. This is something to come back to, and a potential source of C-related errors in the future.
  2. In setup.py, we force using Pillow version 2.4.0. Installing the latest version appears to cause complications in the installation of matplotlib. In the new version of Pillow, Pillow successfully builds using multiple processes, doing some kind of multithreading. However, after Pillow is installed by this method, matplotlib tries to install in the same way, and this produces a fatal exception.
  3. In setup.py, we force using astropy version 0.3.2. At the moment, setup.py automatically grabs version 0.4rc1, which is apparently unreleased as per their changelog. Using this version appears to result in complications with cython, and may be related to their Ticket#1198, which was closed last year.

Additionally, the python test files have been rearranged. Four test files are listed in this request as deletions, however all four of them are already present (in both the master and travis-fix branches) inside tests/legacy. The merge should remove the unnecessary duplications.

It's a little concerning how easily updates to our dependencies can introduce errors in our travis build. However, neither the Pillow or astropy issues appear impact building the pipeline on my STScI laptop, probably because it's using the older versions of those packages by default.

ktfhale commented 10 years ago

Alright, so the Travis build is erroring. It appears to fail to find matplotlib on source forge. It does this:

Searching for matplotlib
Reading https://pypi.python.org/simple/matplotlib/
Reading http://matplotlib.org
Reading http://matplotlib.sourceforge.net
Reading http://sourceforge.net/project/showfiles.php?group_id=80706
Reading http://sourceforge.net/project/showfiles.php?group_id=80706&package_id=82474
Reading http://sourceforge.net/projects/matplotlib/files/matplotlib/matplotlib-1.1.0/
Reading http://sourceforge.net/projects/matplotlib/files/matplotlib/matplotlib-1.1.1/
Reading https://sourceforge.net/project/showfiles.php?group_id=80706&package_id=278194
Reading https://sourceforge.net/project/showfiles.php?group_id=80706&package_id=82474
Reading https://sourceforge.net/projects/matplotlib/files/matplotlib/matplotlib-0.99.1/
Reading https://sourceforge.net/projects/matplotlib/files/matplotlib/matplotlib-0.99.3/
Reading https://sourceforge.net/projects/matplotlib/files/matplotlib/matplotlib-1.0
error: None

When it should be doing this:

Searching for matplotlib
Reading https://pypi.python.org/simple/matplotlib/
Reading http://matplotlib.org
Reading http://matplotlib.sourceforge.net
Reading http://sourceforge.net/project/showfiles.php?group_id=80706
Reading http://sourceforge.net/project/showfiles.php?group_id=80706&package_id=82474
Reading http://sourceforge.net/projects/matplotlib/files/matplotlib/matplotlib-1.1.0/
Reading http://sourceforge.net/projects/matplotlib/files/matplotlib/matplotlib-1.1.1/
Reading https://sourceforge.net/project/showfiles.php?group_id=80706&package_id=278194
Reading https://sourceforge.net/project/showfiles.php?group_id=80706&package_id=82474
Reading https://sourceforge.net/projects/matplotlib/files/matplotlib/matplotlib-0.99.1/
Reading https://sourceforge.net/projects/matplotlib/files/matplotlib/matplotlib-0.99.3/
Reading https://sourceforge.net/projects/matplotlib/files/matplotlib/matplotlib-1.0
Reading https://sourceforge.net/projects/matplotlib/files/matplotlib/matplotlib-1.0.1/
Best match: matplotlib 1.3.1
Downloading https://pypi.python.org/packages/source/m/matplotlib/matplotlib-1.3.1.tar.gz#md5=789699851de28a543f3d244cf657ff68
Processing matplotlib-1.3.1.tar.gz
Writing /tmp/easy_install-x6lSXE/matplotlib-1.3.1/setup.cfg
Running matplotlib-1.3.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-x6lSXE/matplotlib-1.3.1/egg-dist-tmp-e8eKYH
============================================================================
Edit setup.cfg to change the build options

BUILDING MATPLOTLIB
....

I'm rerunning both build#37 and build#38 to see what happens.

ktfhale commented 10 years ago

... and, without changing anything, the build now passes. I guess it was merely a hiccup somewhere in the tubes? The pull request can probably be accepted. Once done, Ticket#102 can also be closed.

acviana commented 10 years ago

@walyssonBarbosa I have accepted this merge so you can now branch off of master to update make_output_dict #81

ktfhale commented 10 years ago

There's some weird stuff going on with our Travis builds. It's failed when trying to just download astropy, with a "Connection reset by peer" error, and now it's failing with the same message when trying to grab stwcs through pip. Maybe something's up with either Travis's connection, or the connection to wherever these packages are hosted.