search

sagemath / sage

Main repository of SageMath
https://www.sagemath.org
Other
1.37k stars 466 forks source link

[with patch and with spkg] Upgrade matplotlib to 0.98.5.3-svn6910 #4774

Closed jasongrout closed 15 years ago

jasongrout commented 15 years ago

Nice things include very nice arrow functionality, new legend functionality, and a fill_between keyword:

http://matplotlib.sourceforge.net/users/whats_new.html

CC: @sagetrac-ekirkman @mwhansen

Component: packages: standard

Issue created by migration from https://trac.sagemath.org/ticket/4774

85eec1a4-3d04-4b4d-b711-d4db03337c41 commented 15 years ago
comment:1

0.98.5 which is a bug fix release over 0.98.4 is out.

Cheers,

Michael

jasongrout commented 15 years ago
comment:3
[03:47] <mabshoff> jason-: if you want to update matplotlib make sure to base it off the spkg in 3.2.2.a2.
[03:47] <jason-> okay
[03:47] <mabshoff> This one: http://sage.math.washington.edu/home/mabshoff/release-cycles-3.2.2/alpha2/matplotlib-0.98.3.p4.spkg
85eec1a4-3d04-4b4d-b711-d4db03337c41 commented 15 years ago
comment:4

Latest upstream is 0.98.5.2.

Cheers,

Michael

jasongrout commented 15 years ago
comment:5

A very preliminary package is up at http://sage.math.washington.edu/home/jason/matplotlib-svn6821.spkg , just in case someone wants to start using it (like Emily).

Lots of deprecation warnings and it probably also breaks stuff. I'm fixing that now. You probably should delete the directory $SAGE_ROOT/local/lib/python2.5/site-packages/matplotlib before installing to get rid of old cruft.

jasongrout commented 15 years ago
comment:7

(ekirkman: I just posted a very preliminary spkg.)

jasongrout commented 15 years ago
comment:8

This upgrade should fix #4465

jasongrout commented 15 years ago

apply in normal sage repository

jasongrout commented 15 years ago
comment:9

Attachment: trac_4774_arrow.patch.gz

I've updated the spkg at http://sage.math.washington.edu/home/jason/matplotlib-svn6821.spkg

It should be ready to be reviewed now.

89c6e537-b2e3-45e6-882d-d4957b74ffe5 commented 15 years ago
comment:11

Once I've deleted the old matplotlibrc files from everywhere, as proposed in

http://groups.google.com/group/sage-devel/browse_thread/thread/11704ed70dc0d6e3

I get failures related to ft2font:

sage: from matplotlib import ft2font

ImportError: dlopen(/Users/rlmill/sage-3.3.alpha5/local/lib/python2.5/site-packages/matplotlib/ft2font.so, 2): Symbol not found: __cg_png_create_info_struct
  Referenced from: /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/ImageIO.framework/Versions/A/ImageIO
  Expected in: /Users/rlmill/sage-3.3.alpha5/local/lib//libpng12.0.dylib

Trying it again doesn't even give the error again: Sage just thrashes.

89c6e537-b2e3-45e6-882d-d4957b74ffe5 commented 15 years ago
comment:12

Moved my /opt/local branch and rebuilt, everything worked this time around. Sorry!

Positive review for the spkg and patches. The thread above resolved the matplotlibrc issue. Full sail!

89c6e537-b2e3-45e6-882d-d4957b74ffe5 commented 15 years ago
comment:13

OK, moving /opt/local out of the way wasn't enough:

sage: from matplotlib.pyplot import scatter

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<SNIP>
/Users/rlmill/sage-3.3.alpha5/local/lib/python2.5/site-packages/matplotlib/backends/backend_macosx.py in <module>()
     18 
     19 import matplotlib
---> 20 from matplotlib.backends import _macosx
     21 
     22 def show():

ImportError: dlopen(/Users/rlmill/sage-3.3.alpha5/local/lib/python2.5/site-packages/matplotlib/backends/_macosx.so, 2): Symbol not found: __cg_png_create_info_struct
  Referenced from: /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/ImageIO.framework/Versions/A/ImageIO
  Expected in: /Users/rlmill/sage-3.3.alpha5/local/lib//libpng12.0.dylib

So... wtf the linker is doing in /System/Library/Frameworks I can't say, but something is not right in Dodge.

85eec1a4-3d04-4b4d-b711-d4db03337c41 commented 15 years ago
comment:14

This spkg is twice as large as the previous:

-rw-r--r--@ 1 michaelabshoff  staff  4845513 Feb  2 17:28 matplotlib-0.98.3.p5.spkg
-rw-r--r--  1 michaelabshoff  staff  8833439 Feb  5 16:52 matplotlib-svn6821.spkg

There was some discussion on the matplotlib list about the new tarballs containing more documentation than the old ones, so we should look into getting rid of some of the documentation.

This spkg is important to get graph related plotting improvements into Sage, so let's try for 3.3.

Cheers,

Michael

85eec1a4-3d04-4b4d-b711-d4db03337c41 commented 15 years ago
comment:15

This needs work:

Aside from that:

Cheers,

Michael

85eec1a4-3d04-4b4d-b711-d4db03337c41 commented 15 years ago
comment:16

I debugged this some more with rlm and the problem is the following:

OPTIONAL USETEX DEPENDENCIES
               dvipng: /System/Library/Frameworks/ApplicationServices.frame
                       work/Versions/A/Frameworks/ImageIO.framework/Version
                       s/A/ImageIO
          ghostscript: 8.62
                latex: 3.1415926

So killing dvipng support on OSX for now should solve the problem. His dvipng comes out of usr/texbin/dvipng.

And while looking at this problem I also come up with a workable fix for the libpng issue once and for all - see

http://groups.google.com/group/sage-devel/t/5c655c1e1a2dc0b8

Cheers,

Michael

85eec1a4-3d04-4b4d-b711-d4db03337c41 commented 15 years ago
comment:17

Ok, the culprit is this extension: 

def build_macosx(ext_modules, packages):
    global BUILT_MACOSX
    if BUILT_MACOSX: return # only build it if you you haven't already
    module = Extension('matplotlib.backends._macosx',
                       ['src/_macosx.m'],
                       extra_link_args = ['-framework','Cocoa'],
                      )
    add_numpy_flags(module)
    ext_modules.append(module)
    BUILT_MACOSX = True

Since it links against a framework the only fix might be to fix the libpng.dylib issue on OSX once and for all.

Cheers,

Michael

jasongrout commented 15 years ago
comment:18

The next version, 0.98.3, is going to be released Real Soon Now (there's already an RC out). So I'm biding my time waiting for that before fixing the issues with the spkg.

jasongrout commented 15 years ago
comment:19

uh, that release is 0.98.5.3.

jasongrout commented 15 years ago
comment:20

Here are the directories that changed by more than 100Kb. A negative number means the directory decreased by that many kb. A positive number means it increased by that many kb:

('src/lib', -4280)
('src/lib/mpl_examples', -3108)
('src/lib/enthought', -1936)
('src/lib/enthought/traits', -1892)
('src/lib/mpl_examples/data', -1840)
('src/lib/enthought/traits/ui', -1012)
('src/lib/mpl_examples/pylab_examples', -728)
('src/lib/enthought/traits/ui/tk', -316)
('src/lib/enthought/traits/tests', -240)
('src/lib/mpl_examples/user_interfaces', -140)
('src/lib/enthought/traits/ui/tests', -124)
('src/lib/mpl_examples/api', -112)
('src/doc/api', 104)
('src/examples/tests/pngsuite', 108)
('src/unit', 116)
('src/examples/tests', 120)
('src/examples/user_interfaces', 144)
('src/examples/api', 144)
('src/lib/matplotlib/mpl-data', 480)
('src/lib/matplotlib/mpl-data/example', 528)
('src/lib/matplotlib', 764)
('src/examples/pylab_examples', 840)
('src/doc/_static', 1216)
('src/src', 1340)
('src/examples/data', 2152)
('src/doc/pyplots', 2424)
('src/examples', 3696)
('src/doc', 3828)
('src/', 4688)
jasongrout commented 15 years ago
comment:21

so from the above, it looks like it was indeed the docs that increased by about 4mb (uncompressed) .

jasongrout commented 15 years ago
comment:22

An updated spkg which addresses all of mabshoff's comments is here: http://sage.math.washington.edu/home/jason/matplotlib-0.98.5.3rc0-svn6910.spkg

I also updated the spkg to the most recent svn, which is after the release candidate for 0.98.5.3.

jasongrout commented 15 years ago
comment:23

The spkg installs cleanly for me (ubuntu 8.10).

jasongrout commented 15 years ago

Attachment: trac_4774_BIN.patch.gz

jasongrout commented 15 years ago
comment:24

I updated the bin repository patch to delete a reference to matplotlib that I missed before (in sage-env).

85eec1a4-3d04-4b4d-b711-d4db03337c41 commented 15 years ago

This patch is on top of Jason's latest spkg and resolved libpng12 linking issues on OSX

85eec1a4-3d04-4b4d-b711-d4db03337c41 commented 15 years ago
comment:25

Attachment: matplotlib-0.98.5.3rc0-svn6910.p0.patch.gz

The new spkg can be found at

http://sage.math.washington.edu/home/mabshoff/release-cycles-3.3/rc1/matplotlib-0.98.5.3rc0-svn6910.p0.spkg

All spkg changes by Jason look good, indeed they are very clean. I will review the patches to the various repos and hopefully Jason will give my changes a review.

Cheers,

Michael

85eec1a4-3d04-4b4d-b711-d4db03337c41 commented 15 years ago
comment:26

Note that even with libpng.dylib and libpng12.dylib on OSX we are definitely linking against libpng12.dylib.

Cheers,

Michael

85eec1a4-3d04-4b4d-b711-d4db03337c41 commented 15 years ago
comment:28

Positive review on Jason's patches to the Sage library as well as the bin repo.

Cheers,

Michael

jasongrout commented 15 years ago
comment:29

mabshoff's new spkg builds fine for me.

85eec1a4-3d04-4b4d-b711-d4db03337c41 commented 15 years ago
comment:30

Ok, there are some doctesting issues left with -long. Jason is looking into this.

Cheers,

Michael

85eec1a4-3d04-4b4d-b711-d4db03337c41 commented 15 years ago
comment:31

On sage.math we are having problems with the tkagg backend. It used to happen only occasionally, but now it happens every time I doctest plot.py:

File "/scratch/mabshoff/sage-3.3.rc1/devel/sage/sage/plot/plot.py", line 172:
    sage: grid(True)
Exception raised:
    Traceback (most recent call last):
      File "/scratch/mabshoff/sage-3.3.rc1/local/bin/ncadoctest.py", line 1231, in run_one_test
        self.run_one_example(test, example, filename, compileflags)
      File "/scratch/mabshoff/sage-3.3.rc1/local/bin/sagedoctest.py", line 38, in run_one_example
        OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
      File "/scratch/mabshoff/sage-3.3.rc1/local/bin/ncadoctest.py", line 1172, in run_one_example
        compileflags, 1) in test.globs
      File "<doctest __main__.example_0[60]>", line 1, in <module>
        grid(True)###line 172:
    sage: grid(True)
      File "/scratch/mabshoff/sage-3.3.rc1/local/lib/python2.5/site-packages/matplotlib/pyplot.py", line 2453, in grid
        ret =  gca().grid(*args, **kwargs)
      File "/scratch/mabshoff/sage-3.3.rc1/local/lib/python2.5/site-packages/matplotlib/pyplot.py", line 572, in gca
        ax =  gcf().gca(**kwargs)
      File "/scratch/mabshoff/sage-3.3.rc1/local/lib/python2.5/site-packages/matplotlib/pyplot.py", line 274, in gcf
        return figure()
      File "/scratch/mabshoff/sage-3.3.rc1/local/lib/python2.5/site-packages/matplotlib/pyplot.py", line 252, in figure
        **kwargs)
      File "/scratch/mabshoff/sage-3.3.rc1/local/lib/python2.5/site-packages/matplotlib/backends/backend_tkagg.py", line 90, in new_figure_manager
        window = Tk.Tk()
      File "/scratch/mabshoff/sage-3.3.rc1/local/lib/python2.5/lib-tk/Tkinter.py", line 1636, in __init__
        self.tk = _tkinter.create(screenName, baseName, className, interactive, wantobjects, useTk, sync, use)
    TclError: no display name and no $DISPLAY environment variable

We already disable tkagg on Itanium/Linux, so let's get rid of it globally.

Cheers,

Michael

85eec1a4-3d04-4b4d-b711-d4db03337c41 commented 15 years ago
comment:32

I have disabled tkagg globally and the problem above goes away. The spkg by the same names as the last one above is at

http://sage.math.washington.edu/home/mabshoff/release-cycles-3.3/rc1/matplotlib-0.98.5.3rc0-svn6910.p0.spkg

Cheers,

Michael

jasongrout commented 15 years ago
comment:33

okay, new spkg up here: http://sage.math.washington.edu/home/jason/matplotlib-0.98.5.3rc0-svn6910.p1.spkg

This adds a fix (i.e., kludge) that lets matplotlib draw very tiny errors, which were giving us problems before. I also cleaned up a few things from mabshoff's changes :).

85eec1a4-3d04-4b4d-b711-d4db03337c41 commented 15 years ago
comment:34

Jason's latest patch spkg fixes the problem, so an overall positive review via crossover between Jason's and my work.

Cheers,

Michael

85eec1a4-3d04-4b4d-b711-d4db03337c41 commented 15 years ago
comment:35

Merged

in Sage 3.3.rc1.

Cheers,

Michael