search

JuliaPlots / Plots.jl

Powerful convenience for Julia visualizations and data analysis
https://docs.juliaplots.org
Other
1.84k stars 354 forks source link

Failing PyPlot tests #70

Closed timholy closed 8 years ago

timholy commented 8 years ago

Most of the failures look like this:

PyPlot
[Plots.jl] Initializing backend: pyplot
INFO: Testing plot: pyplot:1:Lines
WARNING: Image did not match reference image /home/tim/.julia/v0.4/Plots/test/refimg/pyplot/ref1.png. err: ErrorException("Sizes (452,290) and (457,290) do not match")
WARNING: Should we make this the new reference image?
here000
huh0
huh
huh2
here00
here01
here02
here1
here2
here3
exited
  Failure :: (line:-1) :: fact was false
    Expression: image_comparison_tests(pkg,i,debug=debug,sigma=sigma,eps=eps) --> true
      Expected: true
      Occurred: nothing
INFO: Testing plot: pyplot:2:Functions, adding data, and animations
WARNING: Image did not match reference image /home/tim/.julia/v0.4/Plots/test/refimg/pyplot/ref2.png. err: ErrorException("Sizes (450,290) and (454,290) do not match")
WARNING: Should we make this the new reference image?
here000
huh0
huh
huh2
here00
here01
here02
here1
here2
here3
exited
  Failure :: (line:-1) :: fact was false
    Expression: image_comparison_tests(pkg,i,debug=debug,sigma=sigma,eps=eps) --> true
      Expected: true
      Occurred: nothing
INFO: Testing plot: pyplot:3:Parametric plots
WARNING: Image did not match reference image /home/tim/.julia/v0.4/Plots/test/refimg/pyplot/ref3.png. err: ErrorException("Sizes (456,290) and (462,290) do not match")
WARNING: Should we make this the new reference image?
here000
huh0
huh
huh2
here00
here01
here02
here1
here2
here3
exited
  Failure :: (line:-1) :: fact was false
    Expression: image_comparison_tests(pkg,i,debug=debug,sigma=sigma,eps=eps) --> true
      Expected: true
      Occurred: nothing
INFO: Testing plot: pyplot:4:Colors
WARNING: Image did not match reference image /home/tim/.julia/v0.4/Plots/test/refimg/pyplot/ref4.png. err: ErrorException("Sizes (438,290) and (446,290) do not match")
WARNING: Should we make this the new reference image?
here000
huh0
huh
huh2
here00
here01
here02
here1
here2
here3
exited
  Failure :: (line:-1) :: fact was false
    Expression: image_comparison_tests(pkg,i,debug=debug,sigma=sigma,eps=eps) --> true
      Expected: true
      Occurred: nothing
INFO: Testing plot: pyplot:5:Global
WARNING: Image did not match reference image /home/tim/.julia/v0.4/Plots/test/refimg/pyplot/ref5.png. err: ErrorException("Sizes (470,324) and (474,326) do not match")
WARNING: Should we make this the new reference image?
here000
huh0
huh
huh2
here00
here01
here02
here1
n
here2
here3
exited
  Failure :: (line:-1) :: fact was false
    Expression: image_comparison_tests(pkg,i,debug=debug,sigma=sigma,eps=eps) --> true
      Expected: true
      Occurred: nothing
INFO: Testing plot: pyplot:6:Two-axis
WARNING: Image did not match reference image /home/tim/.julia/v0.4/Plots/test/refimg/pyplot/ref6.png. err: ErrorException("Sizes (525,293) and (539,294) do not match")
WARNING: Should we make this the new reference image?
here000
huh0
huh
huh2
here00
here01
here02
here1
here2
here3
exited
  Failure :: (line:-1) :: fact was false
    Expression: image_comparison_tests(pkg,i,debug=debug,sigma=sigma,eps=eps) --> true
      Expected: true
      Occurred: nothing
INFO: Testing plot: pyplot:7:Arguments
WARNING: Image did not match reference image /home/tim/.julia/v0.4/Plots/test/refimg/pyplot/ref7.png. err: ErrorException("Sizes (449,290) and (451,290) do not match")
WARNING: Should we make this the new reference image?
here000
huh0
huh
huh2
here00
here01
here02
here1
here2
here3
exited
  Failure :: (line:-1) :: fact was false
    Expression: image_comparison_tests(pkg,i,debug=debug,sigma=sigma,eps=eps) --> true
      Expected: true
      Occurred: nothing
INFO: Testing plot: pyplot:8:Build plot in pieces
WARNING: Image did not match reference image /home/tim/.julia/v0.4/Plots/test/refimg/pyplot/ref8.png. err: ErrorException("Sizes (458,290) and (462,290) do not match")
WARNING: Should we make this the new reference image?
here000
huh0
huh
huh2
here00
here01
here02
here1
here2
here3
exited
  Failure :: (line:-1) :: fact was false
    Expression: image_comparison_tests(pkg,i,debug=debug,sigma=sigma,eps=eps) --> true
      Expected: true
      Occurred: nothing
INFO: Testing plot: pyplot:9:
WARNING: Image did not match reference image /home/tim/.julia/v0.4/Plots/test/refimg/pyplot/ref9.png. err: ErrorException("Sizes (458,290) and (463,290) do not match")
WARNING: Should we make this the new reference image?
here000
huh0
huh
huh2
here00
here01
here02
here1
here2
here3
exited
  Failure :: (line:-1) :: fact was false
    Expression: image_comparison_tests(pkg,i,debug=debug,sigma=sigma,eps=eps) --> true
      Expected: true
      Occurred: nothing
INFO: Testing plot: pyplot:10:Heatmaps
WARNING: Image did not match reference image /home/tim/.julia/v0.4/Plots/test/refimg/pyplot/ref10.png. err: ErrorException("Sizes (424,287) and (430,289) do not match")
WARNING: Should we make this the new reference image?
here000
huh0
huh
huh2
here00
here01
here02
here1
here2
here3
exited
  Failure :: (line:-1) :: fact was false
    Expression: image_comparison_tests(pkg,i,debug=debug,sigma=sigma,eps=eps) --> true
      Expected: true
      Occurred: nothing
INFO: Testing plot: pyplot:11:Line types
WARNING: Image did not match reference image /home/tim/.julia/v0.4/Plots/test/refimg/pyplot/ref11.png. err: ErrorException("Sizes (456,290) and (461,290) do not match")
WARNING: Should we make this the new reference image?
here000
huh0
huh
huh2
here00
here01
here02
here1
here2
here3
exited
  Failure :: (line:-1) :: fact was false
    Expression: image_comparison_tests(pkg,i,debug=debug,sigma=sigma,eps=eps) --> true
      Expected: true
      Occurred: nothing
INFO: Testing plot: pyplot:12:Line styles
WARNING: Image did not match reference image /home/tim/.julia/v0.4/Plots/test/refimg/pyplot/ref12.png. err: ErrorException("Sizes (452,290) and (457,290) do not match")
WARNING: Should we make this the new reference image?
here000
huh0
huh
huh2
here00
here01
here02
here1
here2
here3
exited
  Failure :: (line:-1) :: fact was false
    Expression: image_comparison_tests(pkg,i,debug=debug,sigma=sigma,eps=eps) --> true
      Expected: true
      Occurred: nothing
INFO: Testing plot: pyplot:13:Marker types
WARNING: Image did not match reference image /home/tim/.julia/v0.4/Plots/test/refimg/pyplot/ref13.png. err: ErrorException("Sizes (445,290) and (449,290) do not match")
WARNING: Should we make this the new reference image?
here000
huh0
huh
huh2
here00
here01
here02
here1
here2
here3
exited
  Failure :: (line:-1) :: fact was false
    Expression: image_comparison_tests(pkg,i,debug=debug,sigma=sigma,eps=eps) --> true
      Expected: true
      Occurred: nothing
INFO: Testing plot: pyplot:14:Bar
WARNING: Image did not match reference image /home/tim/.julia/v0.4/Plots/test/refimg/pyplot/ref14.png. err: ErrorException("Sizes (452,290) and (457,290) do not match")
WARNING: Should we make this the new reference image?
here000
huh0
huh
huh2
here00
here01
here02
here1
here2
here3
exited
  Failure :: (line:-1) :: fact was false
    Expression: image_comparison_tests(pkg,i,debug=debug,sigma=sigma,eps=eps) --> true
      Expected: true
      Occurred: nothing
INFO: Testing plot: pyplot:15:Histogram
WARNING: Image did not match reference image /home/tim/.julia/v0.4/Plots/test/refimg/pyplot/ref15.png. err: ErrorException("Sizes (448,290) and (451,290) do not match")
WARNING: Should we make this the new reference image?
here000
huh0
huh
huh2
here00
here01
here02
here1
here2
here3
exited
  Failure :: (line:-1) :: fact was false
    Expression: image_comparison_tests(pkg,i,debug=debug,sigma=sigma,eps=eps) --> true
      Expected: true
      Occurred: nothing
INFO: Testing plot: pyplot:16:Subplots
WARNING: Image did not match reference image /home/tim/.julia/v0.4/Plots/test/refimg/pyplot/ref16.png. err: ErrorException("Sizes (449,290) and (453,290) do not match")
WARNING: Should we make this the new reference image?
here000
huh0
huh
huh2
here00
here01
here02
here1
here2
here3
exited
  Failure :: (line:-1) :: fact was false
    Expression: image_comparison_tests(pkg,i,debug=debug,sigma=sigma,eps=eps) --> true
      Expected: true
      Occurred: nothing
INFO: Testing plot: pyplot:17:Adding to subplots
WARNING: Image did not match reference image /home/tim/.julia/v0.4/Plots/test/refimg/pyplot/ref17.png. err: ErrorException("Sizes (449,290) and (453,290) do not match")
WARNING: Should we make this the new reference image?
here000
huh0
huh
huh2
here00
here01
here02
here1
here2
here3
exited
  Failure :: (line:-1) :: fact was false
    Expression: image_comparison_tests(pkg,i,debug=debug,sigma=sigma,eps=eps) --> true
      Expected: true
      Occurred: nothing
INFO: Testing plot: pyplot:18:
WARNING: Image did not match reference image /home/tim/.julia/v0.4/Plots/test/refimg/pyplot/ref18.png. err: ErrorException("Sizes (455,290) and (460,290) do not match")
WARNING: Should we make this the new reference image?
here000
huh0
huh
huh2
here00
here01
here02
here1
here2
here3
exited
  Failure :: (line:-1) :: fact was false
    Expression: image_comparison_tests(pkg,i,debug=debug,sigma=sigma,eps=eps) --> true
      Expected: true
      Occurred: nothing
INFO: Testing plot: pyplot:20:Annotations
/usr/lib/pymodules/python2.7/matplotlib/font_manager.py:1236: UserWarning: findfont: Font family ['Helvetica'] not found. Falling back to Bitstream Vera Sans
  (prop.get_family(), self.defaultFamily[fontext]))
/usr/lib/pymodules/python2.7/matplotlib/font_manager.py:1246: UserWarning: findfont: Could not match :family=Bitstream Vera Sans:style=normal:variant=normal:weight=normal:stretch=normal:size=14.0. Returning /usr/share/matplotlib/mpl-data/fonts/ttf/cmb10.ttf
  UserWarning)
/usr/lib/pymodules/python2.7/matplotlib/font_manager.py:1246: UserWarning: findfont: Could not match :family=Bitstream Vera Sans:style=normal:variant=normal:weight=normal:stretch=normal:size=16.0. Returning /usr/share/matplotlib/mpl-data/fonts/ttf/cmb10.ttf
  UserWarning)
/usr/lib/pymodules/python2.7/matplotlib/font_manager.py:1236: UserWarning: findfont: Font family ['courier'] not found. Falling back to Bitstream Vera Sans
  (prop.get_family(), self.defaultFamily[fontext]))
/usr/lib/pymodules/python2.7/matplotlib/font_manager.py:1246: UserWarning: findfont: Could not match :family=Bitstream Vera Sans:style=normal:variant=normal:weight=normal:stretch=normal:size=20.0. Returning /usr/share/matplotlib/mpl-data/fonts/ttf/cmb10.ttf
  UserWarning)
WARNING: Image did not match reference image /home/tim/.julia/v0.4/Plots/test/refimg/pyplot/ref20.png. err: ErrorException("Sizes (448,290) and (451,290) do not match")
WARNING: Should we make this the new reference image?
here000
huh0
huh
huh2
here00
here01
here02
here1
here2
here3
exited
  Failure :: (line:-1) :: fact was false
    Expression: image_comparison_tests(pkg,i,debug=debug,sigma=sigma,eps=eps) --> true
      Expected: true
      Occurred: nothing
Out of 21 total facts:
  Verified: 2
  Failed:   19

Below are the images that looked obviously different (except for some font issues): error1 error2 error3 error4

The last one is subtle, but the fonts look more crowded on the right.

tbreloff commented 8 years ago

Thanks for this, Tim. Sorry about all those random print lines... I was trying to figure out why I can no longer bring up a Gtk dialog on my Centos machine (works fine on my Mac... still haven't resolved this). I also saw similar mismatched sizes on my Centos machine, so I have taken to running all tests on my Mac, which seems to match Travis.

Many of these issues are likely due to differences in the underlying graphics library versions and/or different installed fonts... I have no idea how to resolve that robustly, and I'm open to ideas.

The missing horizontal/vertical lines is just from outdated reference images.

The 2-axis example and the markers example both show real issues... I'll have to investigate those. (though I suspect the markers example is due to a recent attempt to control the z-ordering of series)

Thanks!

tbreloff commented 8 years ago

These should all be fixed on master now. For PyPlot, I hopefully fixed the platform-specific image sizing issue as well.