OverflowError, possibly because of trying to draw too many lines?

[warrickball]

I'm trying to run PBjam on a TESS red giant that I think should work fine using this script:

import numpy as np
import pbjam

s = pbjam.session(ID=299899690, numax=(80.0, 0.8), dnu=(7.75, 0.0775),
                  teff=(5127, 60), bp_rp=(1.0860, 0.0025), 
                  make_plots=True, use_cached=True)
s()

but this fails, @nielsenmb says at the ε plotting, with the following error:

/home/wball/pypi/PBjam/pbjam/jar.py:244: UserWarning: Could not find long cadence data for 299899690 in cache, checking MAST...
  warnings.warn('Could not find %s cadence data for %s in cache, checking MAST...' % (lkwargs['cadence'], id))
Querying MAST for 299899690
Warning: 31% (6168/19692) of the cadences will be ignored due to the quality mask (quality_bitmask=175).
/home/wball/.local/lib/python3.7/site-packages/lightkurve/periodogram.py:752: LightkurveWarning: Input light curve will be normalized.
  LightkurveWarning)
 15%|█████████████████████████▎           | 2996/20000 [00:34<03:14, 87.52it/s]Converged after 3000 iterations.
 15%|█████████████████████████▎           | 3000/20000 [00:34<03:18, 85.73it/s]
Traceback (most recent call last):
  File "/home/wball/.local/lib/python3.7/site-packages/matplotlib/backends/backend_agg.py", line 146, in draw_path
    self._renderer.draw_path(gc, path, transform, rgbFace)
OverflowError: In draw_path: Exceeded cell block limit

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "TOI866.py", line 10, in <module>
    s()
  File "/home/wball/pypi/PBjam/pbjam/jar.py", line 643, in __call__
    st(norders=norders, model_type=self.pb_model_type)
  File "/home/wball/pypi/PBjam/pbjam/star.py", line 146, in __call__
    self.run_epsilon(bw_fac=bw_fac)
  File "/home/wball/pypi/PBjam/pbjam/star.py", line 107, in run_epsilon
    self.epsilon.plot(self.pg).savefig(self.bpath + os.sep + f'epsilon_{self.ID}.png')
  File "/home/wball/.local/lib/python3.7/site-packages/matplotlib/figure.py", line 2180, in savefig
    self.canvas.print_figure(fname, **kwargs)
  File "/home/wball/.local/lib/python3.7/site-packages/matplotlib/backend_bases.py", line 2082, in print_figure
    **kwargs)
  File "/home/wball/.local/lib/python3.7/site-packages/matplotlib/backends/backend_agg.py", line 527, in print_png
    FigureCanvasAgg.draw(self)
  File "/home/wball/.local/lib/python3.7/site-packages/matplotlib/backends/backend_agg.py", line 388, in draw
    self.figure.draw(self.renderer)
  File "/home/wball/.local/lib/python3.7/site-packages/matplotlib/artist.py", line 38, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "/home/wball/.local/lib/python3.7/site-packages/matplotlib/figure.py", line 1709, in draw
    renderer, self, artists, self.suppressComposite)
  File "/home/wball/.local/lib/python3.7/site-packages/matplotlib/image.py", line 135, in _draw_list_compositing_images
    a.draw(renderer)
  File "/home/wball/.local/lib/python3.7/site-packages/matplotlib/artist.py", line 38, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "/home/wball/.local/lib/python3.7/site-packages/matplotlib/axes/_base.py", line 2647, in draw
    mimage._draw_list_compositing_images(renderer, self, artists)
  File "/home/wball/.local/lib/python3.7/site-packages/matplotlib/image.py", line 135, in _draw_list_compositing_images
    a.draw(renderer)
  File "/home/wball/.local/lib/python3.7/site-packages/matplotlib/artist.py", line 38, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "/home/wball/.local/lib/python3.7/site-packages/matplotlib/lines.py", line 802, in draw
    renderer.draw_path(gc, tpath, affine.frozen())
  File "/home/wball/.local/lib/python3.7/site-packages/matplotlib/backends/backend_agg.py", line 148, in draw_path
    raise OverflowError("Exceeded cell block limit (set "
OverflowError: Exceeded cell block limit (set 'agg.path.chunksize' rcparam)

[grd349]

Thanks @warrickball - I'll take a look at this. Did you set the matplotlib backend to be 'agg' somehow? Does this work if you don't do this or is the 'agg' set in pbjam?

P.s. (Is this star particularly metal poor?)

[warrickball]

My default backend is TkAgg, so I just tried TkCairo instead and it works. I've tried my Qt5 options too: Qt5Agg fails and Qt5Cairo succeeds.

I just looked up the star and it seems to be moderately metal-poor ([Fe/H] ≈ -0.6). Also, my input parameters are wrong but a bug is a bug!

[grd349]

@warrickball - See PR on the issue. Could you run again with your back end variations?

[warrickball]

Never sure whether to comment here or the PR but the branch ran to completion with both TkAgg and Qt5Agg backends, which were failing before. It also still works with TkCairo.

[grd349]

Super - thanks Warrick.

Maybe Martin could merge this PR and close the issue. (Subject to review).

Might also be worth an update as the plots are better now.

On Fri, 23 Aug 2019, 18:01 Warrick Ball, notifications@github.com wrote:

Never sure whether to comment here or the PR but the branch ran to completion with both TkAgg and Qt5Agg backends, which were failing before. It also still works with TkCairo.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/grd349/PBjam/issues/154?email_source=notifications&email_token=ADQZOCOJUBYKMVXE4RIYBETQGAJW5A5CNFSM4IOSLDZKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5AYZCA#issuecomment-524389512, or mute the thread https://github.com/notifications/unsubscribe-auth/ADQZOCP76JF2P755SZPVHV3QGAJW5ANCNFSM4IOSLDZA .

[nielsenmb]

Should be fixed now

[warrickball]

I haven't been able to follow the paper trail of what fixed/should have fixed this (PR #155 was closed without merging) but I've just hit this again after cloning master. As a reminder, here's the breaking script:

import pbjam

s = pbjam.session(ID='TIC299899690', numax=(80.0, 0.8), dnu=(7.75, 0.0775),
                  teff=(5127, 60), bp_rp=(1.0860, 0.0025), use_cached=True)
s(make_plots=True)

and the full terminal output:

Warning: 31% (6168/19692) of the cadences will be ignored due to the quality mask (quality_bitmask=175).
Starting KDE estimation
/home/wball/pypi/PBjam/pbjam/priors.py:120: UserWarning: Only 69 star(s) near provided numax. Expanding the range to include ~100 stars.
  'Expanding the range to include ~100 stars.')
Steps taken: 2000
Chains reached stationary state after 2000 iterations.
Star TIC299899690 produced an exception of type OverflowError occurred. Arguments:
("Exceeded cell block limit (set 'agg.path.chunksize' rcparam)",)

As before, it doesn't crash like this if I set my Matplotlib backend to TkCairo.

Instead, it crashes with

Warning: 31% (6168/19692) of the cadences will be ignored due to the quality mask (quality_bitmask=175).
Starting KDE estimation
/home/wball/pypi/PBjam/pbjam/priors.py:120: UserWarning: Only 69 star(s) near provided numax. Expanding the range to include ~100 stars.
  'Expanding the range to include ~100 stars.')
Steps taken: 2000
Chains reached stationary state after 2000 iterations.
Starting asymptotic peakbagging
Steps taken: 2000
Steps taken: 3000
Steps taken: 4000
Steps taken: 5000
Steps taken: 6000
Steps taken: 7000
Steps taken: 8000
Steps taken: 9000
Steps taken: 10000
Steps taken: 11000
Steps taken: 12000
Steps taken: 13000
Steps taken: 14000
Steps taken: 15000
Steps taken: 16000
Chains reached stationary state after 16000 iterations.
/home/wball/.local/lib/python3.7/site-packages/numpy/core/fromnumeric.py:3257: RuntimeWarning: Mean of empty slice.
  out=out, **kwargs)
Star TIC299899690 produced an exception of type ValueError occurred. Arguments:
('Initial state has a large condition number. Make sure that your walkers are linearly independent for the best performance',)

but that's a separate issue.