ZeroDivisionError raised when combining flats

[simontorres]

There is a ZeroDivisionError when combining flats, more specific in .../astropy/combiner.py in this operation xs = int(1986) ys = int(4081) no_chunks = int(81) xstep = max(1, int(xs/no_chunks)) ystep = max(1, int(ys / (1 + no_chunks - int(xs / xstep))))

So far it seems that the reason is no_chunks being too high (81). no_chunks depends on memory_limit (self.memlim) currently set as 5E6. What happens is that the combiner divides the image in no_chunks to limit the use of memory.

I set memory_limit to 6E6 and this problem hasn't occurred yet, in fact is running too slow.

[simontorres]

Got this error again. It is consistent with the first guess and is an issue with ccdproc combiner instead.

Will go deeper with this later.

INFO: Combining and trimming bias frames: [__main__]
INFO: h.0111_bias_750ATTN2.fits [__main__]
INFO: h.0112_bias_750ATTN2.fits [__main__]
INFO: h.0113_bias_750ATTN2.fits [__main__]
INFO: h.0114_bias_750ATTN2.fits [__main__]
INFO: h.0115_bias_750ATTN2.fits [__main__]
INFO: h.0116_bias_750ATTN2.fits [__main__]
INFO: h.0117_bias_750ATTN2.fits [__main__]
INFO: h.0118_bias_750ATTN2.fits [__main__]
INFO: h.0119_bias_750ATTN2.fits [__main__]
INFO: h.0120_bias_750ATTN2.fits [__main__]
INFO: h.0121_bias_750ATTN2.fits [__main__]
INFO: h.0122_bias_750ATTN2.fits [__main__]
INFO: h.0123_bias_750ATTN2.fits [__main__]
INFO: h.0124_bias_750ATTN2.fits [__main__]
INFO: h.0125_bias_750ATTN2.fits [__main__]
INFO: h.0126_bias_750ATTN2.fits [__main__]
INFO: h.0127_bias_750ATTN2.fits [__main__]
INFO: h.0128_bias_750ATTN2.fits [__main__]
INFO: h.0129_bias_750ATTN2.fits [__main__]
INFO: h.0130_bias_750ATTN2.fits [__main__]
INFO: h.0286_bias_750ATTN2.fits [__main__]
INFO: h.0287_bias_750ATTN2.fits [__main__]
INFO: h.0288_bias_750ATTN2.fits [__main__]
INFO: h.0289_bias_750ATTN2.fits [__main__]
INFO: h.0290_bias_750ATTN2.fits [__main__]
INFO: h.0291_bias_750ATTN2.fits [__main__]
INFO: h.0292_bias_750ATTN2.fits [__main__]
INFO: h.0293_bias_750ATTN2.fits [__main__]
INFO: h.0294_bias_750ATTN2.fits [__main__]
INFO: h.0295_bias_750ATTN2.fits [__main__]
INFO: h.0320_bias_750ATTN2.fits [__main__]
INFO: h.0321_bias_750ATTN2.fits [__main__]
INFO: h.0322_bias_750ATTN2.fits [__main__]
INFO: h.0323_bias_750ATTN2.fits [__main__]
INFO: h.0324_bias_750ATTN2.fits [__main__]
INFO: h.0325_bias_750ATTN2.fits [__main__]
INFO: h.0326_bias_750ATTN2.fits [__main__]
INFO: h.0327_bias_750ATTN2.fits [__main__]
INFO: h.0328_bias_750ATTN2.fits [__main__]
INFO: h.0329_bias_750ATTN2.fits [__main__]
INFO: splitting each image into 104 chunks to limit memory usage to 6000000.0 bytes. [ccdproc.combiner]
Traceback (most recent call last):
  File "/user/simon/development/soar/goodman_ccdreduction/goodman_ccdreduction.py", line 730, in <module>
    main()
  File "/user/simon/development/soar/goodman_ccdreduction/goodman_ccdreduction.py", line 160, in __call__
    self.create_master_bias(ic, self.slit, self.memlim)
  File "/user/simon/development/soar/goodman_ccdreduction/goodman_ccdreduction.py", line 545, in create_master_bias
    sigma_clip_low_thresh=3.0, sigma_clip_high_thresh=3.0)
  File "/usr/lib/python2.7/site-packages/ccdproc-1.1.dev899-py2.7.egg/ccdproc/combiner.py", line 587, in combine
    ystep = max(1, int(ys / (1 + no_chunks - int(xs / xstep))))
ZeroDivisionError: division by zero

[simontorres]

Mental Note

This happens when the data to combined is too large and the memlim is too low In this case I am removing extra bias leaving only the first ten instead of thirty

[simontorres]

Did a quick test code to find out when this ZeroDivisionError ocurrs and it seems that is only when the memory limit is set to 1MB and the data above 59MB (I need to double check this last one).

Question: Why would anyone want to set a memory limit to 1MB? I wouldn't but I got here because python rises MemoryError at certain higher values.

Will continue...