search

ACCarnall / bagpipes

Bagpipes is a state of the art code for generating realistic model galaxy spectra and fitting these to spectroscopic and photometric observations. Users should install with pip, not by cloning the repository.
http://bagpipes.readthedocs.io
GNU General Public License v3.0
80 stars 42 forks source link

Possible to add error handling when running bagpipes on galaxy catalogues? #79

Closed shruti-ramaiya closed 5 months ago

shruti-ramaiya commented 5 months ago

Hi,

I'm using BAGPIPES on a large catalogue of galaxies with multiwavelength data. The code runs smoothly however, there are a few galaxies in the image that sometimes BAGPIPES is unable to fit (perhaps due to too few photometric datapoints). In such cases, it throws up the following error when running fit catalogue,

/mnt/users/ramaiyas/.virtualenvs/multinest/lib/python3.10/site-packages/bagpipes/models/star_formation_history.py:106: RuntimeWarning: divide by zero encountered in scalar divide self.component_sfrs[name] = desired_mass/mass_norm /mnt/users/ramaiyas/.virtualenvs/multinest/lib/python3.10/site-packages/bagpipes/models/star_formation_history.py:106: RuntimeWarning: invalid value encountered in multiply self.component_sfrs[name] = desired_mass/mass_norm /mnt/users/ramaiyas/.virtualenvs/multinest/lib/python3.10/site-packages/bagpipes/models/model_galaxy.py:559: RuntimeWarning: divide by zero encountered in log10 self.uvj = -2.5np.log10(self._calculate_photometry(0., uvj=True)) Traceback (most recent call last): File "/mnt/users/ramaiyas/bagpipes-DES/lognorm_CF00.py", line 115, in fit_cat.fit(n_live = 700) File "/mnt/users/ramaiyas/.virtualenvs/multinest/lib/python3.10/site-packages/bagpipes/catalogue/fit_catalogue.py", line 184, in fit self._fit_object(self.IDs[i], verbose=verbose, n_live=n_live, File "/mnt/users/ramaiyas/.virtualenvs/multinest/lib/python3.10/site-packages/bagpipes/catalogue/fit_catalogue.py", line 328, in _fit_object self.obj_fit.plot_sfh_posterior() File "/mnt/users/ramaiyas/.virtualenvs/multinest/lib/python3.10/site-packages/bagpipes/fitting/fit.py", line 289, in plot_sfh_posterior return plotting.plot_sfh_posterior(self, show=show, save=save, File "/mnt/users/ramaiyas/.virtualenvs/multinest/lib/python3.10/site-packages/bagpipes/plotting/plot_sfh_posterior.py", line 26, in plot_sfh_posterior add_sfh_posterior(fit, ax, colorscheme=colorscheme) File "/mnt/users/ramaiyas/.virtualenvs/multinest/lib/python3.10/site-packages/bagpipes/plotting/plot_sfh_posterior.py", line 81, in add_sfh_posterior ax.set_ylim(0., np.max([ax.get_ylim()[1], 1.1np.max(post[:, 2])])) File "/mnt/users/ramaiyas/.virtualenvs/multinest/lib/python3.10/site-packages/matplotlib/axes/_base.py", line 3898, in set_ylim return self.yaxis._set_lim(bottom, top, emit=emit, auto=auto) File "/mnt/users/ramaiyas/.virtualenvs/multinest/lib/python3.10/site-packages/matplotlib/axis.py", line 1211, in _set_lim v1 = self.axes._validate_converted_limits(v1, self.convert_units) File "/mnt/users/ramaiyas/.virtualenvs/multinest/lib/python3.10/site-packages/matplotlib/axes/_base.py", line 3585, in _validate_converted_limits raise ValueError("Axis limits cannot be NaN or Inf") ValueError: Axis limits cannot be NaN or Inf

and after this, the code stops running i.e. it doesnt continue to fit any subsequent objects in the catalogue. Since fit catalogue iterates through all the objects under the hood, it means one cant add error handling e.g. python 'try and except' as the catalogue fitting is done in one line of code. For now, every time this happens I manually remove the object from the catalogue and rerun it. However, my catalogue has ~1000 objects and so far its unable to run through completion as it stops when encountering a bad object (which I do not know it cant fit until the above error shows and bagpipes stops running) and for larger and larger catalogues it may be unfeasible to have to manually remove objects and restart the code from the first object. I was wondering, would it be possible to have this so if bagpipes encounters an object it cant fit, it ignores it and continues running on the remainder of the catalogue ? or perhaps, any suggestions I may not have thought of to sort this myself in the code.

thank you

ACCarnall commented 5 months ago

Hi there,

This appears to be an error with the standard plotting routines within bagpipes, rather than the running of the fitting code itself. If you turn off the make_plots option I'd expect you should be able to run your catalogue without encountering errors. I can't imagine you really want to look at several output plots for all 1000 objects in your catalogue?

The issue is to do with setting the y limit on the output spectrum plot, which is a part of the code that's a little temperamental - it's hard to design this in a way that catches all possibilities for missing flux values etc as you mentioned. You could also try adding something to bagpipes/plotting/plot_sfh_posterior.py just above line 86 that catches cases for which the maximum y limit it calculates is nan and replaces this with some standard value.

Cheers, Adam

shruti-ramaiya commented 5 months ago

Hi Adam,

Thanks for getting back to me on this and also for letting me know what the issue was. That's helpful to know that fit catalogue should run smoothly without the plotting function enabled - I'll try that and if not, your other suggestion to edit the source file.

Cheers,