search

ebachelet / pyLIMA

GNU General Public License v3.0
31 stars 8 forks source link

New example 5: KeyError: 'photometry' #57

Closed anibal-art closed 1 year ago

anibal-art commented 1 year ago

Hello, I got this error when I run the new_example 5 in the last part of the notebook:

pyLIMA_plots.list_of_fake_telescopes = [] pyLIMA_plots.plot_lightcurves(usbl, fit_2.fit_results['best_model']) pyLIMA_plots.plot_geometry(usbl, fit_2.fit_results['best_model']) plt.show()

KeyError Traceback (most recent call last) Cell In [31], line 2 1 pyLIMA_plots.list_of_fake_telescopes = [] ----> 2 pyLIMA_plots.plot_lightcurves(usbl, fit_2.fit_results['best_model']) 3 pyLIMA_plots.plot_geometry(usbl, fit_2.fit_results['best_model']) 4 plt.show()

File ~/.conda/envs/my_environment/lib/python3.10/site-packages/pyLIMA-0.8.3-py3.10.egg/pyLIMA/outputs/pyLIMA_plots.py:263, in plot_lightcurves(microlensing_model, model_parameters) 259 figure, figure_axes = initialize_light_curves_plot(event_name=microlensing_model.event.name) 261 if len(model_parameters) != len(microlensing_model.model_dictionnary): --> 263 telescopes_fluxes = microlensing_model.find_telescopes_fluxes(model_parameters) 265 modelparameters = np.r[model_parameters,telescopes_fluxes] 267 plot_photometric_models(figure_axes[0], microlensing_model, model_parameters, plot_unit='Mag')

File ~/.conda/envs/my_environment/lib/python3.10/site-packages/pyLIMA-0.8.3-py3.10.egg/pyLIMA/models/ML_model.py:365, in MLmodel.find_telescopes_fluxes(self, fancy_parameters) 361 for telescope in self.event.telescopes: 363 if telescope.lightcurve_flux is not None: --> 365 model = self.compute_the_microlensing_model(telescope, pyLIMA_parameters) 367 fluxes.append(model['f_source']) 368 fluxes.append(model['f_blend'])

File ~/.conda/envs/my_environment/lib/python3.10/site-packages/pyLIMA-0.8.3-py3.10.egg/pyLIMA/models/ML_model.py:277, in MLmodel.compute_the_microlensing_model(self, telescope, pyLIMA_parameters) 267 def compute_the_microlensing_model(self, telescope, pyLIMA_parameters): 268 """ Compute the microlens model according the injected parameters. This is modified by child submodel sublclass, 269 if not the default microlensing model is returned. 270 (...) 274 :rtype: array_like 275 """ --> 277 return self._default_microlensing_model(telescope, pyLIMA_parameters)

File ~/.conda/envs/my_environment/lib/python3.10/site-packages/pyLIMA-0.8.3-py3.10.egg/pyLIMA/models/ML_model.py:300, in MLmodel._default_microlensing_model(self, telescope, pyLIMA_parameters) 296 f_blend = None 298 if telescope.lightcurve_flux is not None: --> 300 magnification = self.model_magnification(telescope, pyLIMA_parameters) 302 f_source, f_blend = self.derive_telescope_flux(telescope, pyLIMA_parameters, magnification) 303 photometric_model = f_source * magnification + f_blend

File ~/.conda/envs/my_environment/lib/python3.10/site-packages/pyLIMA-0.8.3-py3.10.egg/pyLIMA/models/USBL_model.py:57, in USBLmodel.model_magnification(self, telescope, pyLIMA_parameters, return_impact_parameter) 53 if telescope.lightcurve_flux is not None: 55 self.u0_t0_from_uc_tc(pyLIMA_parameters) ---> 57 source_trajectoire = self.source_trajectory(telescope, pyLIMA_parameters, data_type='photometry') 59 separation = source_trajectoire[2] + pyLIMA_parameters.separation 60 magnification_USBL = \ 61 magnification_VBB.magnification_USBL(separation, pyLIMA_parameters.mass_ratio, 62 source_trajectoire[0], source_trajectoire[1], 63 pyLIMA_parameters.rho)

File ~/.conda/envs/my_environment/lib/python3.10/site-packages/pyLIMA-0.8.3-py3.10.egg/pyLIMA/models/ML_model.py:462, in MLmodel.source_trajectory(self, telescope, pyLIMA_parameters, data_type) 458 time = lightcurve['time'].value 460 if 'piEN' in pyLIMA_parameters._fields: --> 462 delta_positions = telescope.deltas_positions['photometry'] 464 if data_type == 'astrometry': 466 astrometry = telescope.astrometry

KeyError: 'photometry'

Also when I set perform_long_fit = True when I try to fit as a binary event:

RemoteTraceback Traceback (most recent call last) RemoteTraceback: """ Traceback (most recent call last): File "/home/anibal-art/.conda/envs/my_environment/lib/python3.10/multiprocessing/pool.py", line 125, in worker result = (True, func(*args, *kwds)) File "/home/anibal-art/.conda/envs/my_environment/lib/python3.10/multiprocessing/pool.py", line 48, in mapstar return list(map(args)) File "/home/anibal-art/.conda/envs/my_environment/lib/python3.10/site-packages/scipy/_lib/_util.py", line 407, in call return self.f(x, *self.args) File "/home/anibal-art/.conda/envs/my_environment/lib/python3.10/site-packages/pyLIMA-0.8.3-py3.10.egg/pyLIMA/fits/DE_fit.py", line 55, in objective_function photometric_likelihood = pyLIMA.fits.objective_functions.all_telescope_photometric_likelihood( File "/home/anibal-art/.conda/envs/my_environment/lib/python3.10/site-packages/pyLIMA-0.8.3-py3.10.egg/pyLIMA/fits/objective_functions.py", line 151, in all_telescope_photometric_likelihood residus, errflux = all_telescope_photometric_residuals(model, pyLIMA_parameters, norm=True, File "/home/anibal-art/.conda/envs/my_environment/lib/python3.10/site-packages/pyLIMA-0.8.3-py3.10.egg/pyLIMA/fits/objective_functions.py", line 115, in all_telescope_photometric_residuals microlensing_model = model.compute_the_microlensing_model(telescope, pyLIMA_parameters) File "/home/anibal-art/.conda/envs/my_environment/lib/python3.10/site-packages/pyLIMA-0.8.3-py3.10.egg/pyLIMA/models/ML_model.py", line 277, in compute_the_microlensing_model return self._default_microlensing_model(telescope, pyLIMA_parameters) File "/home/anibal-art/.conda/envs/my_environment/lib/python3.10/site-packages/pyLIMA-0.8.3-py3.10.egg/pyLIMA/models/ML_model.py", line 300, in _default_microlensing_model magnification = self.model_magnification(telescope, pyLIMA_parameters) File "/home/anibal-art/.conda/envs/my_environment/lib/python3.10/site-packages/pyLIMA-0.8.3-py3.10.egg/pyLIMA/models/USBL_model.py", line 57, in model_magnification source_trajectoire = self.source_trajectory(telescope, pyLIMA_parameters, data_type='photometry') File "/home/anibal-art/.conda/envs/my_environment/lib/python3.10/site-packages/pyLIMA-0.8.3-py3.10.egg/pyLIMA/models/ML_model.py", line 462, in source_trajectory delta_positions = telescope.deltas_positions['photometry'] KeyError: 'photometry' """

The above exception was the direct cause of the following exception:

KeyError Traceback (most recent call last) Cell In [33], line 3 1 ### Fit the model: 2 if perform_long_fit == True: ----> 3 fit_2.fit(computational_pool = pool) 5 # Save it 6 np.save('results_USBL_DE_966.npy', fit_2.fit_results['DE_population'])

File ~/.conda/envs/my_environment/lib/python3.10/site-packages/pyLIMA-0.8.3-py3.10.egg/pyLIMA/fits/DE_fit.py:119, in DEfit.fit(self, initial_population, computational_pool) 115 init = initial_population 117 bounds = [self.fit_parameters[key][1] for key in self.fit_parameters.keys()] --> 119 differential_evolution_estimation = scipy.optimize.differential_evolution(self.objective_function, 120 bounds=bounds, 121 mutation=(0.5, 1.5), popsize=int(self.DE_population_size), 122 maxiter=self.max_iteration, tol=0.00, 123 atol=1.0, strategy='rand1bin', 124 recombination=0.5, polish=False, init=init, 125 disp=self.display_progress, workers=worker) 127 print('DE converge to objective function : f(x) = ', str(differential_evolution_estimation['fun'])) 128 print('DE converge to parameters : = ', differential_evolution_estimation['x'].astype(str))

File ~/.conda/envs/my_environment/lib/python3.10/site-packages/scipy/optimize/_differentialevolution.py:329, in differential_evolution(func, bounds, args, strategy, maxiter, popsize, tol, mutation, recombination, seed, callback, disp, polish, init, atol, updating, workers, constraints, x0) 314 # using a context manager means that any created Pool objects are 315 # cleared up. 316 with DifferentialEvolutionSolver(func, bounds, args=args, 317 strategy=strategy, 318 maxiter=maxiter, (...) 327 constraints=constraints, 328 x0=x0) as solver: --> 329 ret = solver.solve() 331 return ret

File ~/.conda/envs/my_environment/lib/python3.10/site-packages/scipy/optimize/_differentialevolution.py:843, in DifferentialEvolutionSolver.solve(self) 838 self.feasible, self.constraint_violation = ( 839 self._calculate_population_feasibilities(self.population)) 841 # only work out population energies for feasible solutions 842 self.population_energies[self.feasible] = ( --> 843 self._calculate_population_energies( 844 self.population[self.feasible])) 846 self._promote_lowest_energy() 848 # do the optimization.

File ~/.conda/envs/my_environment/lib/python3.10/site-packages/scipy/optimize/_differentialevolution.py:968, in DifferentialEvolutionSolver._calculate_population_energies(self, population) 966 parameters_pop = self._scale_parameters(population) 967 try: --> 968 calc_energies = list(self._mapwrapper(self.func, 969 parameters_pop[0:nfevs])) 970 calc_energies = np.squeeze(calc_energies) 971 except (TypeError, ValueError) as e: 972 # wrong number of arguments for _mapwrapper 973 # or wrong length returned from the mapper

File ~/.conda/envs/my_environment/lib/python3.10/site-packages/scipy/_lib/_util.py:477, in MapWrapper.call(self, func, iterable) 474 def call(self, func, iterable): 475 # only accept one iterable because that's all Pool.map accepts 476 try: --> 477 return self._mapfunc(func, iterable) 478 except TypeError as e: 479 # wrong number of arguments 480 raise TypeError("The map-like callable must be of the" 481 " form f(func, iterable)") from e

File ~/.conda/envs/my_environment/lib/python3.10/multiprocessing/pool.py:367, in Pool.map(self, func, iterable, chunksize) 362 def map(self, func, iterable, chunksize=None): 363 ''' 364 Apply func to each element in iterable, collecting the results 365 in a list that is returned. 366 ''' --> 367 return self._map_async(func, iterable, mapstar, chunksize).get()

File ~/.conda/envs/my_environment/lib/python3.10/multiprocessing/pool.py:774, in ApplyResult.get(self, timeout) 772 return self._value 773 else: --> 774 raise self._value

KeyError: 'photometry'

Thanks for the help!

ytsapras commented 1 year ago

Hi Anibal,

I am assuming you are using the Rebranding branch. Could you please print out fit_2.fit_results['best_model'] so we can see what the array fed into the code looks like?

One quick thing to try: afterperform_long_fit = False, set fit_2.fit_results['best_model'] = [2457205.21, 0.0109583755, 1.78218726, -2.89415218, 0.0475121003, -3.79996021, 2.25499875, 0.0227712230, -0.227192561] and try to plot this to see if it works.

ytsapras commented 1 year ago

Note that the code has been tested and works with python 3.9 and you seem to be using python 3.10, which comes with some differences. Of course we want to make the code compatible with later versions of Python so we need to understand where these problems are coming from.

anibal-art commented 1 year ago

Hello Yiannis. Yes, I'm using the Rebranding branch. I think that you request about the outputs from fit_2.fit_results['best_model'] is with perform_long_fit = False, right? here it is

array([ 2.45720521e+06, 1.09583755e-02, 1.78218726e+00, -2.89415218e+00, 4.75121003e-02, -3.79996021e+00, 2.25499875e+00, 2.27712230e-02, -2.27192561e-01])

The quick thing give me the same error that I mentioned:

KeyError Traceback (most recent call last) Cell In [31], line 2 1 pyLIMA_plots.list_of_fake_telescopes = [] ----> 2 pyLIMA_plots.plot_lightcurves(usbl, fit_2.fit_results['best_model']) 3 pyLIMA_plots.plot_geometry(usbl, fit_2.fit_results['best_model']) 4 plt.show()

File ~/.conda/envs/my_environment/lib/python3.10/site-packages/pyLIMA-0.8.3-py3.10.egg/pyLIMA/outputs/pyLIMA_plots.py:263, in plot_lightcurves(microlensing_model, model_parameters) 259 figure, figure_axes = initialize_light_curves_plot(event_name=microlensing_model.event.name) 261 if len(model_parameters) != len(microlensing_model.model_dictionnary): --> 263 telescopes_fluxes = microlensing_model.find_telescopes_fluxes(model_parameters) 265 modelparameters = np.r[model_parameters,telescopes_fluxes] 267 plot_photometric_models(figure_axes[0], microlensing_model, model_parameters, plot_unit='Mag')

File ~/.conda/envs/my_environment/lib/python3.10/site-packages/pyLIMA-0.8.3-py3.10.egg/pyLIMA/models/ML_model.py:365, in MLmodel.find_telescopes_fluxes(self, fancy_parameters) 361 for telescope in self.event.telescopes: 363 if telescope.lightcurve_flux is not None: --> 365 model = self.compute_the_microlensing_model(telescope, pyLIMA_parameters) 367 fluxes.append(model['f_source']) 368 fluxes.append(model['f_blend'])

File ~/.conda/envs/my_environment/lib/python3.10/site-packages/pyLIMA-0.8.3-py3.10.egg/pyLIMA/models/ML_model.py:277, in MLmodel.compute_the_microlensing_model(self, telescope, pyLIMA_parameters) 267 def compute_the_microlensing_model(self, telescope, pyLIMA_parameters): 268 """ Compute the microlens model according the injected parameters. This is modified by child submodel sublclass, 269 if not the default microlensing model is returned. 270 (...) 274 :rtype: array_like 275 """ --> 277 return self._default_microlensing_model(telescope, pyLIMA_parameters)

File ~/.conda/envs/my_environment/lib/python3.10/site-packages/pyLIMA-0.8.3-py3.10.egg/pyLIMA/models/ML_model.py:300, in MLmodel._default_microlensing_model(self, telescope, pyLIMA_parameters) 296 f_blend = None 298 if telescope.lightcurve_flux is not None: --> 300 magnification = self.model_magnification(telescope, pyLIMA_parameters) 302 f_source, f_blend = self.derive_telescope_flux(telescope, pyLIMA_parameters, magnification) 303 photometric_model = f_source * magnification + f_blend

File ~/.conda/envs/my_environment/lib/python3.10/site-packages/pyLIMA-0.8.3-py3.10.egg/pyLIMA/models/USBL_model.py:57, in USBLmodel.model_magnification(self, telescope, pyLIMA_parameters, return_impact_parameter) 53 if telescope.lightcurve_flux is not None: 55 self.u0_t0_from_uc_tc(pyLIMA_parameters) ---> 57 source_trajectoire = self.source_trajectory(telescope, pyLIMA_parameters, data_type='photometry') 59 separation = source_trajectoire[2] + pyLIMA_parameters.separation 60 magnification_USBL = \ 61 magnification_VBB.magnification_USBL(separation, pyLIMA_parameters.mass_ratio, 62 source_trajectoire[0], source_trajectoire[1], 63 pyLIMA_parameters.rho)

File ~/.conda/envs/my_environment/lib/python3.10/site-packages/pyLIMA-0.8.3-py3.10.egg/pyLIMA/models/ML_model.py:462, in MLmodel.source_trajectory(self, telescope, pyLIMA_parameters, data_type) 458 time = lightcurve['time'].value 460 if 'piEN' in pyLIMA_parameters._fields: --> 462 delta_positions = telescope.deltas_positions['photometry'] 464 if data_type == 'astrometry': 466 astrometry = telescope.astrometry

KeyError: 'photometry'

About the Python version , I run the same example on another pc with Python 3.10.9 and run with no problems. But with this that have Python 3.10.6 I have this issue. I don't know if it useful for you but I can print also the other python modules that I have in the environment that have the issue

pip list

Package Version

absl-py 1.3.0 adjustText 0.7.3 alembic 1.8.1 anyio 3.6.2 argon2-cffi 21.3.0 argon2-cffi-bindings 21.2.0 astropy 5.1 asttokens 2.0.8 astunparse 1.6.3 attrs 22.1.0 autopage 0.5.1 Babel 2.10.3 backcall 0.2.0 backports.functools-lru-cache 1.6.4 beautifulsoup4 4.11.1 bidict 0.22.0 bioinfokit 2.1.0 bleach 5.0.1 bokeh 3.0.2 Boruta 0.3 BorutaShap 1.0.16 cachetools 5.2.0 certifi 2022.9.24 cffi 1.15.1 charset-normalizer 2.1.1 cliff 4.0.0 cloudpickle 2.2.0 cmaes 0.8.2 cmd2 2.4.2 colorama 0.4.5 colorlog 6.7.0 contourpy 1.0.5 cycler 0.11.0 Cython 0.29.32 debugpy 1.6.3 decorator 5.1.1 defusedxml 0.7.1 emcee 3.1.3 entrypoints 0.4 executing 1.1.0 fastjsonschema 2.16.2 flatbuffers 22.10.26 fonttools 4.37.3 gast 0.4.0 gensim 4.2.0 google-auth 2.14.1 google-auth-oauthlib 0.4.6 google-pasta 0.2.0 greenlet 1.1.3 grpcio 1.50.0 h5py 3.7.0 healpy 1.16.1 helpy 1.0.1 idna 3.4 igraph 0.10.2 importlib-metadata 4.12.0 importlib-resources 5.9.0 ipykernel 6.16.0 ipympl 0.9.2 ipython 8.5.0 ipython-genutils 0.2.0 ipywidgets 8.0.2 jedi 0.18.1 Jinja2 3.1.2 joblib 1.2.0 json5 0.9.10 jsonschema 4.16.0 jupyter_client 7.3.5 jupyter_core 4.11.1 jupyter-server 1.21.0 jupyterlab 3.4.8 jupyterlab-pygments 0.2.2 jupyterlab_server 2.16.1 jupyterlab-widgets 3.0.3 keras 2.10.0 Keras-Preprocessing 1.1.2 kiwisolver 1.4.4 libclang 14.0.6 llvmlite 0.39.1 Mako 1.2.3 Markdown 3.4.1 MarkupSafe 2.1.1 matplotlib 3.6.0 matplotlib-inline 0.1.6 matplotlib-venn 0.11.7 MicroLIA 1.9.7 missingpy 0.2.0 mistune 2.0.4 Naked 0.1.31 nbclassic 0.4.5 nbclient 0.7.0 nbconvert 7.2.2 nbformat 5.6.1 nest-asyncio 1.5.5 networkx 2.8.7 node2vec 0.4.6 notebook 6.5.1 notebook_shim 0.2.0 numba 0.56.2 numpy 1.23.3 oauthlib 3.2.2 opt-einsum 3.3.0 optuna 3.0.2 packaging 21.3 pandas 1.5.0 pandocfilters 1.5.0 parso 0.8.3 patsy 0.5.2 pbr 5.10.0 PeakUtils 1.3.4 pexpect 4.8.0 pickleshare 0.7.5 Pillow 9.2.0 pip 22.2.2 pkgutil_resolve_name 1.3.10 plfit 1.0.3 prettytable 3.4.1 progress 1.6 prometheus-client 0.15.0 prompt-toolkit 3.0.31 protobuf 3.19.6 psutil 5.9.2 ptyprocess 0.7.0 pure-eval 0.2.2 pyaml 21.10.1 pyasn1 0.4.8 pyasn1-modules 0.2.8 PyAstronomy 0.18.0 pybind11 2.10.1 pycparser 2.21 pyerfa 2.0.0.1 Pygments 2.13.0 pyLIMA 0.8.3 pyparsing 3.0.9 pyperclip 1.8.2 pyrsistent 0.18.1 python-dateutil 2.8.2 python-igraph 0.10.2 python-louvain 0.16 pytz 2022.2.1 PyYAML 6.0 pyzmq 24.0.1 quantities 0.13.0 requests 2.28.1 requests-oauthlib 1.3.1 rsa 4.9 scikit-learn 1.1.2 scikit-optimize 0.9.0 scikit-plot 0.3.7 scipy 1.8.1 seaborn 0.12.0 Send2Trash 1.8.0 setuptools 59.8.0 shap 0.41.0 six 1.16.0 slicer 0.0.7 smart-open 6.2.0 sniffio 1.3.0 soupsieve 2.3.2.post1 SQLAlchemy 1.4.41 stack-data 0.5.1 statsmodels 0.13.2 stevedore 4.0.0 tabulate 0.9.0 tensorboard 2.10.1 tensorboard-data-server 0.6.1 tensorboard-plugin-wit 1.8.1 tensorflow 2.10.0 tensorflow-estimator 2.10.0 tensorflow-io-gcs-filesystem 0.27.0 termcolor 2.1.0 terminado 0.16.0 texttable 1.6.5 textwrap3 0.9.2 threadpoolctl 3.1.0 tinycss2 1.2.1 tomli 2.0.1 tornado 6.2 tqdm 4.64.1 traitlets 5.4.0 typing_extensions 4.3.0 urllib3 1.26.12 VBBinaryLensing 3.4.0 wcwidth 0.2.5 webencodings 0.5.1 websocket-client 1.4.1 Werkzeug 2.2.2 wheel 0.37.1 widgetsnbextension 4.0.3 wordcloud 1.8.2.2 wrapt 1.14.1 xgboost 1.6.2 xyzservices 2022.9.0 zipp 3.8.1

ebachelet commented 1 year ago

I believe this is fixed in 1,9