timothydmorton / VESPA

Validation of Exoplanet Signals using a Probabilistic Algorithm--- calculating false positive probabilities for transit signals
MIT License
36 stars 20 forks source link

KeyError: 'No object named /obs/df in the file' #29

Open reneheller opened 5 years ago

reneheller commented 5 years ago

I have a folder that contains three files: "star.ini", "fpp.ini" and the phase-folded light curve. I then run

heller$ starfit --all .

and MultiNest (v3.10) seems to be running just fine:

====> Sampling finished. Exiting MultiNest (...) triple starfit successful for . in 7.1 minutes. <====

Then I run

heller$ calcfpp

which ends with a RuntimeError:

====> /usr/local/lib/python3.7/site-packages/numpy/lib/npyio.py:2242: VisibleDeprecationWarning: Reading unicode strings without specifying the encoding argument is deprecated. Set the encoding, use None for the system default. output = genfromtxt(fname, **kwargs) WARNING:root:sklearn not available WARNING:root:progressbar not imported WARNING:root:Cadence not provided in fpp.ini; defaulting to Kepler cadence. Cadence not provided in fpp.ini; defaulting to Kepler cadence. WARNING:root:If this is not a Kepler target, please set cadence (in days). If this is not a Kepler target, please set cadence (in days). WARNING:root:Transit band not provided in fpp.ini; defaulting to "Kepler." Transit band not provided in fpp.ini; defaulting to "Kepler." WARNING:root:If this is not a Kepler target, please specify filter. If this is not a Kepler target, please specify filter. 'No object named /obs/df in the file' ERROR:root:FPP calculation failed for .. Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/vespa/fpp.py", line 236, in from_ini single_starmodel = StarModel.load_hdf(single_starmodel_file) File "/usr/local/lib/python3.7/site-packages/isochrones/starmodel.py", line 1256, in load_hdf obs = ObservationTree.load_hdf(filename, path+'/obs', ic=ic) File "/usr/local/lib/python3.7/site-packages/isochrones/observation.py", line 869, in load_hdf samples = store[path+'/df'] File "/usr/local/lib/python3.7/site-packages/pandas/io/pytables.py", line 504, in getitem return self.get(key) File "/usr/local/lib/python3.7/site-packages/pandas/io/pytables.py", line 694, in get raise KeyError('No object named {key} in the file'.format(key=key)) KeyError: 'No object named /obs/df in the file'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/local/bin/calcfpp", line 151, in n=args.n) File "/usr/local/lib/python3.7/site-packages/vespa/fpp.py", line 242, in from_ini 'Please run starfit --all {}.'.format(folder)) RuntimeError: Cannot load StarModels. Please run starfit --all .. FPP calculation failed for .. Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/vespa/fpp.py", line 236, in from_ini single_starmodel = StarModel.load_hdf(single_starmodel_file) File "/usr/local/lib/python3.7/site-packages/isochrones/starmodel.py", line 1256, in load_hdf obs = ObservationTree.load_hdf(filename, path+'/obs', ic=ic) File "/usr/local/lib/python3.7/site-packages/isochrones/observation.py", line 869, in load_hdf samples = store[path+'/df'] File "/usr/local/lib/python3.7/site-packages/pandas/io/pytables.py", line 504, in getitem return self.get(key) File "/usr/local/lib/python3.7/site-packages/pandas/io/pytables.py", line 694, in get raise KeyError('No object named {key} in the file'.format(key=key)) KeyError: 'No object named /obs/df in the file'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/local/bin/calcfpp", line 151, in n=args.n) File "/usr/local/lib/python3.7/site-packages/vespa/fpp.py", line 242, in from_ini 'Please run starfit --all {}.'.format(folder)) RuntimeError: Cannot load StarModels. Please run starfit --all .. <====

So now I'm supposed to tun starfit --all . again although I've just done that before. Anyways, if I do that

heller$ starfit --all .

now gives an error message

====> 1 of 1: . Fitting . single starfit... mist_starmodel_single.h5 exists. Use -o to overwrite. single starfit successful for . in 0.4 minutes. binary starfit... mist_starmodel_binary.h5 exists. Use -o to overwrite. binary starfit calculation failed for .. Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/isochrones/starfit.py", line 127, in starfit fig1,fig2 = mod.corner_plots(corner_base) File "/usr/local/lib/python3.7/site-packages/isochrones/starmodel.py", line 1070, in corner_plots fig1, fig2 = self.corner_physical(kwargs), self.corner_observed(kwargs) File "/usr/local/lib/python3.7/site-packages/isochrones/starmodel.py", line 1696, in corner_physical return self.corner_derived(self.physical_quantities) File "/usr/local/lib/python3.7/site-packages/isochrones/starmodel.py", line 1691, in corner_derived fig = corner.corner(self.derived_samples[cols], labels=cols, kwargs) File "/usr/local/lib/python3.7/site-packages/pandas/core/frame.py", line 2934, in getitem raise_missing=True) File "/usr/local/lib/python3.7/site-packages/pandas/core/indexing.py", line 1354, in _convert_to_indexer return self._get_listlike_indexer(obj, axis, kwargs)[1] File "/usr/local/lib/python3.7/site-packages/pandas/core/indexing.py", line 1161, in _get_listlike_indexer raise_missing=raise_missing) File "/usr/local/lib/python3.7/site-packages/pandas/core/indexing.py", line 1252, in _validate_read_indexer raise KeyError("{} not in index".format(not_found)) KeyError: "['Teff', 'mass', 'radius', 'logg'] not in index" triple starfit... mist_starmodel_triple.h5 exists. Use -o to overwrite. triple starfit calculation failed for .. Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/isochrones/starfit.py", line 127, in starfit fig1,fig2 = mod.corner_plots(corner_base) File "/usr/local/lib/python3.7/site-packages/isochrones/starmodel.py", line 1070, in corner_plots fig1, fig2 = self.corner_physical(kwargs), self.corner_observed(kwargs) File "/usr/local/lib/python3.7/site-packages/isochrones/starmodel.py", line 1696, in corner_physical return self.corner_derived(self.physical_quantities) File "/usr/local/lib/python3.7/site-packages/isochrones/starmodel.py", line 1691, in corner_derived fig = corner.corner(self.derived_samples[cols], labels=cols, kwargs) File "/usr/local/lib/python3.7/site-packages/pandas/core/frame.py", line 2934, in getitem raise_missing=True) File "/usr/local/lib/python3.7/site-packages/pandas/core/indexing.py", line 1354, in _convert_to_indexer return self._get_listlike_indexer(obj, axis, kwargs)[1] File "/usr/local/lib/python3.7/site-packages/pandas/core/indexing.py", line 1161, in _get_listlike_indexer raise_missing=raise_missing) File "/usr/local/lib/python3.7/site-packages/pandas/core/indexing.py", line 1252, in _validate_read_indexer raise KeyError("{} not in index".format(not_found)) KeyError: "['Teff', 'mass', 'radius', 'logg'] not in index" <===

I know that the problem can't be in the files that I'm providing because I've successfully run vespa on them on another machine.

Any help would be appreciated.

timothydmorton commented 5 years ago

What versions of vespa and isochrones are you using? I have recently updated isochrones and vespa is not (yet) compatible with isochrones >=2.0.

On Sat, May 25, 2019 at 6:08 PM reneheller notifications@github.com wrote:

I have a folder that contains three files: "star.ini", "fpp.ini" and the phase-folded light curve. I then run

heller$ starfit --all .

and MultiNest (v3.10) seems to be running just fine:

====> Sampling finished. Exiting MultiNest (...) triple starfit successful for . in 7.1 minutes. <====

Then I run

heller$ calcfpp

which ends with a RuntimeError:

====> /usr/local/lib/python3.7/site-packages/numpy/lib/npyio.py:2242: VisibleDeprecationWarning: Reading unicode strings without specifying the encoding argument is deprecated. Set the encoding, use None for the system default. output = genfromtxt(fname, *kwargs) WARNING:root:sklearn not available WARNING:root:progressbar not imported WARNING:root:Cadence not provided in fpp.ini; defaulting to Kepler cadence. Cadence not provided in fpp.ini; defaulting to Kepler cadence. WARNING:root:If this is not a Kepler target, please set cadence (in days). If this is not a Kepler target, please set cadence (in days). WARNING:root:Transit band not provided in fpp.ini; defaulting to "Kepler." Transit band not provided in fpp.ini; defaulting to "Kepler." WARNING:root:If this is not a Kepler target, please specify filter. If this is not a Kepler target, please specify filter. 'No object named /obs/df in the file' ERROR:root:FPP calculation failed for .. Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/vespa/fpp.py", line 236, in from_ini single_starmodel = StarModel.load_hdf(single_starmodel_file) File "/usr/local/lib/python3.7/site-packages/isochrones/starmodel.py", line 1256, in load_hdf obs = ObservationTree.load_hdf(filename, path+'/obs', ic=ic) File "/usr/local/lib/python3.7/site-packages/isochrones/observation.py", line 869, in load_hdf samples = store[path+'/df'] File "/usr/local/lib/python3.7/site-packages/pandas/io/pytables.py", line 504, in getitem* return self.get(key) File "/usr/local/lib/python3.7/site-packages/pandas/io/pytables.py", line 694, in get raise KeyError('No object named {key} in the file'.format(key=key)) KeyError: 'No object named /obs/df in the file'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/local/bin/calcfpp", line 151, in n=args.n) File "/usr/local/lib/python3.7/site-packages/vespa/fpp.py", line 242, in from_ini 'Please run starfit --all {}.'.format(folder)) RuntimeError: Cannot load StarModels. Please run starfit --all .. FPP calculation failed for .. Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/vespa/fpp.py", line 236, in from_ini single_starmodel = StarModel.load_hdf(single_starmodel_file) File "/usr/local/lib/python3.7/site-packages/isochrones/starmodel.py", line 1256, in load_hdf obs = ObservationTree.load_hdf(filename, path+'/obs', ic=ic) File "/usr/local/lib/python3.7/site-packages/isochrones/observation.py", line 869, in load_hdf samples = store[path+'/df'] File "/usr/local/lib/python3.7/site-packages/pandas/io/pytables.py", line 504, in getitem return self.get(key) File "/usr/local/lib/python3.7/site-packages/pandas/io/pytables.py", line 694, in get raise KeyError('No object named {key} in the file'.format(key=key)) KeyError: 'No object named /obs/df in the file'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/local/bin/calcfpp", line 151, in n=args.n) File "/usr/local/lib/python3.7/site-packages/vespa/fpp.py", line 242, in from_ini 'Please run starfit --all {}.'.format(folder)) RuntimeError: Cannot load StarModels. Please run starfit --all .. <====

So now I'm supposed to tun starfit --all . again although I've just done that before. Anyways, if I do that

heller$ starfit --all .

now gives an error message

====> 1 of 1: . Fitting . single starfit... mist_starmodel_single.h5 exists. Use -o to overwrite. single starfit successful for . in 0.4 minutes. binary starfit... mist_starmodel_binary.h5 exists. Use -o to overwrite. binary starfit calculation failed for .. Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/isochrones/starfit.py", line 127, in starfit fig1,fig2 = mod.corner_plots(corner_base) File "/usr/local/lib/python3.7/site-packages/isochrones/starmodel.py", line 1070, in corner_plots fig1, fig2 = self.corner_physical(kwargs), self.corner_observed(kwargs) File "/usr/local/lib/python3.7/site-packages/isochrones/starmodel.py", line 1696, in corner_physical return self.corner_derived(self.physical_quantities) File "/usr/local/lib/python3.7/site-packages/isochrones/starmodel.py", line 1691, in corner_derived fig = corner.corner(self.derived_samples[cols], labels=cols, kwargs) File "/usr/local/lib/python3.7/site-packages/pandas/core/frame.py", line 2934, in getitem raise_missing=True) File "/usr/local/lib/python3.7/site-packages/pandas/core/indexing.py", line 1354, in _convert_to_indexer return self._get_listlike_indexer(obj, axis, kwargs)[1] File "/usr/local/lib/python3.7/site-packages/pandas/core/indexing.py", line 1161, in _get_listlike_indexer raise_missing=raise_missing) File "/usr/local/lib/python3.7/site-packages/pandas/core/indexing.py", line 1252, in _validate_read_indexer raise KeyError("{} not in index".format(not_found)) KeyError: "['Teff', 'mass', 'radius', 'logg'] not in index" triple starfit... mist_starmodel_triple.h5 exists. Use -o to overwrite. triple starfit calculation failed for .. Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/isochrones/starfit.py", line 127, in starfit fig1,fig2 = mod.corner_plots(corner_base) File "/usr/local/lib/python3.7/site-packages/isochrones/starmodel.py", line 1070, in corner_plots fig1, fig2 = self.corner_physical(kwargs), self.corner_observed(kwargs) File "/usr/local/lib/python3.7/site-packages/isochrones/starmodel.py", line 1696, in corner_physical return self.corner_derived(self.physical_quantities) File "/usr/local/lib/python3.7/site-packages/isochrones/starmodel.py", line 1691, in corner_derived fig = corner.corner(self.derived_samples[cols], labels=cols, kwargs) File "/usr/local/lib/python3.7/site-packages/pandas/core/frame.py", line 2934, in getitem raise_missing=True) File "/usr/local/lib/python3.7/site-packages/pandas/core/indexing.py", line 1354, in _convert_to_indexer return self._get_listlike_indexer(obj, axis, kwargs)[1] File "/usr/local/lib/python3.7/site-packages/pandas/core/indexing.py", line 1161, in _get_listlike_indexer raise_missing=raise_missing) File "/usr/local/lib/python3.7/site-packages/pandas/core/indexing.py", line 1252, in _validate_read_indexer raise KeyError("{} not in index".format(not_found)) KeyError: "['Teff', 'mass', 'radius', 'logg'] not in index" <===

I know that the problem can't be in the files that I'm providing because I've successfully run vespa on them on another machine.

Any help would be appreciated.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/timothydmorton/VESPA/issues/29?email_source=notifications&email_token=AAOOXWY5BRJ5QL24LLQI6YDPXG2FXA5CNFSM4HPVPCZKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4GV3VR7Q, or mute the thread https://github.com/notifications/unsubscribe-auth/AAOOXW4SXI4QJCB5CFXNJXTPXG2FXANCNFSM4HPVPCZA .

-- about.me/tim.morton

reneheller commented 5 years ago

Hi Morten! I installed everything yesterday, and

pip3 list

gives

===> isochrones 2.0.1 VESPA 0.6 <===

So following your comment, it seems to me I'm currently using incompatible versions of vespa and isochrones. Do you see any workaround how I can get vespa running now?

timothydmorton commented 5 years ago

Install isochrones v1.2.2: pip install isochrones==1.2.2 should work. (See the most recent vespa release notes: https://github.com/timothydmorton/VESPA/releases)

You will have to define a new $ISOCHRONES directory (or wipe your existing ~/.isochrones directory). Hope this works for you!

On Sun, May 26, 2019 at 6:59 AM reneheller notifications@github.com wrote:

Hi Morten! I installed everything yesterday, and

pip3 list

gives

===> isochrones 2.0.1 VESPA 0.6 <===

So following your comment, it seems to me I'm currently using incompatible versions of vespa and isochrones. Do you see any workaround how I can get vespa running now?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/timothydmorton/VESPA/issues/29?email_source=notifications&email_token=AAOOXW2EM733JVOYC5A4JZ3PXJUSTA5CNFSM4HPVPCZKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWIDFPQ#issuecomment-495989438, or mute the thread https://github.com/notifications/unsubscribe-auth/AAOOXWYC223YQ7HUQQBXTYTPXJUSTANCNFSM4HPVPCZA .

-- about.me/tim.morton

reneheller commented 5 years ago

I now did things as per your suggestion. I first did pip3 uninstall isochrones then deleted ~/.isochrones, and finally installed isochrones v1.2.2 via pip install isochrones==1.2.2.

Then I deleted all output files that I had previously generated for my star.ini, fpp.ini and CSV light curve files and started from scratch using starfit --all . This seems to have gone smoothly

===> (...) Total Likelihood Evaluations: 988862 Sampling finished. Exiting MultiNest triple starfit successful for . in 13.5 minutes. <===

But when I run calfpp, I receive a new error message that I'm not able to handle:

heller$ calcfpp

===> /usr/local/lib/python3.7/site-packages/numpy/lib/npyio.py:2242: VisibleDeprecationWarning: Reading unicode strings without specifying the encoding argument is deprecated. Set the encoding, use None for the system default. output = genfromtxt(fname, *kwargs) WARNING:root:sklearn not available WARNING:root:progressbar not imported WARNING:root:Cadence not provided in fpp.ini; defaulting to Kepler cadence. Cadence not provided in fpp.ini; defaulting to Kepler cadence. WARNING:root:If this is not a Kepler target, please set cadence (in days). If this is not a Kepler target, please set cadence (in days). WARNING:root:Transit band not provided in fpp.ini; defaulting to "Kepler." Transit band not provided in fpp.ini; defaulting to "Kepler." WARNING:root:If this is not a Kepler target, please specify filter. If this is not a Kepler target, please specify filter. INFO:root:Fitting transitsignal with MCMC... Fitting transitsignal with MCMC... INFO:root:Fitting data to trapezoid shape with MCMC for EPIC201497682_3.... Fitting data to trapezoid shape with MCMC for EPIC201497682_3.... ERROR:root:FPP calculation failed for .. Traceback (most recent call last): File "/usr/local/bin/calcfpp", line 151, in n=args.n) File "/usr/local/lib/python3.7/site-packages/vespa/fpp.py", line 260, in from_ini trsig.MCMC() File "/usr/local/lib/python3.7/site-packages/vespa/transitsignal.py", line 300, in MCMC self.Ts_acor = integrated_time(Ts) File "/usr/local/lib/python3.7/site-packages/emcee/autocorr.py", line 88, in integrated_time f = function(x, axis=axis, fast=fast) File "/usr/local/lib/python3.7/site-packages/emcee/autocorr.py", line 41, in function acf = np.fft.ifft(f np.conjugate(f), axis=axis)[m].real FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use arr[tuple(seq)] instead of arr[seq]. In the future this will be interpreted as an array index, arr[np.array(seq)], which will result either in an error or a different result. FPP calculation failed for .. Traceback (most recent call last): File "/usr/local/bin/calcfpp", line 151, in n=args.n) File "/usr/local/lib/python3.7/site-packages/vespa/fpp.py", line 260, in from_ini trsig.MCMC() File "/usr/local/lib/python3.7/site-packages/vespa/transitsignal.py", line 300, in MCMC self.Ts_acor = integrated_time(Ts) File "/usr/local/lib/python3.7/site-packages/emcee/autocorr.py", line 88, in integrated_time f = function(x, axis=axis, fast=fast) File "/usr/local/lib/python3.7/site-packages/emcee/autocorr.py", line 41, in function acf = np.fft.ifft(f * np.conjugate(f), axis=axis)[m].real FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use arr[tuple(seq)] instead of arr[seq]. In the future this will be interpreted as an array index, arr[np.array(seq)], which will result either in an error or a different result. <===

Any help will be very much appreciated.

timothydmorton commented 5 years ago

Have you tried updating your version of emcee? I believe I successfully ran with 3.0rc2. Let me know if that works.

On Tue, May 28, 2019 at 5:25 AM reneheller notifications@github.com wrote:

I now did things as per your suggestion. I first did pip3 uninstall isochrones then deleted ~/.isochrones, and finally installed isochrones v1.2.2 via pip install isochrones==1.2.2.

Then I deleted all output files that I had previously generated for my star.ini, fpp.ini and CSV light curve files and started from scratch using starfit --all . This seems to have gone smoothly

===> (...) Total Likelihood Evaluations: 988862 Sampling finished. Exiting MultiNest triple starfit successful for . in 13.5 minutes. <===

But when I run calfpp, I receive a new error message that I'm not able to handle:

heller$ calcfpp

===> /usr/local/lib/python3.7/site-packages/numpy/lib/npyio.py:2242: VisibleDeprecationWarning: Reading unicode strings without specifying the encoding argument is deprecated. Set the encoding, use None for the system default. output = genfromtxt(fname, *kwargs) WARNING:root:sklearn not available WARNING:root:progressbar not imported WARNING:root:Cadence not provided in fpp.ini; defaulting to Kepler cadence. Cadence not provided in fpp.ini; defaulting to Kepler cadence. WARNING:root:If this is not a Kepler target, please set cadence (in days). If this is not a Kepler target, please set cadence (in days). WARNING:root:Transit band not provided in fpp.ini; defaulting to "Kepler." Transit band not provided in fpp.ini; defaulting to "Kepler." WARNING:root:If this is not a Kepler target, please specify filter. If this is not a Kepler target, please specify filter. INFO:root:Fitting transitsignal with MCMC... Fitting transitsignal with MCMC... INFO:root:Fitting data to trapezoid shape with MCMC for EPIC201497682_3.... Fitting data to trapezoid shape with MCMC for EPIC201497682_3.... ERROR:root:FPP calculation failed for .. Traceback (most recent call last): File "/usr/local/bin/calcfpp", line 151, in n=args.n) File "/usr/local/lib/python3.7/site-packages/vespa/fpp.py", line 260, in from_ini trsig.MCMC() File "/usr/local/lib/python3.7/site-packages/vespa/transitsignal.py", line 300, in MCMC self.Ts_acor = integrated_time(Ts) File "/usr/local/lib/python3.7/site-packages/emcee/autocorr.py", line 88, in integrated_time f = function(x, axis=axis, fast=fast) File "/usr/local/lib/python3.7/site-packages/emcee/autocorr.py", line 41, in function acf = np.fft.ifft(f np.conjugate(f), axis=axis)[m].real FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use arr[tuple(seq)] instead of arr[seq]. In the future this will be interpreted as an array index, arr[np.array(seq)], which will result either in an error or a different result. FPP calculation failed for .. Traceback (most recent call last): File "/usr/local/bin/calcfpp", line 151, in n=args.n) File "/usr/local/lib/python3.7/site-packages/vespa/fpp.py", line 260, in from_ini trsig.MCMC() File "/usr/local/lib/python3.7/site-packages/vespa/transitsignal.py", line 300, in MCMC self.Ts_acor = integrated_time(Ts) File "/usr/local/lib/python3.7/site-packages/emcee/autocorr.py", line 88, in integrated_time f = function(x, axis=axis, fast=fast) File "/usr/local/lib/python3.7/site-packages/emcee/autocorr.py", line 41, in function acf = np.fft.ifft(f * np.conjugate(f), axis=axis)[m].real FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use arr[tuple(seq)] instead of arr[seq]. In the future this will be interpreted as an array index, arr[np.array(seq)], which will result either in an error or a different result. <===

Any help will be very much appreciated.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/timothydmorton/VESPA/issues/29?email_source=notifications&email_token=AAOOXW3PR4N5JUW7SIPQR7DPXT27RA5CNFSM4HPVPCZKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWLQU7I#issuecomment-496437885, or mute the thread https://github.com/notifications/unsubscribe-auth/AAOOXW5H55OKQN65ZG4BBY3PXT27RANCNFSM4HPVPCZA .

-- about.me/tim.morton

reneheller commented 5 years ago

I have emcee 2.2.1 and emcee3 3.0.0.dev0

heller$ pip3 list

===> emcee 2.2.1
emcee3 3.0.0.dev0 <===

I uninstalled emcee and reinstalled emcee, making sure I have the latest version. But that's still/again emcee 2.2.1.

How can I tell calcfpp to use emcee3 instead of emcee?

timothydmorton commented 5 years ago

Try installing emcee from github master (should be version 3.0rc2) and see if that works.

On Wed, May 29, 2019 at 3:39 AM reneheller notifications@github.com wrote:

I have emcee 2.2.1 and emcee3 3.0.0.dev0

heller$ pip3 list

===> emcee 2.2.1 emcee3 3.0.0.dev0 <===

I uninstalled emcee and reinstalled emcee, making sure I have the latest version. But that's still/again emcee 2.2.1.

How can I tell calcfpp to use emcee3 instead of emcee?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/timothydmorton/VESPA/issues/29?email_source=notifications&email_token=AAOOXW7FC3YLINAJ4CBBIALPXYXLZA5CNFSM4HPVPCZKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWOOTQQ#issuecomment-496822722, or mute the thread https://github.com/notifications/unsubscribe-auth/AAOOXW2NRFUNYYZSC44ZPBLPXYXLZANCNFSM4HPVPCZA .

-- about.me/tim.morton

reneheller commented 5 years ago

This also didn't help. I installed emcee 3.0rc2 from the github master (https://emcee.readthedocs.io/en/latest/user/install), then deleted all previous output files from vespa and tried it again. The first step seems to have worked without problems:

heller$ starfit --all . ===> 1 of 1: . Fitting . single starfit... (...) ln(ev)= -47.953257352215843 +/- 0.16984685405821168
Total Likelihood Evaluations: 879009 Sampling finished. Exiting MultiNest triple starfit successful for . in 11.7 minutes. <===

But then this happens:

heller$ calcfpp

===> /usr/local/lib/python3.7/site-packages/numpy/lib/npyio.py:2242: VisibleDeprecationWarning: Reading unicode strings without specifying the encoding argument is deprecated. Set the encoding, use None for the system default. output = genfromtxt(fname, kwargs) WARNING:root:sklearn not available WARNING:root:progressbar not imported WARNING:root:Cadence not provided in fpp.ini; defaulting to Kepler cadence. Cadence not provided in fpp.ini; defaulting to Kepler cadence. WARNING:root:If this is not a Kepler target, please set cadence (in days). If this is not a Kepler target, please set cadence (in days). WARNING:root:Transit band not provided in fpp.ini; defaulting to "Kepler." Transit band not provided in fpp.ini; defaulting to "Kepler." WARNING:root:If this is not a Kepler target, please specify filter. If this is not a Kepler target, please specify filter. INFO:root:Fitting transitsignal with MCMC... Fitting transitsignal with MCMC... INFO:root:Fitting data to trapezoid shape with MCMC for EPIC201497682_3.... Fitting data to trapezoid shape with MCMC for EPIC201497682_3.... INFO:root:Generating ['beb', 'heb', 'eb', 'beb_Px2', 'heb_Px2', 'eb_Px2', 'pl'] models for PopulationSet... Generating ['beb', 'heb', 'eb', 'beb_Px2', 'heb_Px2', 'eb_Px2', 'pl'] models for PopulationSet... ERROR:root:Error generating HEB population. Error generating HEB population. ERROR:root:FPP calculation failed for .. Traceback (most recent call last): File "/usr/local/bin/calcfpp", line 151, in n=args.n) File "/usr/local/lib/python3.7/site-packages/vespa/fpp.py", line 296, in from_ini savefile=popset_file, kwargs) File "/usr/local/lib/python3.7/site-packages/vespa/populations.py", line 2067, in init do_only=do_only) File "/usr/local/lib/python3.7/site-packages/vespa/populations.py", line 2113, in generate MAfn=MAfn, n=n, heb_kws) File "/usr/local/lib/python3.7/site-packages/vespa/populations.py", line 1558, in init kwargs) File "/usr/local/lib/python3.7/site-packages/vespa/populations.py", line 1577, in generate n=2*n) File "/usr/local/lib/python3.7/site-packages/vespa/stars/populations.py", line 1779, in init period=period, ecc=ecc, kwargs) File "/usr/local/lib/python3.7/site-packages/vespa/stars/populations.py", line 1850, in generate ecc_short=ecc, kwargs) File "/usr/local/lib/python3.7/site-packages/vespa/stars/populations.py", line 1526, in init ecclong=ecc_long, eccshort=ecc_short) File "/usr/local/lib/python3.7/site-packages/vespa/orbits/populations.py", line 108, in init obsx=obsx_long,obsy=obsy_long,obsz=obsz_long) File "/usr/local/lib/python3.7/site-packages/vespa/orbits/populations.py", line 334, in init self.obspos = random_spherepos(n) File "/usr/local/lib/python3.7/site-packages/vespa/orbits/utils.py", line 50, in random_spherepos c = SkyCoord(phis,thetas,1,representation='physicsspherical') File "/usr/local/lib/python3.7/site-packages/astropy/coordinates/sky_coordinate.py", line 250, in init frame_cls, frame_kwargs = _get_frame_without_data(args, kwargs) File "/usr/local/lib/python3.7/site-packages/astropy/coordinates/sky_coordinate_parsers.py", line 201, in _get_frame_without_data _normalize_representation_type(kwargs) File "/usr/local/lib/python3.7/site-packages/astropy/coordinates/baseframe.py", line 141, in _normalize_representation_type _representation_deprecation() File "/usr/local/lib/python3.7/site-packages/astropy/coordinates/baseframe.py", line 128, in _representation_deprecation 'favor of representation_type', AstropyDeprecationWarning) astropy.utils.exceptions.AstropyDeprecationWarning: The representation keyword/property name is deprecated in favor of representation_type FPP calculation failed for .. Traceback (most recent call last): File "/usr/local/bin/calcfpp", line 151, in n=args.n) File "/usr/local/lib/python3.7/site-packages/vespa/fpp.py", line 296, in from_ini savefile=popset_file, kwargs) File "/usr/local/lib/python3.7/site-packages/vespa/populations.py", line 2067, in init do_only=do_only) File "/usr/local/lib/python3.7/site-packages/vespa/populations.py", line 2113, in generate MAfn=MAfn, n=n, heb_kws) File "/usr/local/lib/python3.7/site-packages/vespa/populations.py", line 1558, in init *kwargs) File "/usr/local/lib/python3.7/site-packages/vespa/populations.py", line 1577, in generate n=2n) File "/usr/local/lib/python3.7/site-packages/vespa/stars/populations.py", line 1779, in init period=period, ecc=ecc, kwargs) File "/usr/local/lib/python3.7/site-packages/vespa/stars/populations.py", line 1850, in generate ecc_short=ecc, kwargs) File "/usr/local/lib/python3.7/site-packages/vespa/stars/populations.py", line 1526, in init ecclong=ecc_long, eccshort=ecc_short) File "/usr/local/lib/python3.7/site-packages/vespa/orbits/populations.py", line 108, in init obsx=obsx_long,obsy=obsy_long,obsz=obsz_long) File "/usr/local/lib/python3.7/site-packages/vespa/orbits/populations.py", line 334, in init self.obspos = random_spherepos(n) File "/usr/local/lib/python3.7/site-packages/vespa/orbits/utils.py", line 50, in random_spherepos c = SkyCoord(phis,thetas,1,representation='physicsspherical') File "/usr/local/lib/python3.7/site-packages/astropy/coordinates/sky_coordinate.py", line 250, in init frame_cls, frame_kwargs = _get_frame_without_data(args, kwargs) File "/usr/local/lib/python3.7/site-packages/astropy/coordinates/sky_coordinate_parsers.py", line 201, in _get_frame_without_data _normalize_representation_type(kwargs) File "/usr/local/lib/python3.7/site-packages/astropy/coordinates/baseframe.py", line 141, in _normalize_representation_type _representation_deprecation() File "/usr/local/lib/python3.7/site-packages/astropy/coordinates/baseframe.py", line 128, in _representation_deprecation 'favor of representation_type', AstropyDeprecationWarning) astropy.utils.exceptions.AstropyDeprecationWarning: The representation keyword/property name is deprecated in favor of representation_type <===

wangxianyu7 commented 5 years ago

I met the same problem. Vespa is not (yet) compatible with some latest packages.

pip3 install astropy==1.3 pandas==0.19.2 emcee==3.0rc2 could make it work.

reneheller commented 5 years ago

I tried

pip3 install astropy==1.3 pandas==0.19.2 emcee==3.0rc2

which led to the following problem.

Failed to build astropy pandas vespa 0.6 has requirement pandas>=0.21, but you'll have pandas 0.19.2 which is incompatible.

wangxianyu7 commented 5 years ago

I was wrong. I made it work using below packages.

astropy (1.3) batman-package (2.4.6) isochrones (1.2.2) numpy (1.17.0) pandas (0.21.0) VESPA (0.6)

I met another problem about ''mixed type'' that pandas cant recognize data tpye correctly when isochrone package read HDF. The reason is some values of [Fe/H] are nan or something that are recognized as 'object'. Then tpyes, float and object, exits in the same column.

If you met the same problem, change the file (isochrone package /mist/grid.py) may be helpful.

 189        df = pd.read_table(filename, comment='#', delim_whitespace=True,
 190                            skip_blank_lines=True, names=column_names,  dtype={'[Fe/H]':object}) 
reneheller commented 5 years ago

I can't downgrade to pandas 0.21.

pip3 install pandas==0.21

gives an error:

Command "/usr/local/opt/python/bin/python3.7 /usr/local/lib/python3.7/site-packages/pip install --ignore-installed --no-user --prefix /private/tmp/pip-build-env-macelzw7/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- wheel setuptools Cython "numpy==1.9.3; python_version=='3.5'" "numpy==1.12.1; python_version=='3.6'" "numpy==1.13.1; python_version>='3.7'"" failed with error code 1 in None

So I'm stuck with pandas 0.24.2. Then when I execute

starfit --all .

in my test folder with the fpp.ini, star.ini, and lightcurve.csv files, I receive

Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/isochrones/starfit.py", line 104, in starfit mod.fit(verbose=verbose, overwrite=overwrite, **kwargs) File "/usr/local/lib/python3.7/site-packages/isochrones/starmodel.py", line 556, in fit return self.fit_multinest(**kwargs) File "/usr/local/lib/python3.7/site-packages/isochrones/starmodel.py", line 692, in fit_multinest self._make_samples() File "/usr/local/lib/python3.7/site-packages/isochrones/starmodel.py", line 865, in _make_samples chain = np.loadtxt(filename) File "/usr/local/lib/python3.7/site-packages/numpy/lib/npyio.py", line 1146, in loadtxt for x in read_data(_loadtxt_chunksize): File "/usr/local/lib/python3.7/site-packages/numpy/lib/npyio.py", line 1074, in read_data items = [conv(val) for (conv, val) in zip(converters, vals)] File "/usr/local/lib/python3.7/site-packages/numpy/lib/npyio.py", line 1074, in <listcomp> items = [conv(val) for (conv, val) in zip(converters, vals)] File "/usr/local/lib/python3.7/site-packages/numpy/lib/npyio.py", line 781, in floatconv return float(x) ValueError: could not convert string to float: '0.694466137561774976-309'

wangxianyu7 commented 5 years ago

Try pip3 uninstall pandas before installing pandas==0.21.

I didn' t meet this problem. Check the values in the files of your test folder maybe of some help to you. Find which file includes or which function produces the '0.694466137561774976-309' is important.

The test case I used as below:

https://github.com/timothydmorton/vespa-tutorial/tree/master/Workshop-2018/TestCase1

ltipark commented 3 years ago

Hi, did this ever get resolved?

I've followed through the same steps and found the same errors. I've got stuck at the same place as this thread ends with the error:

triple starfit calculation failed for .. Traceback (most recent call last): File "/home/lsnc45/.local/lib/python3.8/site-packages/isochrones/starfit.py", line 104, in starfit mod.fit(verbose=verbose, overwrite=overwrite, kwargs) File "/home/lsnc45/.local/lib/python3.8/site-packages/isochrones/starmodel.py", line 556, in fit return self.fit_multinest(kwargs) File "/home/lsnc45/.local/lib/python3.8/site-packages/isochrones/starmodel.py", line 692, in fit_multinest self._make_samples() File "/home/lsnc45/.local/lib/python3.8/site-packages/isochrones/starmodel.py", line 865, in _make_samples chain = np.loadtxt(filename) File "/home/lsnc45/.local/lib/python3.8/site-packages/numpy/lib/npyio.py", line 1139, in loadtxt for x in read_data(_loadtxt_chunksize): File "/home/lsnc45/.local/lib/python3.8/site-packages/numpy/lib/npyio.py", line 1067, in read_data items = [conv(val) for (conv, val) in zip(converters, vals)] File "/home/lsnc45/.local/lib/python3.8/site-packages/numpy/lib/npyio.py", line 1067, in items = [conv(val) for (conv, val) in zip(converters, vals)] File "/home/lsnc45/.local/lib/python3.8/site-packages/numpy/lib/npyio.py", line 763, in floatconv return float(x) ValueError: could not convert string to float: '0.695332592751706726-309'

reneheller commented 3 years ago

Nope. This is the place where I stranded last time in trying to install vespa. Now, more than a year later, when I just tried to give this a new shot, I couldn't even reproduce this error. Instead, I'm meeting so many new error messages that I needed to get rid of that I'm inclined to conclude that vespa 0.6 is depreciated (python 3.8, macOS Catalina 10.15.7).

timothydmorton commented 3 years ago

Hi, all. I understand your frustration. #34 lays out a roadmap for bringing vespa up-to-date. If any of you on this issue see vespa as critical to your current research, I encourage you to contribute to its development. I would be happy to help guide if anyone is so interested. My current employment situation does not give me time to do this development myself, and this is a direct result of several declined rounds of funding proposals several years ago that were targeted at supporting continued vespa development. Lesson to the community: fund open-source analysis software if you want it to stay relevant!

priyashmistry commented 2 years ago

Anyone come up with the solution to above problem yet? I'm also facing the similar issues.

reneheller commented 2 years ago

Unfortunately not.

priyashmistry commented 2 years ago

Can't we try to install VESPA via .yml file generated from other machine in which vespa is running perfectly?

timothydmorton commented 2 years ago

Have any of you tried using some of the yaml files posted in https://github.com/timothydmorton/VESPA/issues/36? Also, I know @jessiechristiansen and @sakheebhure have somehow been using vespa recently, so maybe they can post their env files.

priyashmistry commented 2 years ago

I tried but nothing just happened. It is working for starfit --all but not for calcfpp

jessiechristiansen commented 2 years ago

Here is my environment which successfully runs vespa (after much blood, sweat and tears!).

Ah, it won't let me attach a .yml file. Copy this text into your own .yml file:

name: vespa channels:

priyashmistry commented 2 years ago

@jessiechristiansen @timothydmorton I tried to install vespa through above YAML file, but end up getting a weird error.

`(base) priyash@priyash-Latitude-E5520:~/Desktop$ conda env update -n my_env --file vespa6.yml

>>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<

Traceback (most recent call last):
  File "/home/priyash/anaconda3/lib/python3.9/site-packages/conda/exceptions.py", line 1080, in __call__
    return func(*args, **kwargs)
  File "/home/priyash/anaconda3/lib/python3.9/site-packages/conda_env/cli/main.py", line 80, in do_call
    exit_code = getattr(module, func_name)(args, parser)
  File "/home/priyash/anaconda3/lib/python3.9/site-packages/conda_env/cli/main_update.py", line 66, in execute
    spec = install_specs.detect(name=name, filename=get_filename(args.file),
  File "/home/priyash/anaconda3/lib/python3.9/site-packages/conda_env/specs/__init__.py", line 43, in detect
    if spec.can_handle():
  File "/home/priyash/anaconda3/lib/python3.9/site-packages/conda_env/specs/yaml_file.py", line 18, in can_handle
    self._environment = env.from_file(self.filename)
  File "/home/priyash/anaconda3/lib/python3.9/site-packages/conda_env/env.py", line 166, in from_file
    return from_yaml(yamlstr, filename=filename)
  File "/home/priyash/anaconda3/lib/python3.9/site-packages/conda_env/env.py", line 143, in from_yaml
    data = yaml_safe_load(yamlstr)
  File "/home/priyash/anaconda3/lib/python3.9/site-packages/conda/common/serialize.py", line 67, in yaml_safe_load
    return yaml.safe_load(string, version="1.2")
  File "/home/priyash/anaconda3/lib/python3.9/site-packages/ruamel_yaml/main.py", line 980, in safe_load
    return load(stream, SafeLoader, version)
  File "/home/priyash/anaconda3/lib/python3.9/site-packages/ruamel_yaml/main.py", line 935, in load
    return loader._constructor.get_single_data()
  File "/home/priyash/anaconda3/lib/python3.9/site-packages/ruamel_yaml/constructor.py", line 109, in get_single_data
    node = self.composer.get_single_node()
  File "/home/priyash/anaconda3/lib/python3.9/site-packages/ruamel_yaml/composer.py", line 78, in get_single_node
    document = self.compose_document()
  File "/home/priyash/anaconda3/lib/python3.9/site-packages/ruamel_yaml/composer.py", line 101, in compose_document
    node = self.compose_node(None, None)
  File "/home/priyash/anaconda3/lib/python3.9/site-packages/ruamel_yaml/composer.py", line 138, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/home/priyash/anaconda3/lib/python3.9/site-packages/ruamel_yaml/composer.py", line 211, in compose_mapping_node
    while not self.parser.check_event(MappingEndEvent):
  File "/home/priyash/anaconda3/lib/python3.9/site-packages/ruamel_yaml/parser.py", line 141, in check_event
    self.current_event = self.state()
  File "/home/priyash/anaconda3/lib/python3.9/site-packages/ruamel_yaml/parser.py", line 577, in parse_block_mapping_key
    raise ParserError(
ruamel_yaml.parser.ParserError: while parsing a block mapping
  in "<unicode string>", line 1, column 1:
    name: vespa
    ^ (line: 1)
expected <block end>, but found '<block mapping start>'
  in "<unicode string>", line 5, column 2:
     dependencies:
     ^ (line: 5)

$ /home/priyash/anaconda3/bin/conda-env update -n my_env --file vespa6.yml

environment variables: CIO_TEST= CONDA_AUTO_UPDATE_CONDA=false CONDA_DEFAULT_ENV=base CONDA_EXE=/home/priyash/anaconda3/bin/conda CONDA_PREFIX=/home/priyash/anaconda3 CONDA_PROMPT_MODIFIER=(base) CONDA_PYTHON_EXE=/home/priyash/anaconda3/bin/python CONDA_ROOT=/home/priyash/anaconda3 CONDA_SHLVL=1 CURL_CA_BUNDLE= LD_LIBRARY_PATH=:/Desktop/PyMultiNest/lib/:Home/Desktop/MultiNest/lib/:home/desktop/Mu ltiNest/lib/:home/desktop/MultiNest/lib/:/home/priyash/Desktop/MultiNe st/lib/ PATH=/home/priyash/anaconda3/bin:/home/priyash/anaconda3/bin:/home/priyash/ anaconda3/condabin:/home/priyash/.local/bin:/usr/local/sbin:/usr/local /bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/b in REQUESTS_CA_BUNDLE= SSL_CERT_FILE= WINDOWPATH=2

 active environment : base
active env location : /home/priyash/anaconda3
        shell level : 1
   user config file : /home/priyash/.condarc

populated config files : /home/priyash/.condarc conda version : 4.11.0 conda-build version : 3.21.7 python version : 3.9.7.final.0 virtual packages : linux=5.13.0=0 glibc=2.31=0 unix=0=0 archspec=1=x86_64 base environment : /home/priyash/anaconda3 (writable) conda av data dir : /home/priyash/anaconda3/etc/conda conda av metadata url : None channel URLs : https://repo.anaconda.com/pkgs/main/linux-64 https://repo.anaconda.com/pkgs/main/noarch https://repo.anaconda.com/pkgs/r/linux-64 https://repo.anaconda.com/pkgs/r/noarch package cache : /home/priyash/anaconda3/pkgs /home/priyash/.conda/pkgs envs directories : /home/priyash/anaconda3/envs /home/priyash/.conda/envs platform : linux-64 user-agent : conda/4.11.0 requests/2.26.0 CPython/3.9.7 Linux/5.13.0-28-generic ubuntu/20.04.3 glibc/2.31 UID:GID : 1000:1000 netrc file : None offline mode : False

An unexpected error has occurred. Conda has prepared the above report.

If submitted, this report will be used by core maintainers to improve future releases of conda. Would you like conda to send this report to the core maintainers?

[y/N]: Timeout reached. No report sent.`

Is there any problem with my YAML file ? here is the file I used: `name: vespa channels:

jessiechristiansen commented 2 years ago

You probably at least have to change the prefix at the bottom which is still hard-coded to point at my directory!

alexteachey commented 2 years ago

See my solution here -- #36

I have built working .yml files for MacOS (tested on Big Sur) and Linux (Ubuntu 20.04). If you install my MoonPy code, this should all be handled for you (including some modifications to the vespa code, such as the representation --> representation_type keyword issue).

So far I can only get vespa to work with isochrones v1.1.1. Critically, you have to clobber your existing ~/.isochrones folder if you change versions of isochrones. I currently encounter the starfit --all breakdown with isochrones v2.1. I'm working on a fix right now.