search

timothydmorton / isochrones

Pythonic stellar model grid access; easy MCMC fitting of stellar properties
http://isochrones.readthedocs.org
MIT License
122 stars 62 forks source link

Unable to open/create file '/Users/era/.isochrones/mist/MIST_v1.1_vvcrit0.0_UBVRIplus.h5' #81

Open elisabethadams opened 5 years ago

elisabethadams commented 5 years ago

I can't get mist to work (using starfit --all), dartmouth works just fine.

Traceback (most recent call last):
  File "//anaconda/envs/vespa3/lib/python3.6/site-packages/isochrones/starfit.py", line 101, in starfit
    mod.fit(verbose=verbose, overwrite=overwrite, **kwargs)
  File "//anaconda/envs/vespa3/lib/python3.6/site-packages/isochrones/starmodel.py", line 556, in fit
    return self.fit_multinest(**kwargs)
  File "//anaconda/envs/vespa3/lib/python3.6/site-packages/isochrones/starmodel.py", line 692, in fit_multinest
    self._make_samples()
  File "//anaconda/envs/vespa3/lib/python3.6/site-packages/isochrones/starmodel.py", line 896, in _make_samples
    distance=distance, AV=AV)
  File "//anaconda/envs/vespa3/lib/python3.6/site-packages/isochrones/isochrone.py", line 275, in __call__
    Rs = self.radius(*args)*1
  File "//anaconda/envs/vespa3/lib/python3.6/site-packages/isochrones/isochrone.py", line 199, in radius
    return np.sqrt(G*self.mass(*args)*MSUN/10**self.logg(*args))/RSUN
  File "//anaconda/envs/vespa3/lib/python3.6/site-packages/isochrones/isochrone.py", line 687, in logg
    return self.interp_value(mass, age, feh, self.logg_col)
  File "//anaconda/envs/vespa3/lib/python3.6/site-packages/isochrones/isochrone.py", line 747, in interp_value
    self._initialize()
  File "//anaconda/envs/vespa3/lib/python3.6/site-packages/isochrones/isochrone.py", line 610, in _initialize
    _ = getattr(self, attr)
  File "//anaconda/envs/vespa3/lib/python3.6/site-packages/isochrones/isochrone.py", line 616, in df
    self._df = self.modelgrid(self.bands, **self.modelgrid_kwargs).df
  File "//anaconda/envs/vespa3/lib/python3.6/site-packages/isochrones/grid.py", line 97, in df
    self._df = self._get_df()
  File "//anaconda/envs/vespa3/lib/python3.6/site-packages/isochrones/grid.py", line 112, in _get_df
    grids[s] = self.get_hdf(s)
  File "//anaconda/envs/vespa3/lib/python3.6/site-packages/isochrones/grid.py", line 199, in get_hdf
    df = self.write_hdf(phot)
  File "//anaconda/envs/vespa3/lib/python3.6/site-packages/isochrones/grid.py", line 205, in write_hdf
    df.to_hdf(h5file,'df')
  File "//anaconda/envs/vespa3/lib/python3.6/site-packages/pandas/core/generic.py", line 2378, in to_hdf
    return pytables.to_hdf(path_or_buf, key, self, **kwargs)
  File "//anaconda/envs/vespa3/lib/python3.6/site-packages/pandas/io/pytables.py", line 273, in to_hdf
    complib=complib) as store:
  File "//anaconda/envs/vespa3/lib/python3.6/site-packages/pandas/io/pytables.py", line 488, in __init__
    self.open(mode=mode, **kwargs)
  File "//anaconda/envs/vespa3/lib/python3.6/site-packages/pandas/io/pytables.py", line 603, in open
    self._handle = tables.open_file(self._path, self._mode, **kwargs)
  File "//anaconda/envs/vespa3/lib/python3.6/site-packages/tables/file.py", line 320, in open_file
    return File(filename, mode, title, root_uep, filters, **kwargs)
  File "//anaconda/envs/vespa3/lib/python3.6/site-packages/tables/file.py", line 784, in __init__
    self._g_new(filename, mode, **params)
  File "tables/hdf5extension.pyx", line 492, in tables.hdf5extension.File._g_new
tables.exceptions.HDF5ExtError: HDF5 error back trace

  File "H5F.c", line 509, in H5Fopen
    unable to open file
  File "H5Fint.c", line 1400, in H5F__open
    unable to open file
  File "H5Fint.c", line 1700, in H5F_open
    unable to read superblock
  File "H5Fsuper.c", line 623, in H5F__super_read
    truncated file: eof = 824180678, sblock->base_addr = 0, stored_eof = 836049190

End of HDF5 error back trace

Unable to open/create file '/Users/era/.isochrones/mist/MIST_v1.1_vvcrit0.0_UBVRIplus.h5'
elisabethadams commented 5 years ago

And to be clear, this file does exist (from a very fresh re-installation of isochrones):

ls -lat /Users/era/.isochrones/mist/MIST_v1.1_vvcrit0.0_UBVRIplus.h5

-rw-r--r--  1 era  staff  824180678 Feb  9 05:35 /Users/era/.isochrones/mist/MIST_v1.1_vvcrit0.0_UBVRIplus.h5
perkier commented 5 years ago

Have you managed to find where the error was? I'm struggling with the same error

ishdutt commented 5 years ago

Hey hi perkier.This error is probably due to improper download of the package.After running this command nosetests isochrones , it takes a lot of time as it creates a directory .isochrones and if this test shows more number of errors or is quit in between these type of error will show up.

elisabethadams commented 5 years ago

Might there be a problem with accessing Gaia data? Here's where my nosetests isochrones fails:

> ======================================================================
> ERROR: Testing with first entry from Gaia DR1 TGAS table
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "//anaconda/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
>     self.test(*self.arg)
>   File "/Users/era/isochrones/isochrones/tests/test_query.py", line 14, in test_queries
>     assert tm.get_id() == '03000819+0014074'
>   File "/Users/era/isochrones/isochrones/query/catalog.py", line 66, in get_id
>     row = self.closest
>   File "/Users/era/isochrones/isochrones/query/catalog.py", line 53, in closest
>     df = self.df.sort_values(by='_r')
>   File "/Users/era/isochrones/isochrones/query/catalog.py", line 49, in df
>     return pd.DataFrame(np.array(self.table))
>   File "/Users/era/isochrones/isochrones/query/catalog.py", line 44, in table
>     self._run_query()
>   File "/Users/era/isochrones/isochrones/query/vizier.py", line 22, in _run_query
>     self._coords = SkyCoord(self._table['_RAJ2000'], self._table['_DEJ2000'], unit='deg')
>   File "//anaconda/lib/python2.7/site-packages/astropy/table/table.py", line 1222, in __getitem__
>     return self.columns[item]
>   File "//anaconda/lib/python2.7/site-packages/astropy/table/table.py", line 109, in __getitem__
>     return OrderedDict.__getitem__(self, item)
> KeyError: '_RAJ2000'
> -------------------- >> begin captured logging << --------------------
> urllib3.connectionpool: DEBUG: Starting new HTTP connection (1): vizier.u-strasbg.fr:80
> urllib3.connectionpool: DEBUG: http://vizier.u-strasbg.fr:80 "POST /viz-bin/votable HTTP/1.1" 200 1651
> --------------------- >> end captured logging << ---------------------
> ----------------------------------------------------------------------
> Ran 13 tests in 139.666s
> FAILED (errors=1)
timothydmorton commented 5 years ago

Interesting; that test is not a very important one, as the query submodule isn't really central at all. I do note that I see python2.7 in there, and that I'm not really supporting python 2 anymore.

On Tue, Apr 9, 2019 at 3:11 PM Elisabeth Adams notifications@github.com wrote:

Might there be a problem with accessing Gaia data? Here's where my nosetests isochrones fails:

ERROR: Testing with first entry from Gaia DR1 TGAS table Traceback (most recent call last): File "//anaconda/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(self.arg) File "/Users/era/isochrones/isochrones/tests/test_query.py", line 14, in test_queries assert tm.get_id() == '03000819+0014074' File "/Users/era/isochrones/isochrones/query/catalog.py", line 66, in get_id row = self.closest File "/Users/era/isochrones/isochrones/query/catalog.py", line 53, in closest df = self.df.sort_values(by='_r') File "/Users/era/isochrones/isochrones/query/catalog.py", line 49, in df return pd.DataFrame(np.array(self.table)) File "/Users/era/isochrones/isochrones/query/catalog.py", line 44, in table self._run_query() File "/Users/era/isochrones/isochrones/query/vizier.py", line 22, in _run_query self._coords = SkyCoord(self._table['_RAJ2000'], self._table['_DEJ2000'], unit='deg') File "//anaconda/lib/python2.7/site-packages/astropy/table/table.py", line 1222, in getitem return self.columns[item] File "//anaconda/lib/python2.7/site-packages/astropy/table/table.py", line 109, in getitem return OrderedDict.getitem*(self, item) KeyError: '_RAJ2000' -------------------- >> begin captured logging << -------------------- urllib3.connectionpool: DEBUG: Starting new HTTP connection (1): vizier.u-strasbg.fr:80 urllib3.connectionpool: DEBUG: http://vizier.u-strasbg.fr:80 "POST /viz-bin/votable HTTP/1.1" 200 1651 --------------------- >> end captured logging << ---------------------

Ran 13 tests in 139.666s

FAILED (errors=1)

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/timothydmorton/isochrones/issues/81#issuecomment-481396336, or mute the thread https://github.com/notifications/unsubscribe-auth/ABzr225oFg31sW9rMcLiiCd7OS7A5gt3ks5vfOXSgaJpZM4bS6Bj .

-- about.me/tim.morton

ishdutt commented 5 years ago

In my case nosetest isochrones also fails at a different point (1 error and 15 tests). But I am encountering a different error ERROR: isochrones.tests.test_fits.test_fitting and I am not able to use starmodel class(it runs for a very long time and finally I have to terminate the process without getting the results). I have opened an issue also #84 for this.

Might there be a problem with accessing Gaia data? Here's where my nosetests isochrones fails:

> ======================================================================
> ERROR: Testing with first entry from Gaia DR1 TGAS table
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "//anaconda/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
>     self.test(*self.arg)
>   File "/Users/era/isochrones/isochrones/tests/test_query.py", line 14, in test_queries
>     assert tm.get_id() == '03000819+0014074'
>   File "/Users/era/isochrones/isochrones/query/catalog.py", line 66, in get_id
>     row = self.closest
>   File "/Users/era/isochrones/isochrones/query/catalog.py", line 53, in closest
>     df = self.df.sort_values(by='_r')
>   File "/Users/era/isochrones/isochrones/query/catalog.py", line 49, in df
>     return pd.DataFrame(np.array(self.table))
>   File "/Users/era/isochrones/isochrones/query/catalog.py", line 44, in table
>     self._run_query()
>   File "/Users/era/isochrones/isochrones/query/vizier.py", line 22, in _run_query
>     self._coords = SkyCoord(self._table['_RAJ2000'], self._table['_DEJ2000'], unit='deg')
>   File "//anaconda/lib/python2.7/site-packages/astropy/table/table.py", line 1222, in __getitem__
>     return self.columns[item]
>   File "//anaconda/lib/python2.7/site-packages/astropy/table/table.py", line 109, in __getitem__
>     return OrderedDict.__getitem__(self, item)
> KeyError: '_RAJ2000'
> -------------------- >> begin captured logging << --------------------
> urllib3.connectionpool: DEBUG: Starting new HTTP connection (1): vizier.u-strasbg.fr:80
> urllib3.connectionpool: DEBUG: http://vizier.u-strasbg.fr:80 "POST /viz-bin/votable HTTP/1.1" 200 1651
> --------------------- >> end captured logging << ---------------------
> ----------------------------------------------------------------------
> Ran 13 tests in 139.666s
> FAILED (errors=1)
gmedinat commented 3 years ago

Has anybody found a solution for this? I have the same problem, and it occurs when I do the nosetests isochrones test, and when following the instructions of the Quick Start guide (https://isochrones.readthedocs.io/en/latest/quickstart.html). The problem seems to appear when I use tracks.generate, and this is an extract of the shell output:

Traceback (most recent call last):
  File "quickstart_ISOCHRONES.py", line 58, in <module>
    df = tracks.generate(masses, age, feh, distance=distance, AV=AV)
  File "/home/User/anaconda3/envs/isochrones/lib/python3.8/site-packages/isochrones/models.py", line 604, in generate
    eeps = self.get_eep(mass, age, feh, **kwargs)
  File "/home/User/anaconda3/envs/isochrones/lib/python3.8/site-packages/isochrones/models.py", line 535, in get_eep
    grid.masses,
  File "/home/User/anaconda3/envs/isochrones/lib/python3.8/site-packages/isochrones/mist/models.py", line 189, in masses
    self._masses = np.array(self.df.index.levels[1])
  File "/home/User/anaconda3/envs/isochrones/lib/python3.8/site-packages/isochrones/mist/models.py", line 398, in df
    self._df = self.read_hdf()
  File "/home/User/anaconda3/envs/isochrones/lib/python3.8/site-packages/isochrones/grid.py", line 109, in read_hdf
    df = self.write_hdf(orig=orig)
  File "/home/User/anaconda3/envs/isochrones/lib/python3.8/site-packages/isochrones/grid.py", line 113, in write_hdf
    df = self.get_df(orig=orig)
  File "/home/User/anaconda3/envs/isochrones/lib/python3.8/site-packages/isochrones/models.py", line 114, in get_df
    df = self.df_all()
  File "/home/User/anaconda3/envs/isochrones/lib/python3.8/site-packages/isochrones/mist/models.py", line 392, in df_all
    df = pd.concat([self.df_all_feh_interpolated(feh) for feh in self.fehs])
  File "/home/User/anaconda3/envs/isochrones/lib/python3.8/site-packages/isochrones/mist/models.py", line 392, in <listcomp>
    df = pd.concat([self.df_all_feh_interpolated(feh) for feh in self.fehs])
  File "/home/User/anaconda3/envs/isochrones/lib/python3.8/site-packages/isochrones/mist/models.py", line 323, in df_all_feh_interpolated
    df_interp = pd.read_hdf(hdf_filename, "df")
  File "/home/User/anaconda3/envs/isochrones/lib/python3.8/site-packages/pandas/io/pytables.py", line 397, in read_hdf
    store = HDFStore(path_or_buf, mode=mode, errors=errors, **kwargs)
  File "/home/User/anaconda3/envs/isochrones/lib/python3.8/site-packages/pandas/io/pytables.py", line 537, in __init__
    self.open(mode=mode, **kwargs)
  File "/home/User/anaconda3/envs/isochrones/lib/python3.8/site-packages/pandas/io/pytables.py", line 687, in open
    raise IOError(str(err))
OSError: HDF5 error back trace

  File "H5F.c", line 509, in H5Fopen
    unable to open file
  File "H5Fint.c", line 1400, in H5F__open
    unable to open file
  File "H5Fint.c", line 1709, in H5F_open
    unable to read root group
  File "H5Groot.c", line 239, in H5G_mkroot
    can't check if symbol table message exists
  File "H5Omessage.c", line 883, in H5O_msg_exists
    unable to protect object header
  File "H5Oint.c", line 1100, in H5O_protect
    unable to load object header
  File "H5AC.c", line 1625, in H5AC_protect
    H5C_protect() failed
  File "H5C.c", line 2362, in H5C_protect
    can't load entry
  File "H5C.c", line 6712, in H5C_load_entry
    incorrect metadatda checksum after all read attempts
  File "H5Ocache.c", line 219, in H5O__cache_get_final_load_size
    can't deserialize object header prefix
  File "H5Ocache.c", line 1231, in H5O__prefix_deserialize
    bad object header version number

End of HDF5 error back trace

Unable to open/create file '/home/User/.isochrones/mist/tracks/MIST_v1.2_feh_m0.25_afe_p0.0_vvcrit0.4_EEPS/all_masses_interpolated.h5'

I installed Isochrones via:

conda create -n isochrones numpy numba nose pytables pandas
conda activate isochrones
pip install isochrones

and tried installing older versions of numpy, numba, and pandas to solve the problem (1.18.5, 0.50.1 and 1.0.5, respectively), but that didn't work either. The python version installed in the Isochrones conda environment is 3.8.5.