search

timothydmorton / isochrones

Pythonic stellar model grid access; easy MCMC fitting of stellar properties
http://isochrones.readthedocs.org
MIT License
117 stars 63 forks source link

[WIP] Starting to try to fix Pandas compatibility #139

Closed dfm closed 2 years ago

dfm commented 3 years ago

Not ready to merge, just experimenting.

138

timothydmorton commented 3 years ago

My local isochrones dev environment on my laptop still has 0.24.2, so I'm super far behind. Will investigate.

timothydmorton commented 3 years ago

Update: I have made a new environment locally (OSX), following exactly the conda installation pattern in https://github.com/timothydmorton/isochrones/blob/master/.ci/travis.sh, and pytest -v runs successfully for me.

timothydmorton commented 3 years ago

Still works after upgrading pandas. Now I'll try removing the $ISOCHRONES and rerunning mist-initialize.py using current pandas and see if things still work.

timothydmorton commented 3 years ago

Yup, there it is, trying to run mist-initialize.py with current pandas:

Traceback (most recent call last):
  File "/Users/tmorton/miniconda3/envs/isochrones-update/bin/mist-initialize.py", line 4, in <module>
    __import__('pkg_resources').run_script('isochrones==2.1', 'mist-initialize.py')
  File "/Users/tmorton/miniconda3/envs/isochrones-update/lib/python3.8/site-packages/pkg_resources/__init__.py", line 665, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/Users/tmorton/miniconda3/envs/isochrones-update/lib/python3.8/site-packages/pkg_resources/__init__.py", line 1463, in run_script
    exec(code, namespace, namespace)
  File "/Users/tmorton/miniconda3/envs/isochrones-update/lib/python3.8/site-packages/isochrones-2.1-py3.8.egg/EGG-INFO/scripts/mist-initialize.py", line 4, in <module>
    mist.initialize()
  File "/Users/tmorton/miniconda3/envs/isochrones-update/lib/python3.8/site-packages/isochrones-2.1-py3.8.egg/isochrones/models.py", line 356, in initialize
    Teff, logg, feh, mags = self.interp_mag(pars, self.bands)
  File "/Users/tmorton/miniconda3/envs/isochrones-update/lib/python3.8/site-packages/isochrones-2.1-py3.8.egg/isochrones/models.py", line 436, in interp_mag
    self.model_grid.interp.grid,
  File "/Users/tmorton/miniconda3/envs/isochrones-update/lib/python3.8/site-packages/isochrones-2.1-py3.8.egg/isochrones/grid.py", line 136, in interp
    self._interp = DFInterpolator(self.df, filename=filename, is_full=self.is_full)
  File "/Users/tmorton/miniconda3/envs/isochrones-update/lib/python3.8/site-packages/isochrones-2.1-py3.8.egg/isochrones/models.py", line 158, in df
    self._df = self.read_hdf()
  File "/Users/tmorton/miniconda3/envs/isochrones-update/lib/python3.8/site-packages/isochrones-2.1-py3.8.egg/isochrones/grid.py", line 109, in read_hdf
    df = self.write_hdf(orig=orig)
  File "/Users/tmorton/miniconda3/envs/isochrones-update/lib/python3.8/site-packages/isochrones-2.1-py3.8.egg/isochrones/grid.py", line 113, in write_hdf
    df = self.get_df(orig=orig)
  File "/Users/tmorton/miniconda3/envs/isochrones-update/lib/python3.8/site-packages/isochrones-2.1-py3.8.egg/isochrones/models.py", line 114, in get_df
    df = self.df_all()
  File "/Users/tmorton/miniconda3/envs/isochrones-update/lib/python3.8/site-packages/isochrones-2.1-py3.8.egg/isochrones/models.py", line 97, in df_all
    df = pd.concat([self.to_df(f) for f in self.get_filenames()])
  File "/Users/tmorton/miniconda3/envs/isochrones-update/lib/python3.8/site-packages/isochrones-2.1-py3.8.egg/isochrones/models.py", line 97, in <listcomp>
    df = pd.concat([self.to_df(f) for f in self.get_filenames()])
  File "/Users/tmorton/miniconda3/envs/isochrones-update/lib/python3.8/site-packages/isochrones-2.1-py3.8.egg/isochrones/mist/models.py", line 144, in to_df
    df = pd.read_table(
  File "/Users/tmorton/miniconda3/envs/isochrones-update/lib/python3.8/site-packages/pandas/io/parsers.py", line 755, in read_table
    return read_csv(**locals())
  File "/Users/tmorton/miniconda3/envs/isochrones-update/lib/python3.8/site-packages/pandas/io/parsers.py", line 624, in read_csv
    raise ValueError(
ValueError: Specified a delimiter with both sep and delim_whitespace=True; you can only specify one.
timothydmorton commented 3 years ago

I think I found it. Confirmation pending.

timothydmorton commented 3 years ago

OK, the problem is with these lines: https://github.com/timothydmorton/isochrones/blob/3a187b9f75441859527c28e0139a8de270a53f88/isochrones/mist/models.py#L144 https://github.com/timothydmorton/isochrones/blob/3a187b9f75441859527c28e0139a8de270a53f88/isochrones/mist/models.py#L277

But I haven't found the right solution yet. I've tried both changing to read_csv and leaving read_table but with sep='\s+', taking out delim_whitespace kw. Both run mist-initialize.py without error but break in the tests, seemingly reading blank tables. Haven't debugged any further at present.

dawsonri commented 3 years ago

A student and I are running into the same issue. Do we need to install an earlier version of pandas?

dfm commented 3 years ago

@dawsonri: yes, pandas<1.1 should do the trick for now, I think. Let us know if you have any issues.

dawsonri commented 3 years ago

@dfm Thank you, that worked!