Closed JosephKarpinski closed 3 years ago
Did you also try cores=1
as I suggested on Twitter?
Yes,
Changing codes=2 to cores=1, had no impact. With Radial Velocity example: 27/8000 projected to run 12:49:48 plus
Sent from my iPad
On May 6, 2021, at 5:40 PM, Dan Foreman-Mackey @.***> wrote:
Did you also try cores=1 as I suggested on Twitter?
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.
Example:
Tried both
import multiprocessing as mp mp.set_start_method("fork")
and cores=1
No improvement
Additional documentation
(base) josephkarpinski@Josephs-iMac ~ % pip3 install -U exoplanet Collecting exoplanet Downloading exoplanet-0.5.0-py3-none-any.whl (39 kB) Collecting exoplanet-core>=0.1 Downloading exoplanet_core-0.1.1-cp38-cp38-macosx_10_9_x86_64.whl (142 kB) |████████████████████████████████| 142 kB 4.9 MB/s Requirement already satisfied, skipping upgrade: pymc3>=3.9 in ./opt/anaconda3/lib/python3.8/site-packages (from exoplanet) (3.11.2) Collecting aesara-theano-fallback>=0.0.2 Downloading aesara_theano_fallback-0.0.4-py3-none-any.whl (5.4 kB) Requirement already satisfied, skipping upgrade: astropy>=3.1 in ./opt/anaconda3/lib/python3.8/site-packages (from exoplanet) (4.2.1) Requirement already satisfied, skipping upgrade: numpy>=1.13.0 in ./opt/anaconda3/lib/python3.8/site-packages (from exoplanet-core>=0.1->exoplanet) (1.20.2) Requirement already satisfied, skipping upgrade: arviz>=0.11.0 in ./opt/anaconda3/lib/python3.8/site-packages (from pymc3>=3.9->exoplanet) (0.11.2) Requirement already satisfied, skipping upgrade: dill in ./opt/anaconda3/lib/python3.8/site-packages (from pymc3>=3.9->exoplanet) (0.3.3) Requirement already satisfied, skipping upgrade: theano-pymc==1.1.2 in ./opt/anaconda3/lib/python3.8/site-packages (from pymc3>=3.9->exoplanet) (1.1.2) Requirement already satisfied, skipping upgrade: semver in ./opt/anaconda3/lib/python3.8/site-packages (from pymc3>=3.9->exoplanet) (2.13.0) Requirement already satisfied, skipping upgrade: scipy>=1.2.0 in ./opt/anaconda3/lib/python3.8/site-packages (from pymc3>=3.9->exoplanet) (1.6.2) Requirement already satisfied, skipping upgrade: fastprogress>=0.2.0 in ./opt/anaconda3/lib/python3.8/site-packages (from pymc3>=3.9->exoplanet) (1.0.0) Requirement already satisfied, skipping upgrade: patsy>=0.5.1 in ./opt/anaconda3/lib/python3.8/site-packages (from pymc3>=3.9->exoplanet) (0.5.1) Requirement already satisfied, skipping upgrade: typing-extensions>=3.7.4 in ./opt/anaconda3/lib/python3.8/site-packages (from pymc3>=3.9->exoplanet) (3.7.4.3) Requirement already satisfied, skipping upgrade: cachetools>=4.2.1 in ./opt/anaconda3/lib/python3.8/site-packages (from pymc3>=3.9->exoplanet) (4.2.1) Requirement already satisfied, skipping upgrade: pandas>=0.24.0 in ./opt/anaconda3/lib/python3.8/site-packages (from pymc3>=3.9->exoplanet) (1.2.4) Requirement already satisfied, skipping upgrade: pyerfa in ./opt/anaconda3/lib/python3.8/site-packages (from astropy>=3.1->exoplanet) (1.7.2) Requirement already satisfied, skipping upgrade: packaging in ./opt/anaconda3/lib/python3.8/site-packages (from arviz>=0.11.0->pymc3>=3.9->exoplanet) (20.4) Requirement already satisfied, skipping upgrade: matplotlib>=3.0 in ./opt/anaconda3/lib/python3.8/site-packages (from arviz>=0.11.0->pymc3>=3.9->exoplanet) (3.4.1) Requirement already satisfied, skipping upgrade: netcdf4 in ./opt/anaconda3/lib/python3.8/site-packages (from arviz>=0.11.0->pymc3>=3.9->exoplanet) (1.5.6) Requirement already satisfied, skipping upgrade: setuptools>=38.4 in ./opt/anaconda3/lib/python3.8/site-packages (from arviz>=0.11.0->pymc3>=3.9->exoplanet) (56.0.0) Requirement already satisfied, skipping upgrade: xarray>=0.16.1 in ./opt/anaconda3/lib/python3.8/site-packages (from arviz>=0.11.0->pymc3>=3.9->exoplanet) (0.17.0) Requirement already satisfied, skipping upgrade: filelock in ./opt/anaconda3/lib/python3.8/site-packages (from theano-pymc==1.1.2->pymc3>=3.9->exoplanet) (3.0.12) Requirement already satisfied, skipping upgrade: six in ./opt/anaconda3/lib/python3.8/site-packages (from patsy>=0.5.1->pymc3>=3.9->exoplanet) (1.15.0) Requirement already satisfied, skipping upgrade: python-dateutil>=2.7.3 in ./opt/anaconda3/lib/python3.8/site-packages (from pandas>=0.24.0->pymc3>=3.9->exoplanet) (2.8.1) Requirement already satisfied, skipping upgrade: pytz>=2017.3 in ./opt/anaconda3/lib/python3.8/site-packages (from pandas>=0.24.0->pymc3>=3.9->exoplanet) (2021.1) Requirement already satisfied, skipping upgrade: pyparsing>=2.0.2 in ./opt/anaconda3/lib/python3.8/site-packages (from packaging->arviz>=0.11.0->pymc3>=3.9->exoplanet) (2.4.7) Requirement already satisfied, skipping upgrade: cycler>=0.10 in ./opt/anaconda3/lib/python3.8/site-packages (from matplotlib>=3.0->arviz>=0.11.0->pymc3>=3.9->exoplanet) (0.10.0) Requirement already satisfied, skipping upgrade: kiwisolver>=1.0.1 in ./opt/anaconda3/lib/python3.8/site-packages (from matplotlib>=3.0->arviz>=0.11.0->pymc3>=3.9->exoplanet) (1.3.1) Requirement already satisfied, skipping upgrade: pillow>=6.2.0 in ./opt/anaconda3/lib/python3.8/site-packages (from matplotlib>=3.0->arviz>=0.11.0->pymc3>=3.9->exoplanet) (8.2.0) Requirement already satisfied, skipping upgrade: cftime in ./opt/anaconda3/lib/python3.8/site-packages (from netcdf4->arviz>=0.11.0->pymc3>=3.9->exoplanet) (1.4.1) Installing collected packages: exoplanet-core, aesara-theano-fallback, exoplanet Successfully installed aesara-theano-fallback-0.0.4 exoplanet-0.5.0 exoplanet-core-0.1.1
(base) josephkarpinski@Josephs-iMac ~ % pip3 install pymc3_ext Collecting pymc3_ext Downloading pymc3_ext-0.1.0-py2.py3-none-any.whl (22 kB) Requirement already satisfied: aesara-theano-fallback>=0.0.2 in ./opt/anaconda3/lib/python3.8/site-packages (from pymc3_ext) (0.0.4) Requirement already satisfied: pymc3>=3.9 in ./opt/anaconda3/lib/python3.8/site-packages (from pymc3_ext) (3.11.2) Requirement already satisfied: arviz>=0.11.0 in ./opt/anaconda3/lib/python3.8/site-packages (from pymc3>=3.9->pymc3_ext) (0.11.2) Requirement already satisfied: patsy>=0.5.1 in ./opt/anaconda3/lib/python3.8/site-packages (from pymc3>=3.9->pymc3_ext) (0.5.1) Requirement already satisfied: theano-pymc==1.1.2 in ./opt/anaconda3/lib/python3.8/site-packages (from pymc3>=3.9->pymc3_ext) (1.1.2) Requirement already satisfied: semver in ./opt/anaconda3/lib/python3.8/site-packages (from pymc3>=3.9->pymc3_ext) (2.13.0) Requirement already satisfied: scipy>=1.2.0 in ./opt/anaconda3/lib/python3.8/site-packages (from pymc3>=3.9->pymc3_ext) (1.6.2) Requirement already satisfied: pandas>=0.24.0 in ./opt/anaconda3/lib/python3.8/site-packages (from pymc3>=3.9->pymc3_ext) (1.2.4) Requirement already satisfied: typing-extensions>=3.7.4 in ./opt/anaconda3/lib/python3.8/site-packages (from pymc3>=3.9->pymc3_ext) (3.7.4.3) Requirement already satisfied: numpy>=1.15.0 in ./opt/anaconda3/lib/python3.8/site-packages (from pymc3>=3.9->pymc3_ext) (1.20.2) Requirement already satisfied: cachetools>=4.2.1 in ./opt/anaconda3/lib/python3.8/site-packages (from pymc3>=3.9->pymc3_ext) (4.2.1) Requirement already satisfied: fastprogress>=0.2.0 in ./opt/anaconda3/lib/python3.8/site-packages (from pymc3>=3.9->pymc3_ext) (1.0.0) Requirement already satisfied: dill in ./opt/anaconda3/lib/python3.8/site-packages (from pymc3>=3.9->pymc3_ext) (0.3.3) Requirement already satisfied: setuptools>=38.4 in ./opt/anaconda3/lib/python3.8/site-packages (from arviz>=0.11.0->pymc3>=3.9->pymc3_ext) (56.0.0) Requirement already satisfied: xarray>=0.16.1 in ./opt/anaconda3/lib/python3.8/site-packages (from arviz>=0.11.0->pymc3>=3.9->pymc3_ext) (0.17.0) Requirement already satisfied: netcdf4 in ./opt/anaconda3/lib/python3.8/site-packages (from arviz>=0.11.0->pymc3>=3.9->pymc3_ext) (1.5.6) Requirement already satisfied: matplotlib>=3.0 in ./opt/anaconda3/lib/python3.8/site-packages (from arviz>=0.11.0->pymc3>=3.9->pymc3_ext) (3.4.1) Requirement already satisfied: packaging in ./opt/anaconda3/lib/python3.8/site-packages (from arviz>=0.11.0->pymc3>=3.9->pymc3_ext) (20.4) Requirement already satisfied: six in ./opt/anaconda3/lib/python3.8/site-packages (from patsy>=0.5.1->pymc3>=3.9->pymc3_ext) (1.15.0) Requirement already satisfied: filelock in ./opt/anaconda3/lib/python3.8/site-packages (from theano-pymc==1.1.2->pymc3>=3.9->pymc3_ext) (3.0.12) Requirement already satisfied: python-dateutil>=2.7.3 in ./opt/anaconda3/lib/python3.8/site-packages (from pandas>=0.24.0->pymc3>=3.9->pymc3_ext) (2.8.1) Requirement already satisfied: pytz>=2017.3 in ./opt/anaconda3/lib/python3.8/site-packages (from pandas>=0.24.0->pymc3>=3.9->pymc3_ext) (2021.1) Requirement already satisfied: cftime in ./opt/anaconda3/lib/python3.8/site-packages (from netcdf4->arviz>=0.11.0->pymc3>=3.9->pymc3_ext) (1.4.1) Requirement already satisfied: pyparsing>=2.2.1 in ./opt/anaconda3/lib/python3.8/site-packages (from matplotlib>=3.0->arviz>=0.11.0->pymc3>=3.9->pymc3_ext) (2.4.7) Requirement already satisfied: cycler>=0.10 in ./opt/anaconda3/lib/python3.8/site-packages (from matplotlib>=3.0->arviz>=0.11.0->pymc3>=3.9->pymc3_ext) (0.10.0) Requirement already satisfied: pillow>=6.2.0 in ./opt/anaconda3/lib/python3.8/site-packages (from matplotlib>=3.0->arviz>=0.11.0->pymc3>=3.9->pymc3_ext) (8.2.0) Requirement already satisfied: kiwisolver>=1.0.1 in ./opt/anaconda3/lib/python3.8/site-packages (from matplotlib>=3.0->arviz>=0.11.0->pymc3>=3.9->pymc3_ext) (1.3.1) Installing collected packages: pymc3-ext Successfully installed pymc3-ext-0.1.0
Solved it.
Installation of Xcode and accepting the agreement fixed the problem:
multiple samples are projected to run over 19 hours: A quick intro to PyMC3 for exoplaneteers Radial velocity fitting
On iMac with 4 cores, runningMacOS Big Sur
Tried unsuccessfully:
import multiprocessing as mp mp.set_start_method("fork")
Smaller tune and/or draw parameters?