exoplanet-dev / exoplanet

Fast & scalable MCMC for all your exoplanet needs!
https://docs.exoplanet.codes
MIT License
206 stars 52 forks source link

Theano slowdown issue? #155

Closed arjunsavel closed 3 years ago

arjunsavel commented 3 years ago

Describe the bug

edit: updating to the newest exoplanet release removes the theano info logging, but performance of orbit.get_relative_velocity still isn't always near the ~200ms level.

A clear and concise description of what the bug is. I'm running orbit.get_relative_velocity numerous times as part of a sampling routine, and I occasionally find that theano.gof.compilelock logs some info ('Waiting for existing lock by unknown process' or 'Refreshing lock'). Separately, I find that the calculation is quite slow sometimes (on the order of 2 minutes for the former flag, 10s for the latter) and quite fast other times (on the order of 200 ms). I definitely could be missing something obvious, but I'm wondering if there's some deep Theano-level information I'm unaware of.

To Reproduce This section should include a simple, standalone code snippet that demonstrates the issue.

orbit = KeplerianOrbit(
    m_star=1,
    r_star=1.756, # solar radii!
    t0=0, # reference transit at 0!
    period=3,
    ecc=0,
    b = 0.027,
    omega=np.radians(peri) , # periastron, radians
    Omega=0,
    m_planet=0.894 * 0.0009543,
)

vel = np.array(theano.function([], orbit.get_relative_velocity(t))())

Expected behavior A clear and concise description of what you expected to happen.

I think that orbit.get_relative_velocity should be consistently pretty quick — but I'd gladly take any pointers if there's a quicker way to do what I'm trying to do!

Your setup (please complete the following information):

Additional context Add any other context about the problem here.

rodluger commented 3 years ago

Are you running the line

vel = np.array(theano.function([], orbit.get_relative_velocity(t))())

multiple times? If so, theano has to recompile the function every time, which is a huge speed hit (even with caching). You should do

f = theano.function(inputs, orbit.get_relative_velocity(...))

to compile the function a single time, then call the compiled function as many times as you need on different values of "inputs".

On Fri, Mar 26, 2021, 4:49 PM Arjun Savel @.***> wrote:

Describe the bug A clear and concise description of what the bug is. I'm running orbit.get_relative_velocity numerous times as part of a sampling routine, and I occasionally find that theano.gof.compilelock logs some info ('Waiting for existing lock by unknown process' or 'Refreshing lock'). Separately, I find that the calculation is quite slow sometimes (on the order of 2 minutes for the former flag, 10s for the latter) and quite fast other times (on the order of 200 ms). I definitely could be missing something obvious, but I'm wondering if there's some deep Theano-level information I'm unaware of.

To Reproduce This section should include a simple, standalone code snippet that demonstrates the issue.

orbit = KeplerianOrbit(

m_star=1,

r_star=1.756, # solar radii!

t0=0, # reference transit at 0!

period=3,

ecc=0,

b = 0.027,

omega=np.radians(peri) , # periastron, radians

Omega=0,

m_planet=0.894 * 0.0009543,

)

vel = np.array(theano.function([], orbit.get_relative_velocity(t))())

Expected behavior A clear and concise description of what you expected to happen.

I think that orbit.get_relative_velocity should be consistently pretty quick — but I'd gladly take any pointers if there's a quicker way to do what I'm trying to do!

Your setup (please complete the following information):

  • Version of exoplanet: 0.3.3
  • Operating system: Linux
  • Python version & installation method (pip, conda, etc.): conda install

Additional context Add any other context about the problem here.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/exoplanet-dev/exoplanet/issues/155, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACHEKK7ESSWZSCRUHLWHTC3TFTXOBANCNFSM4Z4CJ7YA .

dfm commented 3 years ago

Rodrigo's comment is exactly what I would suggest!

Can you say some more about your sampling routine? Are you (a) using PyMC3, or are you (b) using exoplanet to define a model and then calling it directly from another Python library?

import theano.tensor as tt

# Let's say that we're just fitting for the period and eccentricity, for example
period = tt.dscalar()  # It's a float64 scalar
ecc = tt.dscalar()

orbit = KeplerianOrbit(
    period=period,
    ecc=ecc,
    # all the other arguments would be the same, if you want them to be fixed
)

vel = orbit.get_relative_velocity(t)

vel_func = theano.function([period, ecc], vel)

# Then in your sampling routine
period_val = # some number from your sampler
ecc_val = # some number from your sampler

vel_val = vel_func(period_val, ecc_val)

This will then be fast and not run into compiler collisions.

arjunsavel commented 3 years ago

Fantastic! Thank you both — this is exactly the type of problem I thought I was running into.

I was running the line containing theano.function as I defined a model at every step in an emcee sampling routine, which I now see is not the way to go about this. Thanks again!