TypeError: evaluate_mean_squared_error_with_sporadic_fit() got an unexpected keyword argument 'e'

[jmrichardson]

Hi,

I am testing the examples and get the following error with this example code:

from timemachines.skatertools.tuning.hyperempirical import optimal_r_for_stream
from timemachines.skaters.proph.prophskaterssingular import fbprophet_univariate_r2
from humpday.optimizers.optunacube import optuna_tpe_cube
from timemachines.skaters.proph.prophparams import PROPHET_META, prophet_params
from pprint import pprint
from timemachines.skatertools.data.live import random_regular_stream_name

# Illustrates how to find the best hyper-parameter r in (0,1), and interpret this as two prophet hyper-parameters
# We use a random stream from https://www.microprediction.org/browse_streams.html
# Your should expect this to take many hours. A time update is provided after the first function evaluation.

if __name__=='__main__':
    name, url = random_regular_stream_name(min_len=PROPHET_META['n_warm'], with_url=True)
    print('We will find the best fbprophet hyper-parameters for '+url)
    print("Prophet will be fit for most of them, after a burn_in, and for many different hyper-params. Don't hold your breathe.")

    best_r, best_value, info = optimal_r_for_stream(f=fbprophet_univariate_r2,name=name,k=10,optimizer=optuna_tpe_cube,
                                                    n_burn=PROPHET_META['n_warm']+20,n_trials=50,n_dim=2)
    pprint(info)
    params = prophet_params(r=best_r,dim=2)
    pprint(params)

We will find the best fbprophet hyper-parameters for https://www.microprediction.org/stream_dashboard.html?stream=finance-futures-corn-change
Prophet will be fit for most of them, after a burn_in, and for many different hyper-params. Don't hold your breathe.
Traceback (most recent call last):
  File "D:\Anaconda3\envs\aim\lib\site-packages\IPython\core\interactiveshell.py", line 3444, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-9-3596626c8db7>", line 5, in <module>
    best_r, best_value, info = optimal_r_for_stream(f=fbprophet_univariate_r2,name=name,k=10,optimizer=optuna_tpe_cube,
  File "D:\Anaconda3\envs\aim\lib\site-packages\timemachines\skatertools\tuning\hyperempirical.py", line 36, in optimal_r_for_stream
    return optimal_r(f=f,y=y,k=k, a=None,t=t,e=None,evaluator=evaluator,optimizer=optimizer,n_trials=n_trials,
  File "D:\Anaconda3\envs\aim\lib\site-packages\timemachines\skatertools\tuning\hyper.py", line 54, in optimal_r
    a_test = objective(u=[0.5]*n_dim)  # Fail fast with easier trace
  File "D:\Anaconda3\envs\aim\lib\site-packages\timemachines\skatertools\tuning\hyper.py", line 50, in objective
    return evaluator(f=f, y=y, k=k, a=a, t=t, e=r, r=r, n_burn=n_burn)
TypeError: evaluate_mean_squared_error_with_sporadic_fit() got an unexpected keyword argument 'e'

[microprediction]

Thanks. I think I may have fixed that. Can you try

    pip install --ugprade timemachines 

and see if it still fails?

[microprediction]

Oh wait my bad. Yeah it shouldn't pass e anymore. I'll fix it

[microprediction]

Remind me where that code is?

[jmrichardson]

Thank you for the fast reply! :)

Here is the location:

https://github.com/microprediction/timemachines/tree/main/examples/tuning

[microprediction]

I've pushed a fix

[microprediction]

Try it now? New version should be on pypi in a few minutes or you can do

pip install --upgrade git+https://github.com/microprediction/timemachines.git

[microprediction]

By the way if you encounter more bugs feel free to let us know in the Slack, which I monitor better than issues https://join.slack.com/t/microprediction/shared_invite/zt-xauc5r03-XmIMdy~iL4a3Hw3G1Agv6w

[microprediction]

By the way (2) you could use tuneta to create better stacking of timeseries models I suspect, since we want a collection of those whose performance is uncorrelated with each other. (i.e. it might outperform the elo_skaters)

[jmrichardson]

It works now! Thanks for the fix!

Thank you for the tuneta suggestion, yes, it does make sense to include timeseries models as features. I am trying to educate myself on TS and have been using sktime. I found a reference to timemachines and wanted to try it out as it looks much more user friendly with a unified interface. I will join the slack and perhaps learn along the way. I am a little confused as to how to use in my particular use case and was hoping to get some guidance.