search

tjefferies / pymetalog

Public repo for the pymetalog project
MIT License
38 stars 3 forks source link

Bayesian Updating v0 #6

Closed tjefferies closed 4 years ago

tjefferies commented 4 years ago

Followed the rmetalog lead and implemented metalog Bayesian updating using the standard 3-term metalog and the Student-t distribution quantile function to perform the update and capture stats about what updated.

Functionally, pm.update function takes in a fitted metalog and the new_data to update the metalog distribution. See example below.

For the plots below, I updated PyMetalog_usagetest.py to first split the data into 90/10 and 5/95 sets. I then fit the metalog to each of the data split schemas by first fitting to set on left side of slash, then updating the fitted metalog object with the remainder of the data.

Plots on the left represent resultant metalog after fitting to set on left side of slash, plots on the right represent resultant metalog after updating fit to include all data.

Picture1

Pretty good fit on 90% of data, including 100% of the data smooths out some of the peaks to a lower density. This used L2 regularization in the update method.

Picture1

Pretty bad fit on 5% of data, including 100% of the data converges to expected fit. This used L2 regularization in the update method.

pm.update function allows for l2 regularization to be specified. Tbh I haven't completely wrapped my head around the metalog.output_dict['params']['bayes'] dict the update method returns.

colsmit commented 4 years ago

Looks good!

I also have trouble following the metalog.output_dict['params']['bayes'] dictionary, I'd love to see more documentation/references inline or otherwise at some point.