search

zalando / expan

Open-source Python library for statistical analysis of randomised control trials (A/B tests)
MIT License
334 stars 50 forks source link

why use stan instead of pymc3? #244

Open marcosmarxm opened 5 years ago

marcosmarxm commented 5 years ago

Hi folks! Great lib btw.

Is there any specific point for using stan? I think that pymc3 can perform the same calculations. And already have some implementations for Bayesian tests. (https://docs.pymc.io/notebooks/BEST.html)

Thanks!

shansfolder commented 5 years ago

hi @marcosmarxm, thanks for reaching out! No, there is no specific point of using stan. :)

We started with it three years ago. Back then our developer was most familiar with stan. And then, pymc3 and edward become more popular in the Bayesian community. We are not using the Bayesian functionality internally at the moment, so we haven't put into agenda to improve the implementation.

gbordyugov commented 5 years ago

Hi @marcosmarxm,

Somewhere in 2017, I performed some comparative test runs of stan vs pymc3, my main findings were: 1) stan seems to be a bit faster, up to 2x (which is a deal, but not a huge deal, I think) 2) pymc is a bit friendlier from the interface viewpoint - you can specify the model in Python instead of writing model files in stan's internal language. Since we don't use a lot of different models, this point is not quite acute for us.

Other than that, I couldn't find any huge differences between the two in the scope of typical expan jobs.

But I do agree, the current choice is mostly historical.