Tutorial - steps on how to verify/validate the churn probabilities

[SSMK-wq]

Thanks for this useful package and incorporating some useful functions.

Currently, we are exploring this package for deriving some business insights on customers.

While the expected purchase count and expected average revenue can be verified using a typical sklearn metrics such as MSE, RMSE, am unable to implement how to use arviz for verifying the probabilities of churn. Mainly because, am more of applied data scientist. So, unable to use the arviz package as is for our problem (of verifying churn probability - probability_alive and probability_alive_upto_time_t). I did refer the post here - https://github.com/ColtAllen/btyd/pull/33

But am not sure how I can do it in a simple intuitive manner for typical sklearn scientists

Is there any simple tutorial that you can share on how to validate and interpret the results? would really be helpful

[ColtAllen]

Hey @SSMK-wq,

Customer churn is unobservable, so the only realistic validation approach is to set up a monitoring function to periodically review model predictions and verify if a customer has made a purchase or not. If performance falls below a set threshold, trigger a retraining.

As for incorporating arviz functionality, this is a good time to say I've decided to transition my efforts to the https://github.com/pymc-labs/pymc-marketing library. BTYD has been mainly a solo project on my part, and a day will inevitably come when I'm no longer able to develop & maintain it. An entire community is working on pymc-marketing to ensure its success, and although it's still in early development, in time it will include all of this library's functionality and more.

[SSMK-wq]

Oh okay. I really appreciate your efforts with this btyd package and helping me with the queries.

Sorry, that I could not help with code because am more of scikit-learn fit and predict kind of analysis guy. I don't really know the Oops concept to write heavy functions etc (and also because I don't know the intricate details of algorithms).

If I can be of any other help, please let me know.

So, I think that btyd python package will no more be maintained (or developed). Understood that right?

On Mon, Dec 12, 2022, 11:44 AM Colt Allen @.***> wrote:

Hey @SSMK-wq https://github.com/SSMK-wq,

Customer churn is unobservable, so the only realistic validation approach is to set up a monitoring function to periodically review model predictions and verify if a customer has made a purchase or not. If performance falls below a set threshold, trigger a retraining.

As for incorporating arviz functionality, this is a good time to say I've decided to transition my efforts to the https://github.com/pymc-labs/pymc-marketing library. BTYD has been mainly a solo project on my part, and a day will inevitably come when I'm no longer able to develop & maintain it. An entire community is working on pymc-marketing to ensure its success, and although it's still in early development, in time it will include all of this library's functionality and more.

— Reply to this email directly, view it on GitHub https://github.com/ColtAllen/btyd/issues/83#issuecomment-1345826814, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHKM54P6ZQ5LEOE3NJ264CDWM2NQDANCNFSM6AAAAAASXZVMCM . You are receiving this because you were mentioned.Message ID: @.***>

[michaelwexler]

@ColtAllen This is a big announcement to be buried in a comment thread. Since you know this codebase better than anyone else, will you be able to guide "porting" so many of these improvements over to pymc-marketing?

Your work here has really transformed the utility of Cam's project, which was dormant for so long. It's really great stuff, and I hope we don't have to wait too long to see pymc-marketing reflect the fantastic work you've guided and executed here in btyd.

[ColtAllen]

Thanks @SSMK-wq and @michaelwexler,

I'll be updating the README.md file soon and also making an announcement in the lifetimes repo about this.

Many of the improvements in btyd are being added as we speak to pymc-marketing by the same developers of pymc - the backend library for btyd. My efforts at this time are on porting over the utility functions of lifetimes, and advising on modeling concepts as I'm probably the most familiar with the research.