deployment of the BERTopic model and monitoring

[MohamedMensi]

Hi, first massive congrats and a huge thank you for the library.

I managed to train bertopic on my specific data, and I'm very happy with the relevant results I got.

so I would like to know how to deploy my model reliably, and the most important thing is how to technically do the continuous monitoring (data drift and model drift/deviation), knowing that as performance metrics I use: Coherence score ( c_v , u_mass, npmi ), calinski harabasz score and davies bouldin score.

Thanks in advance.

[MaartenGr]

It is difficult to say as it depends on the underlying architecture, use case, etc. In practice, serialization is a good place to start with respect to inference. Things like data drift and model drift depends on your specific use case. Generally, you can start with checking whether the distribution of topics change over time.

[MohamedMensi]

Thanks for your quick response. @MaartenGr About the use case : i used BERTopic to cluster Incident tickets from the IT support team, so about the distribution of topics, in the first the model generated hundreds of topics, after human interpretation I did Outlier reduction and then Topic Reduction to finally obtain only 90 clusters, so for the moment and from here a few months we are going to work on this number of categories, but after a certain period of time we must detect if there is the appearance of new categories of incidents and at this time we must retrain the model... so what I wanted know is it possible to monitor bertopic always with the metrics I mentioned?

thanks very much!

[MaartenGr]

The metrics that you mention, like topic coherence, are not related to monitoring performance over time since they only evaluate the model as it was trained. Instead, it might be worthwhile to check whether the number of low-probability of outlier topics increases over time. That might be an indication that the distribution of topics is changing and that a new model could be trained.

[MohamedMensi]

Ok it's clear now, thank you.