Open marcomarinodev opened 2 months ago
Apologies for the late reply!
is this one the right approach to get the probabilities?
This is one of the approaches to indeed get probability-like scores.
most of the documents have almost the same (high) probability among all the topics. Does this mean the clustering didn't fit that much the data? What do you suggest?
What you get out of the .transform
are technically the similarity scores between the document and topic embeddings. Depending on the underlying embedding model, they may indeed be high but that shouldn't necessarily be a problem.
What you can do is apply softmax on these similarity scores to make sure they all sum to 1. You often get a much nicer distribution if you do that.
At some point, I want to create some additional parameters in .transform
and .fit_transform
that allow for choosing the type of probability/similarity score is being selected.
Hi @MaartenGr thanks for the reply.
So, as far as I understand, .fit_transform
is not supposed to compute the similarity scores between document and topic embeddings on zero-shot mode, right?
Apart from that I could be a candidate for implementing the point you mentioned :)
@marcomarinodev
So, as far as I understand, .fit_transform is not supposed to compute the similarity scores between document and topic embeddings on zero-shot mode, right?
During .fit_transform
there are two steps. The first is assigning the zero-shot topics to documents using the similarity scores between document and zero-shot topics. The second is running the default BERTopic pipeline over all documents that could not be assigned to zero-shot topics. Therefore, it is a combination of computing the similarity scores between document and topic embeddings and running HDBSCAN (or some other clustering algorithm).
Apart from that I could be a candidate for implementing the point you mentioned :)
That would be great! Note that this feature requires to check whether it is actually possible to either assign using topic embeddings or the HDBSCAN method before actually doing so. With this feature, I believe that the user experience should be the main concern as the code should be nothing more than selecting either technique.
Essentially, it would be something like this:
# Using cosine similarity between topic and document embedding
topics, probs = topic_model.transform(MY_DOCS, method="cosine_similarity")
# Using the clustering algorithm (although I'm not convinced of the name since
# it can also be a logistic regression or even a knowledge graph)
topics, probs = topic_model.transform(MY_DOCS, method="cluster_algorithm")
During
.fit_transform
there are two steps. The first is assigning the zero-shot topics to documents using the similarity scores between document and zero-shot topics. The second is running the default BERTopic pipeline over all documents that could not be assigned to zero-shot topics. Therefore, it is a combination of computing the similarity scores between document and topic embeddings and running HDBSCAN (or some other clustering algorithm).
All clear now regarding the zero shot steps.
That would be great! Note that this feature requires to check whether it is actually possible to either assign using topic embeddings or the HDBSCAN method before actually doing so. With this feature, I believe that the user experience should be the main concern as the code should be nothing more than selecting either technique.
What are your concerns there? Calling the transform with a different similarity score would just recompute the similarities document-topic with the new score metric right ?https://github.com/MaartenGr/BERTopic/blob/2353f4c21d74e33e34e30dbae938304bff094792/bertopic/_bertopic.py#L591
It indeed does nothing more than compute the similarity scores. It's just how you interface with either of these methods. Do you put the hyperparameter in the .transform function or even in the init for easier grid search (for those that use scikit-learn). What would be the name of the hyperparameter? How do you handle errors if you want HDBSCAN-based probabilities but it's not there? Things like that.
Have you searched existing issues? 🔎
Desribe the bug
Hi everyone, first of all I would like to thank @MaartenGr and all the contributors for this amazing project.
Recently I started looking at BERTopic as a method to classify some customer tickets into some categories defined into
zeroshot_topic_list
parameter. After fitting the model by callingfit_transform
my goal was to look for each document, the probability of that document of belonging to all the topics (both predefined and generated ones).probs
isNone
afterfit_transform
, as expected and mentioned at the end of this page https://maartengr.github.io/BERTopic/getting_started/zeroshot/zeroshot.html#example. Therefore, I then calledtransform
to get the probabilities.Now I've got two questions I would like to answer:
Again, I would like to thank you for your effort in advance and I look forward to contribute as well if needed.
Reproduction
The output is:
BERTopic Version
0.16.2