Implemented BERTopic Model for topic segmentation - Githubissues

Samagra-Development / ai-tools

AI Tooling to bootstrap applications fast

41 stars 109 forks source link

Implemented BERTopic Model for topic segmentation #316

Closed kartikbhtt7 closed 2 months ago

kartikbhtt7 commented 2 months ago

Fix for- https://github.com/Samagra-Development/ai-tools/issues/291

kartikbhtt7 commented 2 months ago

Key points(as discussed with @GautamR-Samagra):

This model needs a preprocessed .csv file as input with the preprocessing as provided in the below notebook.
The input .csv must only have one column named "text" containing all the queries/text.
Model still needs to go through a process of trial and error with hyperparameters to get rid of anomaly initial clusters

Notebook with preprocessing for the queries(these must be removed with a specific keyword): https://colab.research.google.com/drive/11aTHF9WFnpg4bb9GiZNNMWI5zNQCYBJv?usp=sharing