search

MIND-Lab / OCTIS

OCTIS: Comparing Topic Models is Simple! A python package to optimize and evaluate topic models (accepted at EACL2021 demo track)
MIT License
718 stars 102 forks source link

how to train a model using the whole corpus (ignoring partitions) #34

Closed cayaluke closed 2 years ago

cayaluke commented 2 years ago

Dear Silvia,

Love your work!

I have a silly question. How do you train a model using the entire corpus (ignoring partitions)?

Thank you for your help.

Luke

silviatti commented 2 years ago

Hello! Thank you for using OCTIS :)

Not a silly question because I've realized there's something to fix in the code and in the documentation :) Every model has a method called "partitioning". If you pass "False" then the whole dataset will be used as training, the dataset will be split (as the default). You must call this method before training or optimization. For example:

model = ETM(num_topics=25) 
model.partitioning(False)
model_output = model.train_model(dataset)

However, some models have also the parameter "use_partitions" in the initialization which can be set to "False" to obtain the same effect. I think this is probably the easiest way to do it, but not all models have this parameter (it seems everyone has it except LSI and LDA). I think I'm going to fix this for the next release. See the example below:

model = ETM(num_topics=25, use_partitions=False) 
model_output = model.train_model(dataset)

Thank you for your patience.

Silvia

cayaluke commented 2 years ago

Thank you for your response.

It worked.

Grazie mille!