search

bstewart / stm

An R Package for the Structural Topic Model
Other
402 stars 98 forks source link

Error using topicQuality with quanteda dfm #124

Closed chris-billingham closed 7 years ago

chris-billingham commented 7 years ago

Hi,

I'm attempting to use topic quality with a model created with a quanteda dfm

topicQuality(model = gsr_l2_model, documents = gsr_dfm)

I get the following error: Error in as.vector(.Call(Csparse_to_vector, x), mode) : Cholmod error 'problem too large' at file ../Core/cholmod_dense.c, line 105

Some light googling suggested this was due to memory issues but even firing it into GCE with 600GB it wouldn't run. Am I doing it wrong?

Thanks

bstewart commented 7 years ago

At this stage quanteda plays nicely with the core stm function but not with the other functions. So just convert the dfm to the stm documents object (there is a conversion function in quanteda) and it should work.

No idea why it throws that particular error though!

chris-billingham commented 7 years ago

Ah that also explains the annoying difficulty I had getting fitNewDocuments to work as well. I shall use the code from that. Thanks and this is an awesome package!

bstewart commented 7 years ago

thanks for the kind words!

If you are doing fitNewDocuments make sure to check out the alignVocab function to be sure that your vocab is indexed in the same way for the new documents.