PolMine / biglda

Tools for fast LDA topic modelling for big corpora
5 stars 1 forks source link

vector space exhausted with `topicmodels::get_terms()`: use `apply` #16

Open ablaette opened 1 year ago

ablaette commented 1 year ago

With a large topic model, I get a vector space exhausted error when trying to use get_terms(). Internally, there is some unnecessary copying of the data.

This snippet does the job - maybe turn it into a function.

foo <- apply(lda@beta, 1, function(row) lda@terms[tail(order(row), 10)])
ablaette commented 6 months ago

One year later same solution:

dim(lda@beta)

terms <- apply(
  lda@beta, 1,
  function(row) lda@terms[head(order(row, decreasing = TRUE), 25)]
)