search

bnosac / udpipe

R package for Tokenization, Parts of Speech Tagging, Lemmatization and Dependency Parsing Based on the UDPipe Natural Language Processing Toolkit
https://bnosac.github.io/udpipe/en
Mozilla Public License 2.0
209 stars 33 forks source link

Perplexity calculation unavailable #62

Closed seonghobae closed 5 years ago

seonghobae commented 5 years ago

Hello, I can not get any perplexity calculation values from DTM object of udpipe library. Could you check this issues as soon as possible?

  library('udpipe')
  library('topicmodels')
  burnin = 1000
  iter = 1000
  keep = 50
  fitted <- topicmodels::LDA(dtm_clean, k = 2, method = "Gibbs",
                control = list(burnin = burnin, iter = iter, keep = keep) )
  perplexity(fitted, dtm_clean)
>   perplexity(fitted, dtm_clean)
Error in (function (classes, fdef, mtable)  : 
  unable to find an inherited method for function ‘perplexity’ for signature ‘"LDA_Gibbs", "dgCMatrix"’

Best, Seongho

jwijffels commented 5 years ago

udpipe does not provide a function called perplexity. That's a function from the topicmodels package. That package works with the slam package, so you need to use that. As in

library(slam)
perplexity(mymodel, newdata = as.simple_triplet_matrix(dtm_clean))