mhahsler / recommenderlab

recommenderlab - Lab for Developing and Testing Recommender Algorithms - R package
213 stars 61 forks source link

Implicit ALS Bug #60

Closed DanielRauser closed 1 year ago

DanielRauser commented 1 year ago

Hi again,

the evaluation schemes work great now, but still I'm facing issues with my implicit dataset. I always get the same error when I try to apply the iALS algorithm on my evaluation Scheme. I'm using the RRM with confidence values between 1 and 3. Matrix sparsity is around 99%, due to the sparse implicit feedback. The dimensions for the sample I use are: 651779 * 4694

I googled the error code and found this: https://stackoverflow.com/questions/58302449/what-does-the-cholmod-error-problem-too-large-means-exactly-problem-when-conv

So it propably has something to do with the number of columns. For a small sample with 100 entries it works. image I don't think it is a memory issue because the RAM-use doesn't spike up when I start evaluating.


algorithms_realRating <- list(
  "ALS"                                   = list(name = "ALS_implicit", param = NULL),
  "LIBMF"                                 = list(
    name = "LIBMF",
    param = list(
      dim = 20,
      costp_l2 = 0.1,
      costq_l2 = 0.01,
      nthread = 4,
      verbose = FALSE
    )
  ),
  "SVDF"                                  = list(name = "SVDF", param = NULL),
  "SVD"                                   = list(name = "SVD", param = NULL),
  "random items"                                = list(name = "RANDOM")
)

es_td_rrm<- evaluationScheme(
  rrm[1:600000],
  method = "cross-validation",
  train = 0.8,
  k = 3,
  given  = -1,
  goodRating = 1
)

results_ratings_rrm <- evaluate(es_td_rrm, 
                                    algorithms_realRating, 
                                    type  = "ratings"
                                    )

Thanks in advance for help. I have to admit I'm not a coding expert and just a bachelor student, which is the reason I have to rely on such great packages for my first hands-on experiences with Recommender Tasks.

Have a great day you all!

Edit: I tested on a different computer image So it propably is an RAM issue after all. So you can either close this issue or put it into enhancements, because it is propably possible for it to run without this sparse to dense matrix transition.

https://github.com/mhahsler/recommenderlab/files/9931355/RealRatingMatrix.zip image image