search

zenogantner / MyMediaLite

recommender system library for the CLR (.NET)
http://mymedialite.net
499 stars 192 forks source link

UserKNN failure (but ItemKNN runs) #477

Closed phsimon closed 5 years ago

phsimon commented 5 years ago

Hi,

After trying with success ItemKNN, I'd like to make an attempt with UserKNN with the same corpus, hereafter what I get:

item_recommendation --training-file=input_reco_001.csv --test-ratio=0,1 --recommender="UserKNN" loading_time 8.33 memory 225 training data: 781743 users, 1502 items, 10272808 events, sparsity 99.12511 test data: 358875 users, 1463 items, 1142557 events, sparsity 99.78238 UserKNN k=80 correlation=Cosine q=1 weighted=False alpha=0,5 (only for BidirectionalConditionalProbability)

An uncaught exception occured. Please send a bug report to zeno.gantner+mymedialite@gmail.com, or report the problem in our issue tracker: https://github.com/zenogantner/MyMediaLite/issues MyMediaLite 3.11

Value cannot be null. Parameter name: destinationArray at System.Array.Copy (System.Array sourceArray, System.Int32 sourceIndex, System.Array destinationArray, System.Int32 destinationIndex, System.Int32 length) [0x00017] in <8f2c484307284b51944a1a13a14c0266>:0 at System.Collections.Generic.List1[T].set_Capacity (System.Int32 value) [0x0003d] in <8f2c484307284b51944a1a13a14c0266>:0 at System.Collections.Generic.List1[T].EnsureCapacity (System.Int32 min) [0x00046] in <8f2c484307284b51944a1a13a14c0266>:0 at System.Collections.Generic.List1[T].Insert (System.Int32 index, T item) [0x00028] in <8f2c484307284b51944a1a13a14c0266>:0 at MyMediaLite.DataType.SparseMatrix1[T].set_Item (System.Int32 x, System.Int32 y, T value) [0x00095] in /home/zgantner/src/MyMediaLite/src/MyMediaLite/DataType/SparseMatrix.cs:84 at MyMediaLite.DataType.SymmetricSparseMatrix`1[T].set_Item (System.Int32 x, System.Int32 y, T value) [0x00015] in /home/zgantner/src/MyMediaLite/src/MyMediaLite/DataType/SymmetricSparseMatrix.cs:40 at MyMediaLite.Correlation.Overlap.ComputeUShort (MyMediaLite.DataType.IBooleanMatrix entity_data) [0x00044] in /home/zgantner/src/MyMediaLite/src/MyMediaLite/Correlation/Overlap.cs:103 at MyMediaLite.Correlation.BinaryDataSymmetricCorrelationMatrix.ComputeCorrelationsUShortOverlap (MyMediaLite.DataType.IBooleanMatrix entity_data) [0x00000] in /home/zgantner/src/MyMediaLite/src/MyMediaLite/Correlation/BinaryDataSymmetricCorrelationMatrix.cs:80 at MyMediaLite.Correlation.BinaryDataSymmetricCorrelationMatrix.ComputeCorrelations (MyMediaLite.DataType.IBooleanMatrix entity_data) [0x00063] in /home/zgantner/src/MyMediaLite/src/MyMediaLite/Correlation/BinaryDataSymmetricCorrelationMatrix.cs:55 at MyMediaLite.ItemRecommendation.KNN.Train () [0x00006] in /home/zgantner/src/MyMediaLite/src/MyMediaLite/ItemRecommendation/KNN.cs:120 at MyMediaLite.ItemRecommendation.UserKNN.Train () [0x00000] in /home/zgantner/src/MyMediaLite/src/MyMediaLite/ItemRecommendation/UserKNN.cs:38 at ItemRecommendation.Train () [0x00000] in /home/zgantner/src/MyMediaLite/src/Programs/ItemRecommendation/ItemRecommendation.cs:185 at ItemRecommendation.m__C () [0x00000] in /home/zgantner/src/MyMediaLite/src/Programs/ItemRecommendation/ItemRecommendation.cs:297 at MyMediaLite.Wrap.MeasureTime (System.Action t) [0x00006] in /home/zgantner/src/MyMediaLite/src/MyMediaLite/Wrap.cs:31 at ItemRecommendation.Run (System.String[] args) [0x0048a] in /home/zgantner/src/MyMediaLite/src/Programs/ItemRecommendation/ItemRecommendation.cs:297 at ItemRecommendation.Main (System.String[] args) [0x00006] in /home/zgantner/src/MyMediaLite/src/Programs/ItemRecommendation/ItemRecommendation.cs:151 Terminate on unhandled exception. Stacktrace:

at <0xffffffff> at (wrapper managed-to-native) System.Environment.Exit (int) <IL 0x00007, 0x00058> at MyMediaLite.Handlers.UnhandledExceptionHandler (object,System.UnhandledExceptionEventArgs) [0x0012a] in /home/zgantner/src/MyMediaLite/src/MyMediaLite/Handlers.cs:56 at (wrapper runtime-invoke) .runtime_invoke_void__this___object_object (object,intptr,intptr,intptr) <IL 0x00027, 0x00078>

Native stacktrace:

mono(+0xc8514) [0x55938056b514]
mono(+0x1217ce) [0x5593805c47ce]
mono(+0x3d7e3) [0x5593804e07e3]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x12890) [0x7efc43010890]
mono(mono_lock_free_alloc+0x391) [0x559380720c61]
mono(+0x238656) [0x5593806db656]
mono(+0x2375ba) [0x5593806da5ba]
mono(+0x23761f) [0x5593806da61f]
mono(+0x245ea5) [0x5593806e8ea5]
mono(+0x2326dd) [0x5593806d56dd]
mono(+0x23399f) [0x5593806d699f]
mono(+0x233da9) [0x5593806d6da9]
mono(+0x236625) [0x5593806d9625]
mono(+0x2369dd) [0x5593806d99dd]
mono(mono_domain_finalize+0x8c) [0x55938068836c]
mono(+0x3a5a4) [0x5593804dd5a4]
mono(+0x16875b) [0x55938060b75b]
[0x40f83639]

Debug info from gdb:

================================================================= Got a SIGSEGV while executing native code. This usually indicates a fatal error in the mono runtime or one of the native libraries used by your application.

/usr/local/bin/item_recommendation : ligne 7 : 23047 Abandon (core dumped) mono --debug $DIR/../lib/mymedialite/item_recommendation.exe "$@"

zenogantner commented 5 years ago

I have not looked at the stack trace in detail, but I am pretty sure MML cannot deal with so many users: 781,743, because it explicitly computes a square similarity matrix of this size. There are other packages out there that may support limiting to computing similarities only for the relevant users, but you need to look for them yourself (feel free to inform us about your findings ...).