search

aksnzhy / xlearn

High performance, easy-to-use, and scalable machine learning (ML) package, including linear model (LR), factorization machines (FM), and field-aware factorization machines (FFM) for Python and CLI interface.
https://xlearn-doc.readthedocs.io/en/latest/index.html
Apache License 2.0
3.08k stars 518 forks source link

segment fault when use FFM with 1400 feature #317

Open sounds opened 4 years ago

sounds commented 4 years ago

training data num: 400000, feature num: 1400

same data, use FM is ok. but use FFM , segment fault when init. if Use data num 80000, and feature num 300, is ok.

its my config: ./xlearn_train ${pre}.${train_tag}.${suf} \ -s 2 \ -v ${pre}.${valid_tag}.${suf} \ -x auc \ -p adagrad \ -r 0.1 \ -b 0.0002 \ -k 4 \ -u 0.66 \ -e 100 \ -sw 10 \ -nthread 16 \

SO, how many num of feature that xlearn-ffm can support?

thestick613 commented 4 years ago

1400 is a lot, maybe there's a maximum number of features, such as 1024.

sounds commented 4 years ago

1400 is a lot, maybe there's a maximum number of features, such as 1024.

I tried 600 features（600 fields），segment fault too

lhdgriver commented 4 years ago

x fields means there are dim*x*x interactions, that's quite a lot of features. You'd better reduce your fields, or maybe you should use features other than fields