search

pytorch / torchrec

Pytorch domain library for recommendation systems
https://pytorch.org/torchrec/
BSD 3-Clause "New" or "Revised" License
1.93k stars 432 forks source link

is your code better performing than other recommender systems packages: just better recommendation/prediction quality #1398

Closed Sandy4321 closed 1 year ago

Sandy4321 commented 1 year ago

can you share you code is better performing than other recommender systems packages

not fancy NNs, but just better recommendation/prediction quality ?

gnahzg commented 1 year ago

Just want to clarify, (1) do you have specific package in mind to compare with TorchRec? (2) when you say no NNs, are you targeting some specific models? It looks like to me you want to some benchmark, right? Thanks

Sandy4321 commented 1 year ago

(1) do you have specific package in mind to compare with TorchRec? no (2) when you say no NNs, are you targeting some specific models? no It looks like to me you want to some benchmark, right? Thanks yes

colin2328 commented 1 year ago

Hi @Sandy4321 , For benchmarks, please see https://github.com/pytorch/torchrec/tree/main/benchmarks (for single device). You can also see ML perf (ML commons) benchmarks https://github.com/mlcommons/training/tree/master/recommendation_v2/torchrec_dlrm

Like is in our description, TorchRec focuses on performance and scale, but with a flexible and clean authoring experience (i.e pure pytorch). TorchRec is not a collection of specific recommendation models with good modeling metrics.

For competitors, feel free to check out hugeCTR (nvidia), TF recommenders, alibabas model zoo

Sandy4321 commented 1 year ago

1 https://github.com/mlcommons/inference measuring how fast systems can run models

THEN NOT FOR PERFORMANCE WHO NEEDS FAST MEDIOCRE PERFORMANCE ? THEN
https://github.com/mlcommons/inference/tree/master/recommendation/dlrm/pytorch USES CRITEO CRITEO IS NOT PURE RECOMENDER SYSTEMS DATA https://www.kaggle.com/competitions/criteo-display-ad-challenge predicting ad click-through rate (CTR) => NOT recommendations 2 hugeCTR https://nvidia-merlin.github.io/HugeCTR/main/performance.html the same story citeo data and not recommendations and also focused on speed 3 https://github.com/tensorflow/recommenders https://www.tensorflow.org/recommenders no good performance mentioned -> nothing to be proud off ? seems be yes 4 alibabas model zoo https://github.com/orgs/Alibaba-MIIL/repositories where is here performing recommenders? or do you mean https://modelzoo.co/ then which one is serious: https://modelzoo.co/model/qrec https://modelzoo.co/model/graphrec-pytorch https://modelzoo.co/model/recsys-pytorch https://modelzoo.co/model/models https://modelzoo.co/model/recvae https://modelzoo.co/model/improving-rnn-recommendation-model https://modelzoo.co/model/mxnet-audio https://modelzoo.co/model/cmn4recosys https://modelzoo.co/model/spotlight https://modelzoo.co/model/neural-attentive-session-based-recommendation-pytorch https://modelzoo.co/model/mxnet-recommendergluon https://modelzoo.co/model/collaborative_filtering

gnahzg commented 1 year ago

@Sandy4321, I guess when you refer to performance, you mean "recommendation models with good modeling metrics". This is not what TorchRec try to resolve. As @colin2328 has explained, "TorchRec focuses on performance (how fast systems can run models) and scale, but with a flexible and clean authoring experience (i.e pure pytorch)".

Sandy4321 commented 1 year ago

yes it is my questing what github repos to use to get good modeling metrics - less errors in recommendation ?

Sandy4321 commented 1 year ago

it is written in first post in this thread just better recommendation/prediction quality ? prediction quality!!!!!

henrylhtsang commented 1 year ago

@Sandy4321 you can take a look at DLRM repo, https://github.com/facebookresearch/dlrm