This is an implementation of the NARM (Neural Attentive Session-based Recommendation) model. The model is one of the earliest proposed neural model (another one is GRU4Rec) in the line of SBR research. NARM is often used as a baseline in the later research papers.
Preprocessing pipeline for SBR benchmark datasets DIGINETICA, YOOCHOOSE is added with a new file examples/matching/data/session_based/preprocess_session_based.py. The pipeline processes the data in the same way as described in the NARM paper. The statistics of output datasets agree with the ones described in the earlier Pytorch implementation of NARM. However, the current implementation is mainly based on pandas thus is faster and easier to understand. This pipeline can be recycled later for processing other SBR benchmark datasets.
Example datasets from both DIGINETICA and YOOCHOOSE are also added under the folder examples/matching/data/session_based.
Example script that evaluates NARM model is added together with a new file examples/matching/run_sbr.py. The hyper parameters are set according to section 4.3.2 in the NARM paper. This file could also be recycled later for evaluating other SBR models.
Results
Recall@20
MRR@20
YOOCHOOSE1/64
0.6746
0.2827
YOOCHOOSE1/4
0.7028
0.2909
DIGINETICA
0.5829
0.2603
These results are comparable to the results reported in the original paper, with DIGINETICA 0.04 off on Recall@20. This might be caused by imprecise description of the data preprocessing process in the original paper as well as its native implementation, i.e., the datasets statistics could not be reproduced by the later python implementation and the current implementation while the later two agree with each other. Nonetheless, there is still room for further improvement by tuning e.g. max epochs on a validation set.
Note there is an issue fired on the earlier pytorch implementation of NARM that the Recall@20 (reported 0.5309) on DIGINETICA is ~0.08 off from the paper (reported 0.6258). This might well be due to that the normalization of the attention weights (in the Theano version) was not implemented in the earlier pytorch version (nor was described in the original paper to be fair). Under this consideration, the normalization is also added in the current implementation.
Future work
Tune Hyperparameter on a validation set. This could potentially improve the quantitative metrics.
Evaluate GRU4Rec under SBR context. With the infrastructure just introduced, this should be fairly easy. Though I noticed the GRU4Rec in the repo deviates a bit from the original implementation, making it compatible with the SBR evaluation pipeline should also be fairly easy I presume.
This is an implementation of the NARM (Neural Attentive Session-based Recommendation) model. The model is one of the earliest proposed neural model (another one is GRU4Rec) in the line of SBR research. NARM is often used as a baseline in the later research papers.
Changes
models/matching/narm.py. A new line that imports the model is added inmodels/matching/__init__.py. The implementation follows the CIKM'17 paper Neural Attentive Session-based Recommendation, its original Thenao implementation, as well as a Pytorch implementation.examples/matching/data/session_based/preprocess_session_based.py. The pipeline processes the data in the same way as described in the NARM paper. The statistics of output datasets agree with the ones described in the earlier Pytorch implementation of NARM. However, the current implementation is mainly based on pandas thus is faster and easier to understand. This pipeline can be recycled later for processing other SBR benchmark datasets.examples/matching/data/session_based.examples/matching/run_sbr.py. The hyper parameters are set according to section 4.3.2 in the NARM paper. This file could also be recycled later for evaluating other SBR models.Results
Future work