search

RUCAIBox / RecBole

A unified, comprehensive and efficient recommendation library
https://recbole.io/
MIT License
3.43k stars 614 forks source link

[🐛BUG] 在train_neg_sample_args 设置了sample_num为4，为什么在训练时只得到一个负样本 #1746

Closed lalunex closed 1 year ago

lalunex commented 1 year ago

您好，我有一个小疑问，我在配置中设置了sample_num为4，为什么在训练时，在interaction里面却是一个正样本对于一个负样本呢？具体情况如下图： image

下面是我的项目的全部配置

23 Apr 12:36    INFO  
General Hyper Parameters:
gpu_id = 0
use_gpu = True
seed = 2023
state = INFO
reproducibility = True
data_path = E:\anaconda3\envs\ll_torch10\Lib\site-packages\recbole\config\../dataset_example/ml-100k
checkpoint_dir = saved
show_progress = True
save_dataset = False
dataset_save_path = None
save_dataloaders = False
dataloaders_save_path = None
log_wandb = False
Training Hyper Parameters:
epochs = 300
train_batch_size = 1024
learner = adam
learning_rate = 1e-05
train_neg_sample_args = {'distribution': 'uniform', 'sample_num': 4, 'alpha': 1.0, 'dynamic': False, 'candidate_num': 0}
eval_step = 1
stopping_step = 10
clip_grad_norm = {'max_norm': 5.0, 'norm_type': 2}
weight_decay = 0.0001
loss_decimal_place = 4
Evaluation Hyper Parameters:
eval_args = {'group_by': 'user', 'order': 'RO', 'split': {'RS': [0.8, 0.1, 0.1]}, 'mode': 'full'}
repeatable = False
metrics = ['Hit', 'NDCG', 'MRR']
topk = [1, 5, 10, 20]
valid_metric = Hit@20
valid_metric_bigger = True
eval_batch_size = 4096
metric_decimal_place = 4
Dataset Hyper Parameters:
field_separator =   
seq_separator =  
USER_ID_FIELD = user_id
ITEM_ID_FIELD = item_id
RATING_FIELD = rating
TIME_FIELD = timestamp
seq_len = None
LABEL_FIELD = label
threshold = None
NEG_PREFIX = neg_
load_col = {'inter': ['user_id', 'item_id', 'rating', 'timestamp']}
unload_col = None
unused_col = None
additional_feat_suffix = None
rm_dup_inter = None
val_interval = None
filter_inter_by_user_or_item = True
user_inter_num_interval = [5, inf)
item_inter_num_interval = [5, inf)
alias_of_user_id = None
alias_of_item_id = None
alias_of_entity_id = None
alias_of_relation_id = None
preload_weight = None
normalize_field = None
normalize_all = True
ITEM_LIST_LENGTH_FIELD = item_length
LIST_SUFFIX = _list
MAX_ITEM_LIST_LENGTH = 50
POSITION_FIELD = position_id
HEAD_ENTITY_ID_FIELD = head_id
TAIL_ENTITY_ID_FIELD = tail_id
RELATION_ID_FIELD = relation_id
ENTITY_ID_FIELD = entity_id
kg_reverse_r = False
entity_kg_num_interval = None
relation_kg_num_interval = None
benchmark_filename = None
Other Hyper Parameters: 
worker = 0
wandb_project = recbole
shuffle = False
require_pow = False
enable_amp = False
enable_scaler = False
transform = None
numerical_features = []
discretization = None
MODEL_TYPE = ModelType.GENERAL
scheduler = False
warm_up_ratio = 0.1
initializer_range = 0.02
loss_type = BPR
hidden_size = 512
embedding_size = 512
pre_train_model_dict = {'amazon-sports-outdoors': 'saved/GRU4Rec-May-17-2022_13-14-44.pth', 'ml-100k': 'saved/GRU4Rec-May-17-2022_13-13-42.pth', 'yelp': 'saved/GRU4Rec-May-17-2022_13-15-01.pth', 'amazon-beauty': 'saved/GRU4Rec-May-17-2022_13-13-26.pth', 'amazon-clothing-shoes-jewelry': 'saved/GRU4Rec-May-17-2022_13-32-16.pth', 'ml-1m': 'saved/GRU4Rec-May-17-2022_13-13-41.pth'}
model_name = HSD
verbose = True
is_gumbel_tau_anneal = True
gumbel_temperature = 0.1
is_spu_cl_tau_anneal = True
supervised_contrastive_temperature = 0.7
our_att_drop_out = 0.3
our_ae_drop_out = 0.3
load_pre_train_emb = False
gcn_layers = 2
item_hg_layers = 2
user_hg_layers = 2
item_category_num = 50
item_category_extract_t = 5
user_interest_num = 30
user_interest_extract_k = 5
neg_number = 4
curriculum_learn_epoch = 15
reweight_loss_alpha = 0.1
reweight_loss_lambda = 9.0
MODEL_INPUT_TYPE = InputType.PAIRWISE
eval_type = EvaluatorType.RANKING
single_spec = True
local_rank = 0
device = cuda
eval_neg_sample_args = {'distribution': 'uniform', 'sample_num': 'none'}
SakuraXiaMF commented 1 year ago

请问一下，要怎么设置负样本个数啊~我按照下面的方法设置，一设置就出问题

`## model config

embedding_size: 32

dataset config

MovieLens, Amazon

field_separator: "\t" #指定数据集field的分隔符 seq_separator: " " #指定数据集中token_seq或者float_seq域里的分隔符 USER_ID_FIELD: user_id #指定用户id域 ITEM_ID_FIELD: item_id #指定物品id域 RATING_FIELD: rating #指定打分rating域 TIME_FIELD: timestamp #指定时间域

指定从什么文件里读什么列，这里就是从ml-1m.inter里面读取user_id, item_id, rating, timestamp这四列,剩下的以此类推

load_col: inter: [user_id, item_id, rating, timestamp]

NEGPREFIX: neg #指定负采样前缀 LABEL_FIELD: label #指定标签域 ITEM_LIST_LENGTH_FIELD: item_length #指定序列长度域 LIST_SUFFIX: _list #指定序列前缀 MAX_ITEM_LIST_LENGTH: 100 #指定最大序列长度 POSITION_FIELD: position_id #指定生成的序列位置id

max_user_inter_num: 100

min_user_inter_num: 5

max_item_inter_num: 100

min_item_inter_num: 5

lowest_val:

timestamp: 1546264800

highest_val:

timestamp: 1577714400

training settings

epochs: 50 #训练的最大轮数 train_batch_size: 256 #训练的batch_size learner: adam #使用的pytorch内置优化器 learning_rate: 0.001 #学习率

train_neg_sample_args: # (dict) Negative sampling configuration for model training.

distribution: uniform # (str) The distribution of negative items.

sample_num: 1 # (int) The sampled num of negative items.

alpha: 1.0 # (float) The power of sampling probability for popularity distribution.

dynamic: False # (bool) Whether to use dynamic negative sampling.

candidate_num: 0 # (int) The number of candidate negative items when dynamic negative sampling.

training_neg_sample_num: 1 #0 负采样数目

train_neg_sample_args = {'distribution': 'uniform', 'sample_num': 1, 'alpha': 1.0, 'dynamic': False, 'candidate_num': 0 } eval_step: 1 #每次训练后做evalaution的次数 stopping_step: 10 #控制训练收敛的步骤数，在该步骤数内若选取的评测标准没有什么变化，就可以提前停止了

evalution settings

eval_setting: TO_LS,full #对数据按时间排序，设置留一法划分数据集，并使用全排序 metrics: ["Recall", "MRR","NDCG","Precision"] #评测标准 valid_metric: MRR@10 #选取哪个评测标准作为作为提前停止训练的标准 eval_batch_size: 256 #评测的batch_size weight_decay: 0 topk: [5, 10, 20, 50]

directory setting

log_root: "./log/" data_path: "/home/xtx/RecBole/dataset/"

lmd: 0.1 lmd_sem: 0.1

tau: 1

choose from {un, su, us, us_x}

contrast: 'us_x'

choose from {dot, cos}

sim: 'dot'

hidden_dropout_prob: 0.5 attn_dropout_prob: 0.5`

Paitesanshi commented 1 year ago

@lalunex 您好，请问运行的是什么模型，截图是在训练阶段还是测试阶段，具体是哪个位置

Paitesanshi commented 1 year ago

@SakuraXiaMF 您好，请问您指的是什么问题，是否有报错信息

lalunex commented 1 year ago

@Paitesanshi 您好，截图是在训练阶段（具体在_train_epoch函数里面的for循环里面，如下图）。这个是我自定义的模型，只是使用了BPR loss，但是原本的BPR loss是pairwise的，我想把他改成listwise，所以设置了负样本为4个，但是bole只返回了一个负样本 image

Paitesanshi commented 1 year ago

@lalunex 您可以设置模型的InputType为LISTWISE,pairwise只会返回一个负样本

lalunex commented 1 year ago

@Paitesanshi 您好，我修改了之后弹出以下错误

!!!Note: hyper-parameter analysis, embedding size:[100]
26 Apr 12:53    INFO  
General Hyper Parameters:
gpu_id = 0
use_gpu = True
seed = 2023
state = INFO
reproducibility = True
data_path = dataset/amazon-beauty
checkpoint_dir = saved
show_progress = True
save_dataset = False
dataset_save_path = None
save_dataloaders = False
dataloaders_save_path = None
log_wandb = False

Training Hyper Parameters:
epochs = 300
train_batch_size = 1024
learner = adam
learning_rate = 0.0001
train_neg_sample_args = {'distribution': 'uniform', 'sample_num': 1, 'alpha': 1.0, 'dynamic': False, 'candidate_num': 0}
eval_step = 1
stopping_step = 10
clip_grad_norm = {'max_norm': 5.0, 'norm_type': 2}
weight_decay = 0.0001
loss_decimal_place = 4

Evaluation Hyper Parameters:
eval_args = {'group_by': 'user', 'order': 'RO', 'split': {'RS': [0.8, 0.1, 0.1]}, 'mode': 'full'}
repeatable = False
metrics = ['Hit', 'NDCG', 'MRR']
topk = [1, 5, 10, 20]
valid_metric = Hit@20
valid_metric_bigger = True
eval_batch_size = 4096
metric_decimal_place = 4

Dataset Hyper Parameters:
field_separator =   
seq_separator =  
USER_ID_FIELD = user_id
ITEM_ID_FIELD = item_id
RATING_FIELD = rating
TIME_FIELD = timestamp
seq_len = None
LABEL_FIELD = label
threshold = None
NEG_PREFIX = neg_
load_col = {'inter': ['user_id', 'item_id', 'rating', 'timestamp']}
unload_col = None
unused_col = None
additional_feat_suffix = None
rm_dup_inter = None
val_interval = None
filter_inter_by_user_or_item = True
user_inter_num_interval = [5, inf)
item_inter_num_interval = [5, inf)
alias_of_user_id = None
alias_of_item_id = None
alias_of_entity_id = None
alias_of_relation_id = None
preload_weight = None
normalize_field = None
normalize_all = None
ITEM_LIST_LENGTH_FIELD = item_length
LIST_SUFFIX = _list
MAX_ITEM_LIST_LENGTH = 50
POSITION_FIELD = position_id
HEAD_ENTITY_ID_FIELD = head_id
TAIL_ENTITY_ID_FIELD = tail_id
RELATION_ID_FIELD = relation_id
ENTITY_ID_FIELD = entity_id
benchmark_filename = None

Other Hyper Parameters: 
worker = 0
wandb_project = recbole
shuffle = False
require_pow = False
enable_amp = False
enable_scaler = False
transform = None
numerical_features = []
discretization = None
kg_reverse_r = False
entity_kg_num_interval = [0,inf)
relation_kg_num_interval = [0,inf)
MODEL_TYPE = ModelType.GENERAL
scheduler = False
warm_up_ratio = 0.1
initializer_range = 0.02
loss_type = BPR
hidden_size = 100
embedding_size = 100
pre_train_model_dict = {'amazon-sports-outdoors': 'saved/GRU4Rec-May-17-2022_13-14-44.pth', 'ml-100k': 'saved/GRU4Rec-May-17-2022_13-13-42.pth', 'yelp': 'saved/GRU4Rec-May-17-2022_13-15-01.pth', 'amazon-beauty': 'saved/GRU4Rec-May-17-2022_13-13-26.pth', 'amazon-clothing-shoes-jewelry': 'saved/GRU4Rec-May-17-2022_13-32-16.pth', 'ml-1m': 'saved/GRU4Rec-May-17-2022_13-13-41.pth'}
model_name = HSD
verbose = False
is_gumbel_tau_anneal = True
gumbel_temperature = 0.1
is_spu_cl_tau_anneal = True
supervised_contrastive_temperature = 0.5
our_ae_drop_out = 0.3
load_pre_train_emb = False
gcn_layers = 2
item_hg_layers = 3
user_hg_layers = 1
item_category_num = 80
item_category_extract_t = 15
user_interest_num = 20
user_interest_extract_k = 10
neg_number = 1
curriculum_learn_epoch = 15
reweight_loss_alpha = 0.3
reweight_loss_lambda = 7.0
MODEL_INPUT_TYPE = InputType.LISTWISE
eval_type = EvaluatorType.RANKING
single_spec = True
local_rank = 0
device = cuda
eval_neg_sample_args = {'distribution': 'uniform', 'sample_num': 'none'}

26 Apr 12:54    INFO  amazon-beauty
The number of users: 22364
Average actions of users: 8.876358270357287
The number of items: 12102
Average actions of items: 16.403768283612923
The number of inters: 198502
The sparsity of the dataset: 99.92665707018277%
Remain Fields: ['user_id', 'item_id', 'rating', 'timestamp']
Traceback (most recent call last):
  File "E:\pycharm-professional-2023.1\PyCharm 2023.1\plugins\python\helpers\pydev\pydevd.py", line 1496, in _exec
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "E:\pycharm-professional-2023.1\PyCharm 2023.1\plugins\python\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "E:\chrome_download\interests\run_hsd.py", line 144, in <module>
    main(model=dic['model_name'],
  File "E:\chrome_download\interests\run_hsd.py", line 45, in main
    train_data, valid_data, test_data = data_preparation(config, dataset)
  File "E:\anaconda3\envs\ll_torch10\lib\site-packages\recbole\data\utils.py", line 171, in data_preparation
    train_data = get_dataloader(config, "train")(
  File "E:\anaconda3\envs\ll_torch10\lib\site-packages\recbole\data\dataloader\general_dataloader.py", line 41, in __init__
    self._set_neg_sample_args(
  File "E:\anaconda3\envs\ll_torch10\lib\site-packages\recbole\data\dataloader\abstract_dataloader.py", line 168, in _set_neg_sample_args
    raise ValueError(
ValueError: `neg sampling by` with dl_format [InputType.LISTWISE] not been implemented.
python-BaseException
Paitesanshi commented 1 year ago

@lalunex 您好，我理解BPRloss只能用于pairwise计算，不清楚您想要的Listwise的BPRloss具体如何实现，伯乐目前不支持类似的功能，可能需要您对代码进行较大程度的修改，自行实现这一功能

lalunex commented 1 year ago

@Paitesanshi 好的，谢谢