search

FlagOpen / FlagEmbedding

Retrieval and Retrieval-augmented LLMs
MIT License
7.56k stars 549 forks source link

微调bge-reranker-v2-m3时loss一直变大 #891

Open LinXin04 opened 4 months ago

LinXin04 commented 4 months ago

配置： !torchrun --nproc_per_node 1 \ -m FlagEmbedding.reranker.run \ --output_dir /bge-reranker-v2-m3-finetune \ --model_name_or_path /bge-reranker-v2-m3/bge-reranker-v2-m3 \ --train_data output.jsonl \ --learning_rate 6e-5 \ --fp16 \ --num_train_epochs 20 \ --per_device_train_batch_size 6 \ --gradient_accumulation_steps 4 \ --dataloader_drop_last True \ --train_group_size 8 \ --max_len 512 \ --weight_decay 0.01 \ --logging_steps 10 样本： {"query":"Dockers Singapore","pos":["DOCKERS"],"neg":["Cosas United","Foot Efx","Lobby Salons","KEDAIBISKUTSINGUANHOE","SMART RING","Coconut cafe&bar","D'CLUB"]} {"query":"Da Paolo HQ","pos":["Da Paolo Group Pte. Ltd."],"neg":["LokLok串","KEKHOENTERPRISE","Hong Kong KooL Chicken Pot","TEOJIUNNKHIM","贝拉维拉都市酒店","Mynews. Com","MJC Nails Studio"]} {"query":"39 Vietnam Cuisine","pos":["Vietnam Cuisine"],"neg":["Unicity Marketing Sdn. Bhd.","Zorba The Greek Taverna","25, Jalan Datuk Sulaiman 2","Wandering Star Tour","Allgood Gourmet","World of Leonardo da Vinci","Qs Empire Sdn Bhd"]} {"query":"Ah Ma Dessert Chendol & Specialties","pos":["Ah Ma Dessert"],"neg":["GoroGoro Steamboat & Korean Buffet","Strandstraße 35\/2087","International Trading Srl","Nyky salon","MONNALISA","BLUE DIAMOND MOBILE","Pasaya Store"]}

日志：

{'loss': 0.0093, 'grad_norm': 0.01756790652871132, 'learning_rate': 5.675505050505051e-05, 'epoch': 1.09} {'loss': 0.005, 'grad_norm': 0.13430514931678772, 'learning_rate': 5.672979797979798e-05, 'epoch': 1.09} {'loss': 0.0176, 'grad_norm': 0.05022358149290085, 'learning_rate': 5.6704545454545454e-05, 'epoch': 1.1} {'loss': 0.0272, 'grad_norm': 0.19523026049137115, 'learning_rate': 5.667929292929293e-05, 'epoch': 1.11} {'loss': 0.0145, 'grad_norm': 0.006024870090186596, 'learning_rate': 5.66540404040404e-05, 'epoch': 1.12} {'loss': 0.0433, 'grad_norm': 0.0005314166191965342, 'learning_rate': 5.662878787878788e-05, 'epoch': 1.13} {'loss': 0.0463, 'grad_norm': 0.4784902334213257, 'learning_rate': 5.6603535353535354e-05, 'epoch': 1.14} {'loss': 0.0001, 'grad_norm': 0.14928492903709412, 'learning_rate': 5.6578282828282834e-05, 'epoch': 1.14} {'loss': 0.0421, 'grad_norm': 1.8677421808242798, 'learning_rate': 5.655303030303031e-05, 'epoch': 1.15} {'loss': 0.0283, 'grad_norm': 35.705631256103516, 'learning_rate': 5.652777777777778e-05, 'epoch': 1.16} {'loss': 0.0062, 'grad_norm': 0.0009605766390450299, 'learning_rate': 5.650252525252525e-05, 'epoch': 1.17} {'loss': 0.0, 'grad_norm': 0.000314085278660059, 'learning_rate': 5.6477272727272726e-05, 'epoch': 1.18} {'loss': 0.0046, 'grad_norm': 0.0017093310598284006, 'learning_rate': 5.64520202020202e-05, 'epoch': 1.19} {'loss': 0.0249, 'grad_norm': 0.4289214313030243, 'learning_rate': 5.642676767676768e-05, 'epoch': 1.2} {'loss': 0.0538, 'grad_norm': 0.1132945641875267, 'learning_rate': 5.640151515151515e-05, 'epoch': 1.2} {'loss': 0.0022, 'grad_norm': 2.8937032222747803, 'learning_rate': 5.6376262626262626e-05, 'epoch': 1.21} {'loss': 0.0004, 'grad_norm': 0.00865804310888052, 'learning_rate': 5.6351010101010106e-05, 'epoch': 1.22} {'loss': 0.0194, 'grad_norm': 0.1950988471508026, 'learning_rate': 5.632575757575758e-05, 'epoch': 1.23} {'loss': 0.0158, 'grad_norm': 112.79564666748047, 'learning_rate': 5.630050505050505e-05, 'epoch': 1.24} {'loss': 0.0006, 'grad_norm': 2.294863952556625e-05, 'learning_rate': 5.6275252525252525e-05, 'epoch': 1.25} {'loss': 0.0033, 'grad_norm': 0.277766615152359, 'learning_rate': 5.625e-05, 'epoch': 1.25} {'loss': 0.0083, 'grad_norm': 0.0001543212856631726, 'learning_rate': 5.622474747474748e-05, 'epoch': 1.26} 6%|██▎ | 1500/23760 [09:36<1:59:29, 3.10it/s]Checkpoint destination directory /bge-reranker-v2-m3-finetune/checkpoint-1500 already exists and is non-empty. Saving will proceed but saved results may be invalid. 06/18/2024 15:10:38 - INFO - FlagEmbedding.reranker.trainer - Saving model checkpoint to /bge-reranker-v2-m3-finetune/checkpoint-1500 {'loss': 0.0589, 'grad_norm': 32.28901290893555, 'learning_rate': 5.619949494949495e-05, 'epoch': 1.27} {'loss': 0.0172, 'grad_norm': 1.4971367120742798, 'learning_rate': 5.6174242424242425e-05, 'epoch': 1.28} {'loss': 0.0236, 'grad_norm': 68.1385498046875, 'learning_rate': 5.61489898989899e-05, 'epoch': 1.29} {'loss': 0.0318, 'grad_norm': 3.701099395751953, 'learning_rate': 5.612373737373738e-05, 'epoch': 1.3} {'loss': 0.0153, 'grad_norm': 5.6919264793396, 'learning_rate': 5.609848484848485e-05, 'epoch': 1.3} {'loss': 0.003, 'grad_norm': 3.2488622665405273, 'learning_rate': 5.6073232323232324e-05, 'epoch': 1.31} {'loss': 0.0219, 'grad_norm': 4.8588561185169965e-05, 'learning_rate': 5.60479797979798e-05, 'epoch': 1.32} {'loss': 0.0009, 'grad_norm': 0.00012108163355151191, 'learning_rate': 5.602272727272728e-05, 'epoch': 1.33} {'loss': 0.0112, 'grad_norm': 0.0002269998221891001, 'learning_rate': 5.599747474747475e-05, 'epoch': 1.34} {'loss': 0.0022, 'grad_norm': 2.5974725303967716e-06, 'learning_rate': 5.5972222222222224e-05, 'epoch': 1.35} {'loss': 0.0167, 'grad_norm': 4.9581403732299805, 'learning_rate': 5.59469696969697e-05, 'epoch': 1.36} {'loss': 0.024, 'grad_norm': 15.311726570129395, 'learning_rate': 5.592171717171717e-05, 'epoch': 1.36} {'loss': 0.0099, 'grad_norm': 0.109784334897995, 'learning_rate': 5.589646464646465e-05, 'epoch': 1.37} {'loss': 0.0107, 'grad_norm': 15.562332153320312, 'learning_rate': 5.587121212121212e-05, 'epoch': 1.38} {'loss': 0.0083, 'grad_norm': 0.027846520766615868, 'learning_rate': 5.5845959595959596e-05, 'epoch': 1.39} {'loss': 0.0272, 'grad_norm': 2.4102218958432786e-05, 'learning_rate': 5.5820707070707076e-05, 'epoch': 1.4} {'loss': 0.0046, 'grad_norm': 0.0032317941077053547, 'learning_rate': 5.579545454545455e-05, 'epoch': 1.41} {'loss': 0.0207, 'grad_norm': 2.6822562217712402, 'learning_rate': 5.577020202020202e-05, 'epoch': 1.41} {'loss': 0.0113, 'grad_norm': 0.14608246088027954, 'learning_rate': 5.5744949494949496e-05, 'epoch': 1.42} {'loss': 0.0067, 'grad_norm': 7.260042877987871e-08, 'learning_rate': 5.571969696969697e-05, 'epoch': 1.43} {'loss': 0.0007, 'grad_norm': 1.3852005622538854e-06, 'learning_rate': 5.569444444444444e-05, 'epoch': 1.44} {'loss': 0.004, 'grad_norm': 5.244543899607379e-06, 'learning_rate': 5.5669191919191915e-05, 'epoch': 1.45} {'loss': 0.0241, 'grad_norm': 58.14476776123047, 'learning_rate': 5.5643939393939395e-05, 'epoch': 1.46} {'loss': 0.0954, 'grad_norm': 179.63710021972656, 'learning_rate': 5.5618686868686875e-05, 'epoch': 1.46} {'loss': 0.0268, 'grad_norm': 6.957734512980096e-07, 'learning_rate': 5.559343434343435e-05, 'epoch': 1.47} {'loss': 0.0469, 'grad_norm': 26.24024200439453, 'learning_rate': 5.556818181818182e-05, 'epoch': 1.48} {'loss': 0.0149, 'grad_norm': 0.7500708699226379, 'learning_rate': 5.5542929292929295e-05, 'epoch': 1.49} {'loss': 0.0093, 'grad_norm': 1.4250688552856445, 'learning_rate': 5.551767676767677e-05, 'epoch': 1.5} {'loss': 0.0973, 'grad_norm': 0.023221028968691826, 'learning_rate': 5.549242424242424e-05, 'epoch': 1.51} {'loss': 0.0117, 'grad_norm': 0.014937836676836014, 'learning_rate': 5.5467171717171714e-05, 'epoch': 1.52} {'loss': 0.0408, 'grad_norm': 2.3218261048896238e-05, 'learning_rate': 5.544191919191919e-05, 'epoch': 1.52} {'loss': 0.0165, 'grad_norm': 0.3234850764274597, 'learning_rate': 5.5416666666666674e-05, 'epoch': 1.53} {'loss': 0.0053, 'grad_norm': 4.5968936319695786e-05, 'learning_rate': 5.539141414141415e-05, 'epoch': 1.54} {'loss': 0.0117, 'grad_norm': 0.017188785597682, 'learning_rate': 5.536616161616162e-05, 'epoch': 1.55} {'loss': 0.0159, 'grad_norm': 0.020273273810744286, 'learning_rate': 5.5340909090909094e-05, 'epoch': 1.56} {'loss': 0.0161, 'grad_norm': 16.025604248046875, 'learning_rate': 5.531565656565657e-05, 'epoch': 1.57} {'loss': 0.1338, 'grad_norm': 0.011700368486344814, 'learning_rate': 5.529040404040404e-05, 'epoch': 1.57} {'loss': 0.0053, 'grad_norm': 0.006936835125088692, 'learning_rate': 5.526515151515151e-05, 'epoch': 1.58} {'loss': 0.0289, 'grad_norm': 15.2365140914917, 'learning_rate': 5.5239898989898986e-05, 'epoch': 1.59} {'loss': 0.0078, 'grad_norm': 0.0022484296932816505, 'learning_rate': 5.5214646464646466e-05, 'epoch': 1.6} {'loss': 0.003, 'grad_norm': 0.0008051854674704373, 'learning_rate': 5.5189393939393946e-05, 'epoch': 1.61} {'loss': 0.0002, 'grad_norm': 5.998564665787853e-05, 'learning_rate': 5.516414141414142e-05, 'epoch': 1.62} {'loss': 0.0056, 'grad_norm': 16.861833572387695, 'learning_rate': 5.513888888888889e-05, 'epoch': 1.62} {'loss': 0.0067, 'grad_norm': 11.897306442260742, 'learning_rate': 5.5113636363636366e-05, 'epoch': 1.63} {'loss': 0.0124, 'grad_norm': 0.0005116038955748081, 'learning_rate': 5.508838383838384e-05, 'epoch': 1.64} {'loss': 0.0179, 'grad_norm': 0.8882714509963989, 'learning_rate': 5.506313131313131e-05, 'epoch': 1.65} {'loss': 0.0367, 'grad_norm': 1.8888447812059894e-05, 'learning_rate': 5.5037878787878785e-05, 'epoch': 1.66} {'loss': 0.01, 'grad_norm': 0.00020534262876026332, 'learning_rate': 5.5012626262626265e-05, 'epoch': 1.67} {'loss': 0.0201, 'grad_norm': 3.0491963798340294e-07, 'learning_rate': 5.498737373737374e-05, 'epoch': 1.68} {'loss': 0.0546, 'grad_norm': 0.0003539647732395679, 'learning_rate': 5.496212121212121e-05, 'epoch': 1.68} 8%|███ | 2000/23760 [13:04<1:58:19, 3.07it/s]Checkpoint destination directory /bge-reranker-v2-m3-finetune/checkpoint-2000 already exists and is non-empty. Saving will proceed but saved results may be invalid. 06/18/2024 15:14:05 - INFO - FlagEmbedding.reranker.trainer - Saving model checkpoint to /bge-reranker-v2-m3-finetune/checkpoint-2000 {'loss': 1.0735, 'grad_norm': inf, 'learning_rate': 5.493939393939394e-05, 'epoch': 1.69} {'loss': 0.126, 'grad_norm': 0.28912654519081116, 'learning_rate': 5.4914141414141413e-05, 'epoch': 1.7} {'loss': 0.0396, 'grad_norm': 2.875431776046753, 'learning_rate': 5.4888888888888893e-05, 'epoch': 1.71} {'loss': 0.008, 'grad_norm': 10.879746437072754, 'learning_rate': 5.4863636363636367e-05, 'epoch': 1.72} {'loss': 0.02, 'grad_norm': 1.8592783135318314e-07, 'learning_rate': 5.483838383838384e-05, 'epoch': 1.73} {'loss': 0.009, 'grad_norm': 2.541109367371064e-08, 'learning_rate': 5.481313131313131e-05, 'epoch': 1.73} {'loss': 0.0011, 'grad_norm': 0.11128569394350052, 'learning_rate': 5.4787878787878786e-05, 'epoch': 1.74} {'loss': 0.0093, 'grad_norm': 0.0009128592209890485, 'learning_rate': 5.4762626262626266e-05, 'epoch': 1.75} {'loss': 0.0208, 'grad_norm': 55.6444206237793, 'learning_rate': 5.473737373737374e-05, 'epoch': 1.76} {'loss': 0.0254, 'grad_norm': 3.2432677699034684e-07, 'learning_rate': 5.471212121212121e-05, 'epoch': 1.77} {'loss': 0.006, 'grad_norm': 1.3209187984466553, 'learning_rate': 5.4686868686868686e-05, 'epoch': 1.78} {'loss': 0.0005, 'grad_norm': 0.001798776094801724, 'learning_rate': 5.4661616161616166e-05, 'epoch': 1.78} {'loss': 0.0975, 'grad_norm': 258.0670471191406, 'learning_rate': 5.463636363636364e-05, 'epoch': 1.79} {'loss': 0.0763, 'grad_norm': 0.031031429767608643, 'learning_rate': 5.461111111111111e-05, 'epoch': 1.8} {'loss': 0.0051, 'grad_norm': 0.0004995210329070687, 'learning_rate': 5.4585858585858585e-05, 'epoch': 1.81} {'loss': 0.0018, 'grad_norm': 0.004655703902244568, 'learning_rate': 5.4560606060606065e-05, 'epoch': 1.82} {'loss': 0.0162, 'grad_norm': 0.11281134188175201, 'learning_rate': 5.453535353535354e-05, 'epoch': 1.83} {'loss': 0.0364, 'grad_norm': 0.23705099523067474, 'learning_rate': 5.451010101010101e-05, 'epoch': 1.84} {'loss': 0.0181, 'grad_norm': 5.897599697113037, 'learning_rate': 5.4484848484848485e-05, 'epoch': 1.84} {'loss': 0.036, 'grad_norm': 40.02043151855469, 'learning_rate': 5.445959595959596e-05, 'epoch': 1.85} {'loss': 0.0102, 'grad_norm': 0.12148799002170563, 'learning_rate': 5.443434343434344e-05, 'epoch': 1.86} {'loss': 0.021, 'grad_norm': 0.060332104563713074, 'learning_rate': 5.440909090909091e-05, 'epoch': 1.87} {'loss': 0.0128, 'grad_norm': 2.0891788210519735e-07, 'learning_rate': 5.4383838383838384e-05, 'epoch': 1.88} {'loss': 0.0235, 'grad_norm': 7.263846055138856e-05, 'learning_rate': 5.4358585858585864e-05, 'epoch': 1.89} {'loss': 0.1572, 'grad_norm': 1.2230504751205444, 'learning_rate': 5.433333333333334e-05, 'epoch': 1.89} {'loss': 0.2683, 'grad_norm': 26.9536075592041, 'learning_rate': 5.4310606060606066e-05, 'epoch': 1.9} {'loss': 0.0325, 'grad_norm': 0.14049798250198364, 'learning_rate': 5.428535353535354e-05, 'epoch': 1.91} {'loss': 0.061, 'grad_norm': 0.034856975078582764, 'learning_rate': 5.426010101010101e-05, 'epoch': 1.92} {'loss': 0.0041, 'grad_norm': 1.0403106212615967, 'learning_rate': 5.4234848484848485e-05, 'epoch': 1.93} {'loss': 0.0084, 'grad_norm': 0.10378583520650864, 'learning_rate': 5.420959595959596e-05, 'epoch': 1.94} {'loss': 0.0079, 'grad_norm': 5.192396770326013e-07, 'learning_rate': 5.418434343434343e-05, 'epoch': 1.94} {'loss': 0.012, 'grad_norm': 0.0, 'learning_rate': 5.415909090909091e-05, 'epoch': 1.95} {'loss': 0.0098, 'grad_norm': 5.0317822797296685e-08, 'learning_rate': 5.4133838383838385e-05, 'epoch': 1.96} {'loss': 0.0331, 'grad_norm': 39.18370819091797, 'learning_rate': 5.4108585858585865e-05, 'epoch': 1.97} {'loss': 0.002, 'grad_norm': 0.0013543523382395506, 'learning_rate': 5.408333333333334e-05, 'epoch': 1.98} {'loss': 0.0142, 'grad_norm': 2.2480250549961056e-07, 'learning_rate': 5.405808080808081e-05, 'epoch': 1.99} {'loss': 0.06, 'grad_norm': 0.00027097799465991557, 'learning_rate': 5.4032828282828284e-05, 'epoch': 1.99} {'loss': 0.0484, 'grad_norm': 36.98856735229492, 'learning_rate': 5.400757575757576e-05, 'epoch': 2.0} {'loss': 0.001, 'grad_norm': 0.8801558613777161, 'learning_rate': 5.398232323232323e-05, 'epoch': 2.01} {'loss': 0.0337, 'grad_norm': 0.34060463309288025, 'learning_rate': 5.3957070707070704e-05, 'epoch': 2.02} {'loss': 0.0357, 'grad_norm': 41.0213508605957, 'learning_rate': 5.393181818181818e-05, 'epoch': 2.03} {'loss': 0.0094, 'grad_norm': 1.802323339461509e-07, 'learning_rate': 5.3906565656565664e-05, 'epoch': 2.04} {'loss': 0.0336, 'grad_norm': 99.14340209960938, 'learning_rate': 5.388131313131314e-05, 'epoch': 2.05} {'loss': 0.6684, 'grad_norm': 80.79241180419922, 'learning_rate': 5.385606060606061e-05, 'epoch': 2.05} {'loss': 0.3803, 'grad_norm': 181.61868286132812, 'learning_rate': 5.383080808080808e-05, 'epoch': 2.06} {'loss': 0.1951, 'grad_norm': 0.012232287786900997, 'learning_rate': 5.3805555555555556e-05, 'epoch': 2.07} {'loss': 0.0858, 'grad_norm': 936.8713989257812, 'learning_rate': 5.378030303030303e-05, 'epoch': 2.08} {'loss': 0.2, 'grad_norm': 24.759891510009766, 'learning_rate': 5.37550505050505e-05, 'epoch': 2.09} {'loss': 0.1681, 'grad_norm': 34.64580154418945, 'learning_rate': 5.3729797979797976e-05, 'epoch': 2.1} {'loss': 0.1937, 'grad_norm': 0.9295755624771118, 'learning_rate': 5.3704545454545456e-05, 'epoch': 2.1} 11%|███▊ | 2500/23760 [16:28<1:51:14, 3.19it/s]Checkpoint destination directory /bge-reranker-v2-m3-finetune/checkpoint-2500 already exists and is non-empty. Saving will proceed but saved results may be invalid. 06/18/2024 15:17:30 - INFO - FlagEmbedding.reranker.trainer - Saving model checkpoint to /bge-reranker-v2-m3-finetune/checkpoint-2500 {'loss': 0.0386, 'grad_norm': 0.00017392029985785484, 'learning_rate': 5.3679292929292936e-05, 'epoch': 2.11} {'loss': 0.2917, 'grad_norm': 2.559640645980835, 'learning_rate': 5.365404040404041e-05, 'epoch': 2.12} {'loss': 1.1177, 'grad_norm': 44.51304244995117, 'learning_rate': 5.362878787878788e-05, 'epoch': 2.13} {'loss': 0.3164, 'grad_norm': 8.839674949645996, 'learning_rate': 5.3603535353535355e-05, 'epoch': 2.14} {'loss': 0.9214, 'grad_norm': 2.311915159225464, 'learning_rate': 5.357828282828283e-05, 'epoch': 2.15} {'loss': 2.1259, 'grad_norm': 2.232788324356079, 'learning_rate': 5.35530303030303e-05, 'epoch': 2.15} {'loss': 3.1265, 'grad_norm': 3.3011722564697266, 'learning_rate': 5.3527777777777775e-05, 'epoch': 2.16} {'loss': 2.4629, 'grad_norm': inf, 'learning_rate': 5.3505050505050503e-05, 'epoch': 2.17} {'loss': 2.3032, 'grad_norm': 10.117233276367188, 'learning_rate': 5.3479797979797977e-05, 'epoch': 2.18} {'loss': 0.2142, 'grad_norm': 0.0, 'learning_rate': 5.3454545454545457e-05, 'epoch': 2.19} {'loss': 0.096, 'grad_norm': 0.0, 'learning_rate': 5.342929292929293e-05, 'epoch': 2.2} {'loss': 0.1037, 'grad_norm': 0.9075354337692261, 'learning_rate': 5.340404040404041e-05, 'epoch': 2.21} {'loss': 0.2361, 'grad_norm': 0.5660173892974854, 'learning_rate': 5.337878787878788e-05, 'epoch': 2.21} {'loss': 0.1721, 'grad_norm': 1.2064839601516724, 'learning_rate': 5.3353535353535356e-05, 'epoch': 2.22} {'loss': 0.285, 'grad_norm': 0.20751313865184784, 'learning_rate': 5.332828282828283e-05, 'epoch': 2.23} {'loss': 0.3375, 'grad_norm': 1.0079563856124878, 'learning_rate': 5.33030303030303e-05, 'epoch': 2.24} {'loss': 0.2671, 'grad_norm': 0.8880073428153992, 'learning_rate': 5.3277777777777776e-05, 'epoch': 2.25} {'loss': 0.2024, 'grad_norm': 0.3787584602832794, 'learning_rate': 5.3252525252525255e-05, 'epoch': 2.26} {'loss': 0.0957, 'grad_norm': 0.44110560417175293, 'learning_rate': 5.322727272727273e-05, 'epoch': 2.26} {'loss': 0.1074, 'grad_norm': 0.6095941662788391, 'learning_rate': 5.32020202020202e-05, 'epoch': 2.27} {'loss': 0.1365, 'grad_norm': 830.3858642578125, 'learning_rate': 5.317676767676768e-05, 'epoch': 2.28} {'loss': 0.1474, 'grad_norm': 0.7444500923156738, 'learning_rate': 5.3151515151515155e-05, 'epoch': 2.29} {'loss': 0.2625, 'grad_norm': 22.683469772338867, 'learning_rate': 5.312626262626263e-05, 'epoch': 2.3} {'loss': 0.2137, 'grad_norm': 0.6679496169090271, 'learning_rate': 5.31010101010101e-05, 'epoch': 2.31} {'loss': 0.32, 'grad_norm': 1.977393388748169, 'learning_rate': 5.307828282828283e-05, 'epoch': 2.31} {'loss': 0.5425, 'grad_norm': 1.7041646242141724, 'learning_rate': 5.30530303030303e-05, 'epoch': 2.32} {'loss': 0.2425, 'grad_norm': 283.9384765625, 'learning_rate': 5.3027777777777776e-05, 'epoch': 2.33} {'loss': 0.7408, 'grad_norm': 2.008556842803955, 'learning_rate': 5.3002525252525256e-05, 'epoch': 2.34} {'loss': 2.0776, 'grad_norm': 2.0565271377563477, 'learning_rate': 5.297727272727273e-05, 'epoch': 2.35} {'loss': 2.1188, 'grad_norm': 2.030200958251953, 'learning_rate': 5.29520202020202e-05, 'epoch': 2.36} {'loss': 2.1132, 'grad_norm': 2.0532548427581787, 'learning_rate': 5.2926767676767676e-05, 'epoch': 2.37} {'loss': 2.0757, 'grad_norm': 1.9414801597595215, 'learning_rate': 5.290151515151515e-05, 'epoch': 2.37} {'loss': 2.0957, 'grad_norm': 2.039264678955078, 'learning_rate': 5.287626262626263e-05, 'epoch': 2.38} {'loss': 2.0916, 'grad_norm': 1.969018816947937, 'learning_rate': 5.28510101010101e-05, 'epoch': 2.39} {'loss': 2.1178, 'grad_norm': 2.1072139739990234, 'learning_rate': 5.2825757575757575e-05, 'epoch': 2.4} {'loss': 2.0732, 'grad_norm': 2.0081801414489746, 'learning_rate': 5.2800505050505055e-05, 'epoch': 2.41} {'loss': 2.0948, 'grad_norm': 2.0296790599823, 'learning_rate': 5.277525252525253e-05, 'epoch': 2.42} {'loss': 2.1184, 'grad_norm': 2.04976487159729, 'learning_rate': 5.275e-05, 'epoch': 2.42} {'loss': 2.1329, 'grad_norm': 2.0749306678771973, 'learning_rate': 5.2724747474747475e-05, 'epoch': 2.43} {'loss': 2.0653, 'grad_norm': 2.002948760986328, 'learning_rate': 5.269949494949495e-05, 'epoch': 2.44} {'loss': 2.1158, 'grad_norm': 2.036961793899536, 'learning_rate': 5.267424242424242e-05, 'epoch': 2.45} {'loss': 2.108, 'grad_norm': 2.078641176223755, 'learning_rate': 5.26489898989899e-05, 'epoch': 2.46} {'loss': 2.1039, 'grad_norm': 2.030880928039551, 'learning_rate': 5.2623737373737374e-05, 'epoch': 2.47} {'loss': 2.1049, 'grad_norm': 2.019310712814331, 'learning_rate': 5.2598484848484854e-05, 'epoch': 2.47} {'loss': 2.1116, 'grad_norm': 2.0421335697174072, 'learning_rate': 5.257323232323233e-05, 'epoch': 2.48} {'loss': 2.0784, 'grad_norm': 1.999159812927246, 'learning_rate': 5.25479797979798e-05, 'epoch': 2.49} {'loss': 2.1045, 'grad_norm': 2.08054518699646, 'learning_rate': 5.2522727272727274e-05, 'epoch': 2.5} {'loss': 2.0962, 'grad_norm': 1.9705493450164795, 'learning_rate': 5.249747474747475e-05, 'epoch': 2.51} {'loss': 2.066, 'grad_norm': 1.9970495700836182, 'learning_rate': 5.247222222222222e-05, 'epoch': 2.52} {'loss': 2.12, 'grad_norm': 2.0762932300567627, 'learning_rate': 5.244696969696969e-05, 'epoch': 2.53} 13%|████▌ | 3000/23760 [20:16<1:53:19, 3.05it/s]Checkpoint destination directory /bge-reranker-v2-m3-finetune/checkpoint-3000 already exists and is non-empty. Saving will proceed but saved results may be invalid. 06/18/2024 15:21:17 - INFO - FlagEmbedding.reranker.trainer - Saving model checkpoint to /bge-reranker-v2-m3-finetune/checkpoint-3000 {'loss': 2.0792, 'grad_norm': 1.9682689905166626, 'learning_rate': 5.242171717171717e-05, 'epoch': 2.53} {'loss': 2.1027, 'grad_norm': 2.0776031017303467, 'learning_rate': 5.239646464646465e-05, 'epoch': 2.54} {'loss': 2.0938, 'grad_norm': 2.043138027191162, 'learning_rate': 5.2371212121212126e-05, 'epoch': 2.55} {'loss': 2.1173, 'grad_norm': 2.0212814807891846, 'learning_rate': 5.23459595959596e-05, 'epoch': 2.56} {'loss': 2.0975, 'grad_norm': 2.0120596885681152, 'learning_rate': 5.232070707070707e-05, 'epoch': 2.57} {'loss': 2.0966, 'grad_norm': 2.02529239654541, 'learning_rate': 5.2295454545454546e-05, 'epoch': 2.58} {'loss': 2.094, 'grad_norm': 1.9973883628845215, 'learning_rate': 5.227020202020202e-05, 'epoch': 2.58} {'loss': 2.0879, 'grad_norm': 2.0289065837860107, 'learning_rate': 5.224494949494949e-05, 'epoch': 2.59} {'loss': 2.0969, 'grad_norm': 2.023595094680786, 'learning_rate': 5.2219696969696965e-05, 'epoch': 2.6} {'loss': 2.1058, 'grad_norm': 1.9871649742126465, 'learning_rate': 5.219444444444445e-05, 'epoch': 2.61} {'loss': 2.0917, 'grad_norm': 2.0561208724975586, 'learning_rate': 5.2169191919191925e-05, 'epoch': 2.62} {'loss': 2.0926, 'grad_norm': 1.9349627494812012, 'learning_rate': 5.21439393939394e-05, 'epoch': 2.63} {'loss': 2.0897, 'grad_norm': 2.0516836643218994, 'learning_rate': 5.211868686868687e-05, 'epoch': 2.63} {'loss': 2.1026, 'grad_norm': 2.0437283515930176, 'learning_rate': 5.2093434343434345e-05, 'epoch': 2.64} {'loss': 2.0953, 'grad_norm': 1.9620471000671387, 'learning_rate': 5.206818181818182e-05, 'epoch': 2.65} {'loss': 2.0876, 'grad_norm': 2.032616376876831, 'learning_rate': 5.204292929292929e-05, 'epoch': 2.66} {'loss': 2.0775, 'grad_norm': 2.020014524459839, 'learning_rate': 5.2017676767676764e-05, 'epoch': 2.67} {'loss': 2.0957, 'grad_norm': 2.02555251121521, 'learning_rate': 5.1992424242424244e-05, 'epoch': 2.68} {'loss': 2.0665, 'grad_norm': 2.042630672454834, 'learning_rate': 5.196717171717172e-05, 'epoch': 2.69} {'loss': 2.0942, 'grad_norm': 2.0313820838928223, 'learning_rate': 5.19419191919192e-05, 'epoch': 2.69} {'loss': 2.0945, 'grad_norm': 1.9612358808517456, 'learning_rate': 5.191666666666667e-05, 'epoch': 2.7} {'loss': 2.0953, 'grad_norm': 2.035776138305664, 'learning_rate': 5.1891414141414144e-05, 'epoch': 2.71} {'loss': 2.0936, 'grad_norm': 2.030928373336792, 'learning_rate': 5.186616161616162e-05, 'epoch': 2.72} {'loss': 2.1074, 'grad_norm': 2.0329926013946533, 'learning_rate': 5.184090909090909e-05, 'epoch': 2.73} {'loss': 2.1131, 'grad_norm': 2.038585901260376, 'learning_rate': 5.181565656565656e-05, 'epoch': 2.74} {'loss': 2.102, 'grad_norm': 2.03714656829834, 'learning_rate': 5.179040404040404e-05, 'epoch': 2.74} {'loss': 2.1168, 'grad_norm': 1.9533993005752563, 'learning_rate': 5.1765151515151516e-05, 'epoch': 2.75} {'loss': 2.0951, 'grad_norm': 2.0026047229766846, 'learning_rate': 5.173989898989899e-05, 'epoch': 2.76} {'loss': 2.0941, 'grad_norm': 2.0478105545043945, 'learning_rate': 5.171464646464647e-05, 'epoch': 2.77} {'loss': 2.0977, 'grad_norm': 1.9859627485275269, 'learning_rate': 5.168939393939394e-05, 'epoch': 2.78} {'loss': 2.1045, 'grad_norm': 2.0018675327301025, 'learning_rate': 5.1664141414141416e-05, 'epoch': 2.79} {'loss': 2.111, 'grad_norm': 1.97560715675354, 'learning_rate': 5.163888888888889e-05, 'epoch': 2.79} {'loss': 2.0741, 'grad_norm': 2.002556324005127, 'learning_rate': 5.161363636363636e-05, 'epoch': 2.8} {'loss': 2.1138, 'grad_norm': 2.0048749446868896, 'learning_rate': 5.158838383838384e-05, 'epoch': 2.81} {'loss': 2.0928, 'grad_norm': 2.0539207458496094, 'learning_rate': 5.1563131313131315e-05, 'epoch': 2.82} {'loss': 2.1123, 'grad_norm': 2.032982349395752, 'learning_rate': 5.153787878787879e-05, 'epoch': 2.83} {'loss': 2.1187, 'grad_norm': 2.0528159141540527, 'learning_rate': 5.151262626262626e-05, 'epoch': 2.84} {'loss': 2.1047, 'grad_norm': 2.111104726791382, 'learning_rate': 5.148737373737374e-05, 'epoch': 2.85} {'loss': 2.0934, 'grad_norm': 1.9932841062545776, 'learning_rate': 5.1462121212121215e-05, 'epoch': 2.85} {'loss': 2.0962, 'grad_norm': 1.9658905267715454, 'learning_rate': 5.143686868686869e-05, 'epoch': 2.86} {'loss': 2.0807, 'grad_norm': 2.023836612701416, 'learning_rate': 5.141161616161616e-05, 'epoch': 2.87} {'loss': 2.1012, 'grad_norm': 2.0127310752868652, 'learning_rate': 5.138636363636364e-05, 'epoch': 2.88} {'loss': 2.0967, 'grad_norm': 1.986143708229065, 'learning_rate': 5.1361111111111114e-05, 'epoch': 2.89} {'loss': 2.1062, 'grad_norm': 2.0810837745666504, 'learning_rate': 5.133585858585859e-05, 'epoch': 2.9} {'loss': 2.0874, 'grad_norm': 2.020179033279419, 'learning_rate': 5.131060606060606e-05, 'epoch': 2.9} {'loss': 2.0895, 'grad_norm': 1.9634133577346802, 'learning_rate': 5.1285353535353534e-05, 'epoch': 2.91} {'loss': 2.1006, 'grad_norm': 2.0037992000579834, 'learning_rate': 5.126010101010101e-05, 'epoch': 2.92} {'loss': 2.0874, 'grad_norm': 1.9418680667877197, 'learning_rate': 5.123484848484849e-05, 'epoch': 2.93} {'loss': 2.1048, 'grad_norm': 2.0951955318450928, 'learning_rate': 5.120959595959596e-05, 'epoch': 2.94} {'loss': 2.1065, 'grad_norm': 2.013732671737671, 'learning_rate': 5.118434343434344e-05, 'epoch': 2.95} 15%|█████▎ | 3500/23760 [24:08<1:51:34, 3.03it/s]Checkpoint destination directory /bge-reranker-v2-m3-finetune/checkpoint-3500 already exists and is non-empty. Saving will proceed but saved results may be invalid. 06/18/2024 15:25:10 - INFO - FlagEmbedding.reranker.trainer - Saving model checkpoint to /bge-reranker-v2-m3-finetune/checkpoint-3500 {'loss': 2.0873, 'grad_norm': 2.0073935985565186, 'learning_rate': 5.115909090909091e-05, 'epoch': 2.95} {'loss': 2.0854, 'grad_norm': 2.0986244678497314, 'learning_rate': 5.1133838383838386e-05, 'epoch': 2.96} {'loss': 2.103, 'grad_norm': 1.980019211769104, 'learning_rate': 5.110858585858586e-05, 'epoch': 2.97} {'loss': 2.1002, 'grad_norm': 2.0264763832092285, 'learning_rate': 5.108333333333333e-05, 'epoch': 2.98} {'loss': 2.1078, 'grad_norm': 1.9887807369232178, 'learning_rate': 5.1058080808080806e-05, 'epoch': 2.99} {'loss': 2.0997, 'grad_norm': 2.065340042114258, 'learning_rate': 5.103282828282828e-05, 'epoch': 3.0} {'loss': 2.103, 'grad_norm': 1.9552637338638306, 'learning_rate': 5.100757575757576e-05, 'epoch': 3.01} {'loss': 2.0884, 'grad_norm': 2.040771007537842, 'learning_rate': 5.098232323232324e-05, 'epoch': 3.01} {'loss': 2.1007, 'grad_norm': 2.0140607357025146, 'learning_rate': 5.095707070707071e-05, 'epoch': 3.02} {'loss': 2.0962, 'grad_norm': 1.9713252782821655, 'learning_rate': 5.0931818181818185e-05, 'epoch': 3.03} {'loss': 2.0848, 'grad_norm': 2.018899917602539, 'learning_rate': 5.090656565656566e-05, 'epoch': 3.04} {'loss': 2.1112, 'grad_norm': 2.084656238555908, 'learning_rate': 5.088131313131313e-05, 'epoch': 3.05} {'loss': 2.1026, 'grad_norm': 1.9733964204788208, 'learning_rate': 5.0856060606060605e-05, 'epoch': 3.06} {'loss': 2.0865, 'grad_norm': 2.0735559463500977, 'learning_rate': 5.083080808080808e-05, 'epoch': 3.06} {'loss': 2.0894, 'grad_norm': 1.997558355331421, 'learning_rate': 5.080555555555555e-05, 'epoch': 3.07} {'loss': 2.0901, 'grad_norm': 2.0256588459014893, 'learning_rate': 5.078030303030304e-05, 'epoch': 3.08} {'loss': 2.1009, 'grad_norm': 2.112149477005005, 'learning_rate': 5.075505050505051e-05, 'epoch': 3.09} {'loss': 2.0998, 'grad_norm': 2.001399278640747, 'learning_rate': 5.0729797979797984e-05, 'epoch': 3.1} {'loss': 2.0921, 'grad_norm': 2.0737903118133545, 'learning_rate': 5.070454545454546e-05, 'epoch': 3.11} {'loss': 2.1008, 'grad_norm': 2.032071113586426, 'learning_rate': 5.067929292929293e-05, 'epoch': 3.11} {'loss': 2.1049, 'grad_norm': 1.9937902688980103, 'learning_rate': 5.0654040404040404e-05, 'epoch': 3.12} {'loss': 2.0998, 'grad_norm': 2.0569865703582764, 'learning_rate': 5.062878787878788e-05, 'epoch': 3.13} {'loss': 2.1004, 'grad_norm': 1.9839125871658325, 'learning_rate': 5.060353535353535e-05, 'epoch': 3.14} {'loss': 2.119, 'grad_norm': 1.9326714277267456, 'learning_rate': 5.057828282828283e-05, 'epoch': 3.15} {'loss': 2.0913, 'grad_norm': 1.9277666807174683, 'learning_rate': 5.055303030303031e-05, 'epoch': 3.16} {'loss': 2.0982, 'grad_norm': 1.9463118314743042, 'learning_rate': 5.052777777777778e-05, 'epoch': 3.16} {'loss': 2.0866, 'grad_norm': 2.0670173168182373, 'learning_rate': 5.0502525252525256e-05, 'epoch': 3.17} {'loss': 2.0937, 'grad_norm': 2.0291764736175537, 'learning_rate': 5.047727272727273e-05, 'epoch': 3.18} {'loss': 2.1174, 'grad_norm': 2.0290634632110596, 'learning_rate': 5.04520202020202e-05, 'epoch': 3.19} {'loss': 2.0913, 'grad_norm': 2.014747381210327, 'learning_rate': 5.0426767676767676e-05, 'epoch': 3.2} {'loss': 2.1095, 'grad_norm': 2.0343189239501953, 'learning_rate': 5.040151515151515e-05, 'epoch': 3.21} {'loss': 2.0931, 'grad_norm': 1.974791169166565, 'learning_rate': 5.037626262626263e-05, 'epoch': 3.22} {'loss': 2.0735, 'grad_norm': 1.9767214059829712, 'learning_rate': 5.03510101010101e-05, 'epoch': 3.22} {'loss': 2.0955, 'grad_norm': 1.952515721321106, 'learning_rate': 5.0325757575757575e-05, 'epoch': 3.23} {'loss': 2.0945, 'grad_norm': 2.0902280807495117, 'learning_rate': 5.0300505050505055e-05, 'epoch': 3.24} {'loss': 2.0807, 'grad_norm': 2.0410678386688232, 'learning_rate': 5.027525252525253e-05, 'epoch': 3.25} {'loss': 2.0925, 'grad_norm': 1.9657814502716064, 'learning_rate': 5.025e-05, 'epoch': 3.26} {'loss': 2.1264, 'grad_norm': 2.022275447845459, 'learning_rate': 5.0224747474747475e-05, 'epoch': 3.27} {'loss': 2.088, 'grad_norm': 1.9936144351959229, 'learning_rate': 5.019949494949495e-05, 'epoch': 3.27} {'loss': 2.0937, 'grad_norm': 2.0525431632995605, 'learning_rate': 5.017424242424243e-05, 'epoch': 3.28} {'loss': 2.1035, 'grad_norm': 2.8155758380889893, 'learning_rate': 5.01489898989899e-05, 'epoch': 3.29} {'loss': 2.0707, 'grad_norm': 2.00917387008667, 'learning_rate': 5.0123737373737374e-05, 'epoch': 3.3} {'loss': 2.1148, 'grad_norm': 4.4301066398620605, 'learning_rate': 5.009848484848485e-05, 'epoch': 3.31} {'loss': 2.0831, 'grad_norm': 2.375608444213867, 'learning_rate': 5.007323232323233e-05, 'epoch': 3.32} {'loss': 2.1803, 'grad_norm': 2.131770372390747, 'learning_rate': 5.00479797979798e-05, 'epoch': 3.32} {'loss': 2.1073, 'grad_norm': 2.1126272678375244, 'learning_rate': 5.0022727272727274e-05, 'epoch': 3.33} {'loss': 2.0882, 'grad_norm': 2.083080291748047, 'learning_rate': 4.999747474747475e-05, 'epoch': 3.34} {'loss': 2.0865, 'grad_norm': 2.078681468963623, 'learning_rate': 4.997222222222223e-05, 'epoch': 3.35} {'loss': 2.1174, 'grad_norm': 2.052236795425415, 'learning_rate': 4.99469696969697e-05, 'epoch': 3.36} {'loss': 2.0944, 'grad_norm': 2.0600247383117676, 'learning_rate': 4.992171717171717e-05, 'epoch': 3.37} 17%|██████ | 4000/23760 [27:51<1:50:50, 2.97it/s]Checkpoint destination directory /bge-reranker-v2-m3-finetune/checkpoint-4000 already exists and is non-empty. Saving will proceed but saved results may be invalid. 06/18/2024 15:28:53 - INFO - FlagEmbedding.reranker.trainer - Saving model checkpoint to /bge-reranker-v2-m3-finetune/checkpoint-4000

staoxiao commented 4 months ago

You can try to reduce the learning_rate and num_train_epochs

LinXin04 commented 4 months ago

训练两个字符串的相似读，为什么loss 一直都是0.05左右呢？没有明显的下降，合并之后效果也不好。@staoxiao 配置： !torchrun --nproc_per_node 1 \ -m FlagEmbedding.reranker.run \ --output_dir /bge-reranker-v2-m3-finetune \ --model_name_or_path /bge-reranker-v2-m3/bge-reranker-v2-m3 \ --train_data output.jsonl \ --learning_rate 1e-7 \ --fp16 \ --num_train_epochs 5 \ --per_device_train_batch_size 6 \ --gradient_accumulation_steps 4 \ --dataloader_drop_last True \ --train_group_size 8 \ --max_len 512 \ --save_steps 3000 \ --weight_decay 0.01 \ --logging_steps 10

合并： from LM_Cocktail import mix_models, mix_models_with_data model = mix_models( model_names_or_paths=["/bge-reranker-v2-m3/bge-reranker-v2-m3", "/bge-reranker-v2-m3-finetune/checkpoint-5500/"], model_type='encoder', weights=[0.1, 0.9], # you can change the weights to get a better trade-off. output_path='/mixed_bge-reranker-v2-m3')

训练前： reranker = FlagReranker('/bge-reranker-v2-m3/bge-reranker-v2-m3', use_fp16=True) #use fp16 can speed up computing score = reranker.compute_score(['Dockers', 'Dockers Singapore'], normalize=True) 0.5990160972353846

训练后： reranker = FlagReranker('/mixed_bge-reranker-v2-m3', use_fp16=True) #use fp16 can speed up computing score = reranker.compute_score(['Dockers', 'Dockers Singapore'], normalize=True) 0.543166200316204

训练数据： {"query":"Dockers Singapore","pos":["DOCKERS"],"neg":["Cosas United","Foot Efx","Lobby Salons","KEDAIBISKUTSINGUANHOE","SMART RING","Coconut cafe&bar","D'CLUB"]}

staoxiao commented 4 months ago

训练目标是交叉熵，将正样本排在负样本前面即可，并没有严格约束要增加正样本的相似度。 另外，模型合并是可选选项，可以尝试不做。

waileong-leong commented 3 months ago

@LinXin04 有找到什么解决方案吗

magicleo commented 11 hours ago

@staoxiao 同问，微调reranker模型发现我的loss也是从开始0.1陡然变成2.。之后模型对任意两个文本间相似度计算都是0.999

{'loss': 0.142, 'grad_norm': 4.939193242100627, 'learning_rate': 5.0699337059664634e-05, 'epoch': 0.48} {'loss': 0.1164, 'grad_norm': 0.14117748951263004, 'learning_rate': 5.030937215650592e-05, 'epoch': 0.49} {'loss': 0.1206, 'grad_norm': 0.026069257073033935, 'learning_rate': 4.9919407253347196e-05, 'epoch': 0.5} {'loss': 2.1139, 'grad_norm': 2.5500297432455112, 'learning_rate': 4.9545040946314835e-05, 'epoch': 0.51} {'loss': 2.1174, 'grad_norm': 2.601261614104946, 'learning_rate': 4.915507604315611e-05, 'epoch': 0.53} {'loss': 2.1125, 'grad_norm': 2.4397848131315203, 'learning_rate': 4.8765111139997404e-05, 'epoch': 0.54} {'loss': 2.0982, 'grad_norm': 2.501285317937507, 'learning_rate': 4.837514623683869e-05, 'epoch': 0.55}

微调的语句是： torchrun --nproc_per_node 1 --master_port=29501 \ -m FlagEmbedding.finetune.reranker.encoder_only.base \ --model_name_or_path /ymt-ai/models/bge-reranker-v2-m3 \ --cache_dir ./cache/model \ --train_data /nas/rcmd-mart/embedding/search_query_supply/train_data \ --cache_path ./cache/data \ --train_group_size 8 \ --query_max_len 64 \ --passage_max_len 256 \ --pad_to_multiple_of 8 \ --knowledge_distillation False \ --output_dir /nas/rcmd-mart/embedding/search_query_supply/models/reranker_finetune \ --overwrite_output_dir \ --learning_rate 6e-5 \ --fp16 \ --num_train_epochs 2 \ --per_device_train_batch_size 16 \ --gradient_accumulation_steps 1 \ --dataloader_drop_last True \ --warmup_ratio 0.1 \ --gradient_checkpointing \ --weight_decay 0.01 \ --deepspeed /mnt/storage/FlagEmbedding/examples/finetune/ds_stage0.json \ --logging_steps 100 \ --save_steps 1000