wzzheng / HDML

Implementation of Hardness-Aware Deep Metric Learning (CVPR 2019 Oral) in Tensorflow.
151 stars 28 forks source link

Error for products dataset #12

Closed batrlatom closed 5 years ago

batrlatom commented 5 years ago

Hello, I am having problems to make products dataset work. I was able to convert dataset into hd5y format and start the main worker with

python main_npair.py --dataSet='products' --batch_size=128 --Regular_factor=5e-3 --init_learning_rate=7e-5 --load_formalVal=False --embedding_size=128 --loss_l2_reg=3e-3 --init_batch_per_epoch=640 --batch_per_epoch=64 --max_steps=6400 --beta=1e+4 --lr_gen=1e-2 --num_class=11319 --_lambda=0.5 --s_lr=1e-3 --Apply_HDML

But after few iterations, I am having errors. I think that it is related to the batch construction, but I do not see right solution right yet. The error is:

weight_decay : is nan, not record J_metric : is nan, not record Jm : is nan, not record

The situation is the same if I use tripletloss. Do you know how to solve the issue? Thanks, Tom