search

vita-epfl / detection-attributes-fields

PyTorch implementation of "Detecting 32 Pedestrian Attributes for Autonomous Vehicles"
https://arxiv.org/abs/2012.02647
Other
31 stars 6 forks source link

loss become negative after 100 epochs #3

Open lucasjinreal opened 3 years ago

lucasjinreal commented 3 years ago 
INFO:openpifpaf.network.trainer:{'type': 'train', 'epoch': 65, 'batch': 840, 'n_batches': 1266, 'time': 0.948, 'data_time': 0.0, 'lr': 0.0005, 'loss': -31.85, 'head_losses': [0.003, 0.097, 0.076, 0.111, 0.02, 0.25, 0.016, 0.027, 0.001, 0.0, 0.014, 0.015, 0.026, 0.019, 0.073, 0.021, 0.013, 0.004, 0.0, 0.039, 0.09, 0.088, 0.037, 0.006, 0.0, 0.003, 0.014, 0.003, 0.012, 0.011, 0.002, 0.002, 0.001, 0.002, 0.012], 'mtl_sigmas': [0.002, 0.301, 0.275, 0.367, 0.081, 0.518, 0.089, 0.171, 0.016, 0.0, 0.1, 0.092, 0.124, 0.122, 0.218, 0.046, 0.09, 0.027, 0.0, 0.176, 0.262, 0.26, 0.157, 0.037, 0.0, 0.003, 0.09, 0.003, 0.087, 0.065, 0.006, 0.01, 0.0, 0.002, 0.072]}

is this normal?

taylormordan commented 3 years ago

Yes, negative loss values can happen when using the --auto-tune-mtl option, this is not an issue. This is due to the formulation used for the loss function. Note that if you train on JAAD with initialization from the OpenPifPaf checkpoint, you should need much fewer epochs to converge.

lucasjinreal commented 3 years ago

@taylormordan I set a 100 epochs, but loss came to -31, what's the normal value of loss, is it become close to 0?

taylormordan commented 3 years ago

If you train for 5-10 epochs, the loss should be around 0 on average over an epoch, but loss values for individual batches may get higher or lower easily. I also observe this behavior.

lucasjinreal commented 3 years ago

@taylormordan Will the result become worse if train more epochs?

taylormordan commented 3 years ago

It will start to overfit at one point. The optimal number of epochs might depend on your hyper-parameter choice though (learning rate, batch size...).

Affanabbbas commented 2 years ago

hi can you plz help me with this issue?

python3 -m openpifpaf.train: error: unrecognized arguments: --datasets jaad --jaad-root-dir /content/drive/MyDrive/jaad/JAAD_clips/ --jaad-subset default --jaad-training-set train --jaad-validation-set val --pifpaf-pretraining --detection-bias-prior 0.01 --jaad-head-upsample 2 --jaad-pedestrian-attributes all --fork-normalization-operation power --fork-normalization-duplicates 35 --attribute-regression-loss l1 --attribute-focal-gamma 2

This is what I get after running the script. How have u donwloaded the dataset and what have you done with the annotations?