Closed wilke closed 1 year ago
Command: IMPROVE_RESULT val_loss: 1.7727327054408248
Output:
Validation at Epoch 2 with loss:1.60653, MSE: 1.86214 , Pearson Correlation: 0.87763 with p-value: 0.0 Spearman Correlation: 0.83264 with p_value: 0.0 , Concordance Index: 0.82473
{'val_loss': 1.8621445139104555, 'rmse': 1.3646041601543122, 'pcc': 0.8776368581577472, 'scc': 0.8326412524002879, 'r2': 0.7668459324633321, 'best_epoch': 1}
Training at Epoch 3 iteration 0 with loss 1.86644. Total time 0.02583 hours
Currenf MSE: 1.7727327054408248
Validation at Epoch 3 with loss:1.71383, MSE: 1.77273 , Pearson Correlation: 0.88492 with p-value: 0.0 Spearman Correlation: 0.84164 with p_value: 0.0 , Concordance Index: 0.82975
{'val_loss': 1.7727327054408248, 'rmse': 1.3314400870639373, 'pcc': 0.8849223686446626, 'scc': 0.8416452175582434, 'r2': 0.7780409426651591, 'best_epoch': 2}
IMPROVE_RESULT val_loss: 1.7727327054408248
Status: OK
Missing commits - develop is failing
singularity run --nv --bind `pwd`/tmp:/candle_data_dir build/DeepTTC.sif train.sh 3 /candle_data_dir --epochs 1
....
Checkpoint save interval == 0 -> checkpoints are disabled.
Training at Epoch 1 iteration 0 with loss 14.3121. Total time 0.00055 hours
Currenf MSE: 2.1630810230494517
Validation at Epoch 1 with loss:1.75420, MSE: 2.16308 , Pearson Correlation: 0.85460 with p-value: 0.0 Spearman Correlation: 0.80406 with p_value: 0.0 , Concordance Index: 0.80849
{'val_loss': 2.1630810230494517, 'rmse': 1.4707416574808274, 'pcc': 0.8546015608904385, 'scc': 0.8040606021397423, 'r2': 0.7291664877951529, 'best_epoch': 0}
IMPROVE_RESULT val_loss: 2.1630810230494517
Status: Failed Command:
singularity exec --bind ${TEST_DIR}:/candle_data_dir build/DeepTTC.sif train.sh 4 /candle_data_dir
Depends on:Output: