Changing entry.py for MisconfigurationException error

[Sangyoon-Bae]

Hi! This is Stella from Seoul National University, I'm getting a lot of help from your code. I have a question about entry.py line 87. Originally it has metric = 'valid_' + get_dataset(dm.dataset_name)['metric']

but when I run model, I faced error like this: 'pytorch_lightning.utilities.exceptions.MisconfigurationException: ModelCheckpoint(monitor='valid_mae') not found in the returned metrics: ['train_loss']. HINT: Did you call self.log('valid_mae', value) in the LightningModule?'

So I changed the line 87 as metric = 'train_loss'

and it runs well.

I'm quite afraid that I'm doing something wrong, is it right way to modify the code? Here are some useful information for my project:

1. task: regression
2. input type : integer (originally continuous value, but discretized)
3. target type : real value
4. eval metric : rmse
5. features from data.py
   • 'num_class': 1,
   • 'loss_fn': F.l1_loss,
   • 'metric': 'mae',
   • 'metric_mode': 'min',

[zhengsx]

You can have a look about the usage of ModelCheckpoint in pytorch_lightning (see here). Basically it helps to monitor a self-defined metric during the training process.

[Sangyoon-Bae]

Thank you for kind reply! I'm always grateful for your help :) I used shell script in training ZINC dataset, and is this script just for pre-training? If I want to validate model and test it, then should I do transfer learning like molhiv training example with pre-trained model which is made with this script?

[zhengsx]

The reproduction of the ZINC results reported in our paper doesn't need pre-training. You can directly use the script. Btw, remember to test your model on ZINC testset, refer to #35 .

[Sangyoon-Bae]

Then shell script in training ZINC dataset do training and validation, and should I use test dataset with another script to reproduce test MAE 0.122?

then loss in log bar (it is not ZINC data) means validation loss?

[Sangyoon-Bae]

I solved the second problem! the loss in log bar means average loss about the past 100 batches in training.

[zhengsx]

Then shell script in training ZINC dataset do training and validation, and should I use test dataset with another script to reproduce test MAE 0.122?

then loss in log bar (it is not ZINC data) means validation loss?

Yes, pls save your ckpt according to best val then test it on Testset.

[zhengsx]

Close this issue due to inactive. Feel free to raise a new one or reopen this one for any further question.