Model gives different scores for the same drug-target pairs

[pykao]

Hi Kexin,

I tried to write the inference code to predict the binding affinity probability given the drug-target pairs. However, I found that the model always gives different scores for the same inputs d, p, d_mask, p_mask.

        score = model(d.long().cuda(), p.long().cuda(), d_mask.long().cuda(), p_mask.long().cuda())

Then, I entered two checkpoints in the train.py:

https://github.com/pykao/MolTrans/blob/c91a98eced0e18f9b63d439f84355f6225567287/train.py#L58

https://github.com/pykao/MolTrans/blob/c91a98eced0e18f9b63d439f84355f6225567287/train.py#L178

Then, I entered

CUDA_VISIBLE_DEVICES=2,3,5,6 python train.py --task davis

It entered the IPython interface.

In   [1]:   score = model(d.long().cuda(), p.long().cuda(), d_mask.long().cuda(), p_mask.long().cuda())

In   [2]:   score_1 = model(d.long().cuda(), p.long().cuda(), d_mask.long().cuda(), p_mask.long().cuda())

In   [3]:   score == score_1

Out[3]: 
tensor([[False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],                                  
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False],
        [False]], device='cuda:0')

Could you tell me why the model give different scores while the input drug-target pairs are the same?

Best regards, Po-Yu Kao

[kexinhuang12345]

hi sorry for the late reply. Have you tried model.eval() before feeding the input?

[pykao]

Hi @kexinhuang12345,

I have tried model.eval() before feeding the input.

https://github.com/pykao/MolTrans/blob/c91a98eced0e18f9b63d439f84355f6225567287/train.py#L54

But it still gives me different logits for the same input.

Could you please help me to solve this problem?

[pykao]

@kexinhuang12345

I think there is a issue on the dropout layer in model.py

https://github.com/kexinhuang12345/MolTrans/blob/47ac16b8c158b080ba6cdaec74cd7aa9c1332b73/models.py#L103

even if you enter model.eval() before the input.

This dropout layer is still enable.

Best, Po-Yu (Ken) Kao

[pykao]

Hi @kexinhuang12345,

Could you find the reason why I get different predictions for the same drug-target pairs?