search

chainer / chainer-chemistry

Chainer Chemistry: A Library for Deep Learning in Biology and Chemistry
MIT License
618 stars 129 forks source link

add shifted softplus #395

Closed mist714 closed 4 years ago

mist714 commented 4 years ago

Add shifted-softplus for SchNet for stability Add test of shifted softplus

qm9 examples

before

epoch       main/loss   main/mae    main/rmse   validation/main/loss  validation/main/mae  validation/main/rmse  elapsed_time
1           9.16072e+07  4635.03     5493.88     2.04751e+06           1172.89              1430.77               4.68606       
2           736497      626.273     809.699     175424                307.404              418.077               8.61846       
3           111058      246.223     331.111     83034.3               203.431              286.713               12.6977       
...
20          8439.32     56.8124     90.6312     9510.97               55.5326              92.7485               81.2714

after

epoch       main/loss   main/mae    main/rmse   validation/main/loss  validation/main/mae  validation/main/rmse  elapsed_time
1           1665.32     28.0805     33.3155     163.623               7.94275              12.7757               4.86237       
2           106.288     4.85469     10.128      69.1701               3.48232              8.2747                9.02798       
3           59.911      2.95717     7.69376     41.7183               2.6097               6.42595               13.3118       
...
20          0.719011    0.659139    0.847248    0.741667              0.641484             0.85456               86.3188
codecov-io commented 4 years ago

Codecov Report

Merging #395 into master will decrease coverage by 4.73%. The diff coverage is 76.78%.

@@            Coverage Diff             @@
##           master     #395      +/-   ##
==========================================
- Coverage    91.3%   86.57%   -4.74%     
==========================================
  Files         227      229       +2     
  Lines       10999    11051      +52     
==========================================
- Hits        10043     9567     -476     
- Misses        956     1484     +528
corochann commented 4 years ago

SchNetReadout の部分も shifted_softplus に変えてもらって、性能変わるか見ていただけないでしょうか。