Dimaio/block pair scoring

[fdimaio]

Adds block-pair scoring to ljlk. The scoring module now returns an nsubterm x npose x nblock x nblock tensor where element [i,j,k,l] is the score of subterm i (fa_atr/rep/sol), pose j, between blocks k and l.

The implementation now no longer computes derivatives in the forward pass but rather stashes the interaction graph during forward and performs another kernel launch in backwards to compute gradients.

This strategy allows reweighing the interaction graph in loss computation. A new test is added to evaluate this.

To do before merge:

• ensure this still works with other pose scoring tests
• split lj to ljatr and ljrep subterms

[fdimaio]

Unfortunately, current version seems to be a modest slowdown.

Old:

(time in us) 
res_centric_score_benchmark[cpu-ljlk-backward-001]           13,781.8485 (0.46)            72.5592 (2.18)          0.6324 (2.39)   
res_centric_score_benchmark[cpu-ljlk-backward-003]           10,182.1291 (0.34)            98.2113 (2.94)          1.1502 (4.35)   
res_centric_score_benchmark[cpu-ljlk-backward-010]            7,440.0088 (0.25)           134.4084 (4.03)          4.6277 (17.50)  
res_centric_score_benchmark[cpu-ljlk-backward-030]            5,648.6673 (0.19)           177.0329 (5.31)         30.8044 (116.46) 
res_centric_score_benchmark[cpu-ljlk-backward-100]            4,065.4750 (0.14)           245.9737 (7.37)         26.1320 (98.80)  
res_centric_score_benchmark[cpu-ljlk-forward-001]                64.3741 (0.00)        15,534.1950 (465.72)        9.5880 (36.25)  
res_centric_score_benchmark[cpu-ljlk-forward-003]                23.9382 (0.00)        41,774.1649 (>1000.0)  10,762.5443 (>1000.0)
res_centric_score_benchmark[cpu-ljlk-forward-010]                 7.7218 (0.00)       129,504.1755 (>1000.0)      46.3361 (175.19) 
res_centric_score_benchmark[cpu-ljlk-forward-030]                 2.8104 (0.00)       355,820.0859 (>1000.0)      81.7738 (309.17) 
res_centric_score_benchmark[cpu-ljlk-forward-100]                 0.9096 (0.00)     1,099,345.6833 (>1000.0)   1,750.7356 (>1000.0)
res_centric_score_benchmark[cpu-ljlk-full-001]                   41.1934 (0.00)        24,275.7231 (727.80)       17.7175 (66.99)  
res_centric_score_benchmark[cpu-ljlk-full-003]                   17.0274 (0.00)        58,728.9095 (>1000.0)  16,831.9289 (>1000.0)
res_centric_score_benchmark[cpu-ljlk-full-010]                    4.9262 (0.00)       202,997.6688 (>1000.0)      69.3686 (262.27) 
res_centric_score_benchmark[cpu-ljlk-full-030]                    1.8745 (0.00)       533,471.9777 (>1000.0)     336.7001 (>1000.0)
res_centric_score_benchmark[cpu-ljlk-full-100]                    0.5910 (0.00)     1,692,100.0294 (>1000.0)   3,399.7176 (>1000.0)

res_centric_score_benchmark[cuda-ljlk-backward-001]           5,249.2890 (0.18)           190.5020 (5.71)         63.6904 (240.80) 
res_centric_score_benchmark[cuda-ljlk-backward-003]           6,936.6846 (0.23)           144.1611 (4.32)          5.6773 (21.46)  
res_centric_score_benchmark[cuda-ljlk-backward-010]           6,615.5291 (0.22)           151.1595 (4.53)         43.1985 (163.32) 
res_centric_score_benchmark[cuda-ljlk-backward-030]           5,172.7020 (0.17)           193.3226 (5.80)          4.9397 (18.68)  
res_centric_score_benchmark[cuda-ljlk-backward-100]           5,291.4930 (0.18)           188.9826 (5.67)          2.8871 (10.92)  
res_centric_score_benchmark[cuda-ljlk-forward-001]            6,199.6860 (0.21)           161.2985 (4.84)          7.6471 (28.91)  
res_centric_score_benchmark[cuda-ljlk-forward-003]            4,149.3384 (0.14)           241.0023 (7.23)          7.2001 (27.22)  
res_centric_score_benchmark[cuda-ljlk-forward-010]            1,947.0398 (0.06)           513.6002 (15.40)         4.4648 (16.88)  
res_centric_score_benchmark[cuda-ljlk-forward-030]              775.2810 (0.03)         1,289.8548 (38.67)         5.9046 (22.32)  
res_centric_score_benchmark[cuda-ljlk-forward-100]              248.5519 (0.01)         4,023.3041 (120.62)        6.8648 (25.95)  
res_centric_score_benchmark[cuda-ljlk-full-001]               2,734.9663 (0.09)           365.6352 (10.96)        34.3192 (129.75) 
res_centric_score_benchmark[cuda-ljlk-full-003]               2,582.3235 (0.09)           387.2481 (11.61)        11.2206 (42.42)  
res_centric_score_benchmark[cuda-ljlk-full-010]               1,210.3848 (0.04)           826.1835 (24.77)         8.2105 (31.04)  
res_centric_score_benchmark[cuda-ljlk-full-030]                 474.0573 (0.02)         2,109.4498 (63.24)         5.5749 (21.08)  
res_centric_score_benchmark[cuda-ljlk-full-100]                 150.7994 (0.01)         6,631.3267 (198.81)       11.4171 (43.17)  

(time in ms) 
combined_res_centric_score_benchmark[cuda-disulfide_elec_hbond_ljlk_lkb_omega_rama-backward-001]     2,012.8517 (1.0)          0.4968 (1.0)       0.0213 (5.84)   
combined_res_centric_score_benchmark[cuda-disulfide_elec_hbond_ljlk_lkb_omega_rama-backward-003]     1,688.2339 (0.84)         0.5923 (1.19)      0.0101 (2.78)   
combined_res_centric_score_benchmark[cuda-disulfide_elec_hbond_ljlk_lkb_omega_rama-backward-010]       814.0743 (0.40)         1.2284 (2.47)      0.0636 (17.43)  
combined_res_centric_score_benchmark[cuda-disulfide_elec_hbond_ljlk_lkb_omega_rama-backward-030]       302.7624 (0.15)         3.3029 (6.65)      0.0606 (16.62)  
combined_res_centric_score_benchmark[cuda-disulfide_elec_hbond_ljlk_lkb_omega_rama-backward-100]       106.1108 (0.05)         9.4241 (18.97)     0.0149 (4.09)   
combined_res_centric_score_benchmark[cuda-disulfide_elec_hbond_ljlk_lkb_omega_rama-forward-001]      1,798.0751 (0.89)         0.5562 (1.12)      0.0036 (1.0)    
combined_res_centric_score_benchmark[cuda-disulfide_elec_hbond_ljlk_lkb_omega_rama-forward-003]      1,695.0487 (0.84)         0.5900 (1.19)      0.0039 (1.06)   
combined_res_centric_score_benchmark[cuda-disulfide_elec_hbond_ljlk_lkb_omega_rama-forward-010]        853.9412 (0.42)         1.1710 (2.36)      0.0067 (1.84)   
combined_res_centric_score_benchmark[cuda-disulfide_elec_hbond_ljlk_lkb_omega_rama-forward-030]        353.2318 (0.18)         2.8310 (5.70)      0.0055 (1.52)   
combined_res_centric_score_benchmark[cuda-disulfide_elec_hbond_ljlk_lkb_omega_rama-forward-100]        115.5034 (0.06)         8.6578 (17.43)     0.0168 (4.60)   
combined_res_centric_score_benchmark[cuda-disulfide_elec_hbond_ljlk_lkb_omega_rama-full-001]           784.2963 (0.39)         1.2750 (2.57)      0.1755 (48.12)  
combined_res_centric_score_benchmark[cuda-disulfide_elec_hbond_ljlk_lkb_omega_rama-full-003]           731.0021 (0.36)         1.3680 (2.75)      0.0592 (16.23)  
combined_res_centric_score_benchmark[cuda-disulfide_elec_hbond_ljlk_lkb_omega_rama-full-010]           372.3608 (0.18)         2.6856 (5.41)      0.0123 (3.37)   
combined_res_centric_score_benchmark[cuda-disulfide_elec_hbond_ljlk_lkb_omega_rama-full-030]           144.7948 (0.07)         6.9063 (13.90)     0.0086 (2.36)   
combined_res_centric_score_benchmark[cuda-disulfide_elec_hbond_ljlk_lkb_omega_rama-full-100]            46.0915 (0.02)        21.6960 (43.67)     0.0504 (13.82)  

New:

(time in us) 
res_centric_score_benchmark[cpu-ljlk-backward-001]               42.4722 (0.00)        23,544.7958 (714.25)       34.7802 (137.30)
res_centric_score_benchmark[cpu-ljlk-backward-003]               14.2295 (0.00)        70,276.3721 (>1000.0)      26.4505 (104.42)
res_centric_score_benchmark[cpu-ljlk-backward-010]                4.3911 (0.00)       227,735.0992 (>1000.0)     122.3898 (483.14)
res_centric_score_benchmark[cpu-ljlk-backward-030]                1.6398 (0.00)       609,841.6753 (>1000.0)     626.5817 (>1000.0)
res_centric_score_benchmark[cpu-ljlk-backward-100]                0.5137 (0.00)     1,946,682.4077 (>1000.0)   5,518.5305 (>1000.0)
res_centric_score_benchmark[cpu-ljlk-forward-001]                60.3668 (0.00)        16,565.3961 (502.52)       44.5768 (175.97)
res_centric_score_benchmark[cpu-ljlk-forward-003]                23.8788 (0.00)        41,878.0841 (>1000.0)     158.6713 (626.37)
res_centric_score_benchmark[cpu-ljlk-forward-010]                 7.1713 (0.00)       139,444.6336 (>1000.0)      29.4428 (116.23)
res_centric_score_benchmark[cpu-ljlk-forward-030]                 2.6277 (0.00)       380,554.8996 (>1000.0)   4,265.0448 (>1000.0)
res_centric_score_benchmark[cpu-ljlk-forward-100]                 0.8482 (0.00)     1,178,923.8602 (>1000.0)   5,770.8640 (>1000.0)
res_centric_score_benchmark[cpu-ljlk-full-001]                   24.6876 (0.00)        40,506.2355 (>1000.0)   8,045.6343 (>1000.0)
res_centric_score_benchmark[cpu-ljlk-full-003]                    7.8291 (0.00)       127,728.3855 (>1000.0)  19,399.3244 (>1000.0)
res_centric_score_benchmark[cpu-ljlk-full-010]                    2.7103 (0.00)       368,960.0810 (>1000.0)     807.5498 (>1000.0)
res_centric_score_benchmark[cpu-ljlk-full-030]                    1.0052 (0.00)       994,789.5534 (>1000.0)   2,879.3178 (>1000.0)
res_centric_score_benchmark[cpu-ljlk-full-100]                    0.3189 (0.00)     3,135,887.1415 (>1000.0)  25,197.4240 (>1000.0)

res_centric_score_benchmark[cuda-ljlk-backward-001]           3,973.3809 (0.13)           251.6748 (7.63)         13.4278 (53.01)
res_centric_score_benchmark[cuda-ljlk-backward-003]           2,592.9300 (0.09)           385.6641 (11.70)        14.0835 (55.60)
res_centric_score_benchmark[cuda-ljlk-backward-010]           1,117.6120 (0.04)           894.7649 (27.14)         9.0785 (35.84)
res_centric_score_benchmark[cuda-ljlk-backward-030]             419.4987 (0.01)         2,383.7975 (72.31)        11.3100 (44.65)
res_centric_score_benchmark[cuda-ljlk-backward-100]             142.0562 (0.00)         7,039.4658 (213.55)       13.9661 (55.13)
res_centric_score_benchmark[cuda-ljlk-forward-001]            6,502.9265 (0.21)           153.7769 (4.66)          7.2066 (28.45)
res_centric_score_benchmark[cuda-ljlk-forward-003]            4,381.4324 (0.14)           228.2359 (6.92)          3.8333 (15.13)
res_centric_score_benchmark[cuda-ljlk-forward-010]            2,085.5387 (0.07)           479.4924 (14.55)         7.8250 (30.89)
res_centric_score_benchmark[cuda-ljlk-forward-030]              845.6566 (0.03)         1,182.5131 (35.87)         4.7330 (18.68)
res_centric_score_benchmark[cuda-ljlk-forward-100]              272.4629 (0.01)         3,670.2244 (111.34)        8.9984 (35.52)
res_centric_score_benchmark[cuda-ljlk-full-001]               2,067.1728 (0.07)           483.7525 (14.68)        27.4107 (108.21)
res_centric_score_benchmark[cuda-ljlk-full-003]               1,609.7171 (0.05)           621.2272 (18.85)        22.2027 (87.65)
res_centric_score_benchmark[cuda-ljlk-full-010]                 786.8443 (0.03)         1,270.8995 (38.55)         9.0692 (35.80)
res_centric_score_benchmark[cuda-ljlk-full-030]                 307.2419 (0.01)         3,254.7639 (98.74)         7.7905 (30.75)
res_centric_score_benchmark[cuda-ljlk-full-100]                  96.1419 (0.00)        10,401.2929 (315.53)        8.4415 (33.32)

(time in ms) 
combined_res_centric_score_benchmark[cuda-disulfide_elec_hbond_ljlk_lkb_omega_rama-backward-001]     1,735.6317 (0.98)         0.5762 (1.02)       0.0126 (3.46)
combined_res_centric_score_benchmark[cuda-disulfide_elec_hbond_ljlk_lkb_omega_rama-backward-003]     1,186.6749 (0.67)         0.8427 (1.50)       0.0190 (5.21)
combined_res_centric_score_benchmark[cuda-disulfide_elec_hbond_ljlk_lkb_omega_rama-backward-010]       467.0209 (0.26)         2.1412 (3.81)       0.1208 (33.18)
combined_res_centric_score_benchmark[cuda-disulfide_elec_hbond_ljlk_lkb_omega_rama-backward-030]       185.9254 (0.10)         5.3785 (9.56)       0.0264 (7.24)
combined_res_centric_score_benchmark[cuda-disulfide_elec_hbond_ljlk_lkb_omega_rama-backward-100]        62.1912 (0.03)        16.0795 (28.59)      0.0165 (4.53)
combined_res_centric_score_benchmark[cuda-disulfide_elec_hbond_ljlk_lkb_omega_rama-forward-001]      1,778.3381 (1.0)          0.5623 (1.0)        0.0036 (1.0)
combined_res_centric_score_benchmark[cuda-disulfide_elec_hbond_ljlk_lkb_omega_rama-forward-003]      1,730.5573 (0.97)         0.5778 (1.03)       0.0070 (1.92)
combined_res_centric_score_benchmark[cuda-disulfide_elec_hbond_ljlk_lkb_omega_rama-forward-010]        879.2208 (0.49)         1.1374 (2.02)       0.0064 (1.75)
combined_res_centric_score_benchmark[cuda-disulfide_elec_hbond_ljlk_lkb_omega_rama-forward-030]        367.7860 (0.21)         2.7190 (4.84)       0.0071 (1.96)
combined_res_centric_score_benchmark[cuda-disulfide_elec_hbond_ljlk_lkb_omega_rama-forward-100]        120.5783 (0.07)         8.2934 (14.75)      0.0124 (3.41)
combined_res_centric_score_benchmark[cuda-disulfide_elec_hbond_ljlk_lkb_omega_rama-full-001]           800.7832 (0.45)         1.2488 (2.22)       0.0408 (11.20)
combined_res_centric_score_benchmark[cuda-disulfide_elec_hbond_ljlk_lkb_omega_rama-full-003]           646.7408 (0.36)         1.5462 (2.75)       0.0456 (12.53)
combined_res_centric_score_benchmark[cuda-disulfide_elec_hbond_ljlk_lkb_omega_rama-full-010]           325.4509 (0.18)         3.0727 (5.46)       0.0198 (5.44)
combined_res_centric_score_benchmark[cuda-disulfide_elec_hbond_ljlk_lkb_omega_rama-full-030]           123.8594 (0.07)         8.0737 (14.36)      0.0105 (2.90)
combined_res_centric_score_benchmark[cuda-disulfide_elec_hbond_ljlk_lkb_omega_rama-full-100]            39.2704 (0.02)        25.4644 (45.28)      0.0200 (5.49)

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 96.99399% with 15 lines in your changes are missing coverage. Please review.

Project coverage is 95.20%. Comparing base (ace29a8) to head (7e23853). Report is 12 commits behind head on master.

Files	Patch %	Lines
tmol/tests/score/common/test_energy_term.py	93.81%	12 Missing :warning:
tmol/score/score_function.py	66.66%	3 Missing :warning:

Additional details and impacted files

```diff @@ Coverage Diff @@ ## master #264 +/- ## ========================================== + Coverage 94.93% 95.20% +0.27% ========================================== Files 374 385 +11 Lines 23883 25609 +1726 ========================================== + Hits 22673 24381 +1708 - Misses 1210 1228 +18 ``` | [Flag](https://app.codecov.io/gh/uw-ipd/tmol/pull/264/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=uw-ipd) | Coverage Δ | | |---|---|---| | [_shrug_Testing_CPU](https://app.codecov.io/gh/uw-ipd/tmol/pull/264/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=uw-ipd) | `90.74% <96.59%> (+0.71%)` | :arrow_up: | | [_shrug_Testing_CPU_debug_w_o_jit](https://app.codecov.io/gh/uw-ipd/tmol/pull/264/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=uw-ipd) | `92.43% <96.59%> (?)` | | | [_shrug_Testing_CPU_w_o_jit](https://app.codecov.io/gh/uw-ipd/tmol/pull/264/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=uw-ipd) | `?` | | | [_shrug_Testing_CUDA](https://app.codecov.io/gh/uw-ipd/tmol/pull/264/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=uw-ipd) | `92.81% <96.99%> (+0.43%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=uw-ipd#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[jflat06]

It looks like beta2016 is still referencing fa_lj instead of fa_ljatr and fa_ljrep, and this is causing some test to fail. I'm guessing beta2016 should just have both of these at weight 1.0?

[fdimaio]

It looks like beta2016 is still referencing fa_lj instead of fa_ljatr and fa_ljrep, and this is causing some test to fail. I'm guessing beta2016 should just have both of these at weight 1.0?

fa_ljatr: 1.0 fa_ljrep: 0.55

[jflat06]

Ok, this should be good for review. I've given it a once over, but might be good for others to take a look one last time. @aleaverfay @fdimaio