Mismatched gas limit between derived_gas_outputs and message_gas_economy

[birdychang]

Theoretically message_gas_economy.gas_limit_unique_total should match the sum of gas_limit from derived_gas_outputs, but sometimes, the former is larger.

SELECT 
    A.height,
    B.gas_limit_unique_total as glu,
    B.gas_limit_total as glt,
    SUM(A.gas_limit) as gl
FROM derived_gas_outputs AS A

JOIN message_gas_economy AS B
  ON A.height=B.height 

WHERE A.height between 2600000 and 2600100
GROUP BY 1,2,3
ORDER BY 1 ASC
limit 100

2600000 22395070866 29981896531 22049049943
2600001 22288409072 39981662198 22288409072
2600002 23071139501 39981178557 22300998573
2600003 18626218944 19976805837 18593796122
2600004 17559907062 49977169021 17420526387
2600005 17102443373 49907574933 16892686773
2600006 18846055147 79831937624 17920590231
2600007 24215221840 78399445008 24215221840
2600008 19212720498 26659009104 19212720498
2600009 9985075770  9985075770  9985075770
2600010 22269696807 49952870317 20776739632
2600011 27594766807 99927097628 27494589334
2600012 14320312129 19962125646 13106339292
2600013 23022656021 69912223463 23022656021
2600014 9989198713  9989198713  9989198713
2600015 20053482302 29982031714 17000206341
2600016 26835641076 29981600933 26835641076
2600017 15466298121 19984455360 11503908819
2600018 9989599498  9989599498  9989599498
2600019 28683980900 39961945898 28683980900
2600020 34934496205 89933836516 33882759195
2600021 25322897811 49953028847 23155072901
2600022 22517824552 79727106735 20874166844
2600023 22320424733 49955556802 19685025644
2600024 16284116029 19981366034 16284116029
2600025 26692474629 29985398376 26692474629
2600026 24519008076 39970294119 23504422859
2600027 19127965142 19984135782 18950433732
2600028 24918976066 99957139990 22975444273
2600029 7494716531  14895006599 7494716531
2600030 27919192851 59979008269 27322856925
2600031 19323085809 49473704684 18622150911
2600032 31780759227 79953193710 31557994633
2600033 23565106478 89902182337 23312332267
2600034 20803957611 39979921053 20594337271
2600035 9981989380  9981989380  9981989380
2600036 21130692993 69959065748 19846394219
2600037 12911217386 19988460467 12911217386
2600038 17599941637 58987664201 17316005707
2600039 19375786973 49527867970 18487901021
2600040 12126947169 24156331749 12126947169
2600041 16386224416 45585157142 16241825890
2600042 18084409487 19996799563 18084409487
2600043 33635130240 89955001680 33576605358
2600044 20470144480 47078957063 18935490649
2600045 14564693132 43310264839 14564693132
2600046 20883533610 56299602312 20795955397
2600047 9742695473  26261639716 9583696707
2600048 17827164502 49940133737 17690624478
2600049 34276823380 79970080732 33844719222
2600050 17488508057 38040736415 17400360395
2600051 19013427754 44171071165 18760954551
2600052 12858620561 18263964333 12858620561
2600053 23411533660 39979972741 22420770885
2600054 16010363115 38778055456 15867015375
2600055 15360800489 32122019212 13986482706
2600056 19919713087 49930701270 19919713087
2600057 17852673953 50579598156 16484396474
2600058 26007310263 49897835290 24276452483
2600059 25875227309 59823547131 25732322409
2600060 12478321246 42129645255 12295147332
2600061 17203583797 36236982646 17177219357
2600062 9284826318  35250507129 9284826318
2600063 20987746956 29980383059 20926871648
2600064 27324491559 89967811975 25668996926
2600065 18918939543 49889624466 18837296811
2600066 26395549089 79955199347 26132299653
2600067 21522796176 56051470189 20270416866
2600068 20692154497 39961756415 19955692234
2600069 24271817576 49850584820 24138406182
2600070 17635170331 39346933420 17635170331
2600071 17654180498 49330531444 17583564012
2600072 21034808766 39971411431 20978334664
2600073 18686248397 39968033347 18686248397
2600074 12089527172 22995155324 12089527172
2600075 24853479101 59955475057 24789607421
2600076 14529457396 37925898399 14485770309
2600077 20684490869 79942585839 20684490869
2600078 6332407430  6332407430  6332407430
2600079 25936627057 59967336086 25936627057
2600080 20096388454 69940597119 19881017153
2600081 16439288236 29944959548 16439288236
2600082 34125709783 69964230266 25564837421
2600083 8934073374  8934073374  8934073374
2600084 12947158680 19991486625 12882157372
2600085 16428767478 19992750483 16428767478
2600086 13806227826 19985222012 13512280804
2600087 23690765115 39963701301 22487137018
2600088 21647997601 39942697675 20842474356
2600089 23064122019 79899080679 22796205919
2600090 9775046237  44434243538 9775046237
2600091 16365692893 29968321949 16365692893
2600092 33997199098 59957275407 24528993081
2600093 25595088746 39971540946 25595088746
2600094 25457535046 69947233763 25053749989
2600095 25486532943 69932095685 25438027172
2600096 16672585768 69084353087 16147241695
2600097 19721550101 29924973875 19612636029
2600098 17508392701 66594924967 17354195503
2600099 23981811823 61644420771 23908627842

[birdychang]

Notes from @Terryhung

derived_gas_outputs -> TipSetMessageReceipts -> lotus code: BlockMsgsForTipset: 
after MessagesForBlock got messages (BlsMessages & SecpkMessages), it uses selectMsg to filter message before returning

message_gas_economy -> TipSetBlockMessages -> MessagesForTipSetBlocks 
-> lotus code : ChainGetBlockMessages ->  MessagesForBlock to get messages (BlsMessages & SecpkMessages)

[Terryhung]

The root cause is that we inadvertently duplicated the calculation of messages with nonce duplication in message_gas_economy, causing a mismatch between message_gas_economy and derived_gas_output.

[birdychang]

Here's an example of such message:

SELECT *
FROM visor.messages
WHERE height=3268061 AND "from"='f3wt7zk5zvyyfi2gh5lth4eav2qogeyobr2w6v4w3iafcxzn7hwcdsx5m2zxkwr2u2a6cpkxkb55mdhpiyyv6q'

cid	to	nonce	gas_limit
bafy2bzacebxegun7jrajoxoq66amvxjodoiklraht66qqvvlqjtarasheru36	f02252173	67581	60934215
bafy2bzaceazdl3h5d32ph2fhbcv3a6dye56jybb3mjcexmszytjoclcgkmtqc	f02252173	67581	60934215