reserve validation does not check for duplicates

[ajtowns]

Hi,

There's no checks in validate_reserves.py to prevent double counting, eg:

height:
  779999
blockhash:
  000000000000000000063a840d1ae5a1090da46e1ae749bf668ba9f7b2a30efe
chain:
  main
claim:
  m: 3
  n: 4
total:
  1831904
keys:
  - 04a24db5c0e8ed34da1fd3b6f9f797244981b928a8750c8f11f9252041daad7b2d95309074fed791af77dc85abdd8bb2774ed8d53379d28cd49f251b9c08cab7fc
  - 04220936c3245597b1513a9a7fe96d96facf1a840ee21432a1b73c2cf42c1810284dd730f21ded9d818b84402863a2b5cd1afe3a3d13719d524482592fb23c88a3
  - 0472225d3abc8665cf01f703a270ee65be5421c6a495ce34830061eb0690ec27dfd1194e27b6b0b659418d9f91baec18923078aac18dc19699aae82583561fefe5
xpub:
  - xpub661MyMwAqRbcFvqQ14RrhJ5seDNrUeJGcaKYxmXwCfQKrCzUv8ScZDjaHoKvdjHuneaDQGGFbrozqw7JVoqHfBSs5i4igzf8zfUPxySeL6N
address:
  - {'addr_type': 'sh', 'addr': '3BMEXbSSrK2K7cRgqxrtqUWfxowBBrW1BE', 'script': '534104220936c3245597b1513a9a7fe96d96facf1a840ee21432a1b73c2cf42c1810284dd730f21ded9d818b84402863a2b5cd1afe3a3d13719d524482592fb23c88a3410472225d3abc8665cf01f703a270ee65be5421c6a495ce34830061eb0690ec27dfd1194e27b6b0b659418d9f91baec18923078aac18dc19699aae82583561fefe54104a24db5c0e8ed34da1fd3b6f9f797244981b928a8750c8f11f9252041daad7b2d95309074fed791af77dc85abdd8bb2774ed8d53379d28cd49f251b9c08cab7fc4104c026c274b9aa88e6ebbef0acb091ecf6cac7ab348d6379ed98fb04c55e7c6d88643a9cc0c7cc463ce9a09e4c9bfbf5c4d03acc447af38b479e6552320e9791cc54ae', 'balance': '171258'}
  - {'addr_type': 'sh', 'addr': '3BMEX95VgAacEZRJksocYrPzJ328pcSFXG', 'script': '534104220936c3245597b1513a9a7fe96d96facf1a840ee21432a1b73c2cf42c1810284dd730f21ded9d818b84402863a2b5cd1afe3a3d13719d524482592fb23c88a341042c3d5ab3a8a8a72e89f8fa9fad015af23fe32e43b30b27515921c308f66eabb62c2a349633e4af987927bdef94dff20a8abee7b99215600b19b1ef7a2f0546fe410472225d3abc8665cf01f703a270ee65be5421c6a495ce34830061eb0690ec27dfd1194e27b6b0b659418d9f91baec18923078aac18dc19699aae82583561fefe54104a24db5c0e8ed34da1fd3b6f9f797244981b928a8750c8f11f9252041daad7b2d95309074fed791af77dc85abdd8bb2774ed8d53379d28cd49f251b9c08cab7fc54ae', 'balance': '830323'}
  - {'addr_type': 'sh', 'addr': '3BMEX95VgAacEZRJksocYrPzJ328pcSFXG', 'script': '534104220936c3245597b1513a9a7fe96d96facf1a840ee21432a1b73c2cf42c1810284dd730f21ded9d818b84402863a2b5cd1afe3a3d13719d524482592fb23c88a341042c3d5ab3a8a8a72e89f8fa9fad015af23fe32e43b30b27515921c308f66eabb62c2a349633e4af987927bdef94dff20a8abee7b99215600b19b1ef7a2f0546fe410472225d3abc8665cf01f703a270ee65be5421c6a495ce34830061eb0690ec27dfd1194e27b6b0b659418d9f91baec18923078aac18dc19699aae82583561fefe54104a24db5c0e8ed34da1fd3b6f9f797244981b928a8750c8f11f9252041daad7b2d95309074fed791af77dc85abdd8bb2774ed8d53379d28cd49f251b9c08cab7fc54ae', 'balance': '830323'}

where 3BMEX95VgAacEZRJksocYrPzJ328pcSFXG is repeated gives no error. It does correctly report Proven amount(BTC): 0.01001581, however does not indicate this doesn't match the claimed 0.01831904 BTC. I think that due tochunk_size = 60000 you could successfully double count utxos and get an incorrect "Proven amount" by repeating a scriptPubKey in different chunks.

Requiring the "addresses" section be sorted (so that address[n].addr < address[n+1].addr would likely be a simple way of fixing this.

(I didn't observe any repeated addresses or repeated utxos in the 2023-03-09 PoR data)

[shuckc]

Thanks for reporting this. I have added a check for duplicates.

The total field in the input data is passed about a bit in the script, but never actually printed. We only emit the verified total from the utxo set. We should probably exit non-zero if this doesn't match though.