Data Mismatch - vegatools diff tool reports data mismatch for markets

[MuthuVega]

As part of all the system test pipelines we run the vegatools database diff tool to check if the data in the datanode matches with core.

The vegatools datadiff command reports a mismatch for markets, however looking at the data , looks like the data is correct BUT there are additional fields returned from core i.e. 13:"0.1" 14:"0.1" thats making the comparison to fail.

Output:

key=markets, matchResult=mismatching values, coreLength=1, datanodeLength=1, coreResult=id:"9219b88bc203438cb5d4f3be485987e643c0b4ac84e9344fe82123e1c04c8d75" tradable_instrument:{instrument:{code:"CRYPTO:VEGA CORE SYS TEST" name:"Futures System-Tests Market" metadata:{tags:"asset_class:fx/crypto" tags:"product:futures"} future:{settlement_asset:"5628fb7220cf0f1efd0a87fac4f796224b10315458b5d662a74331a63fdfee04" quote_name:"USD" data_source_spec_for_settlement_data:{id:"f2504b6fde5e2212841bff25f55681c7418674d9b69eb59acf62960481045145" data:{external:{oracle:{signers:{pub_key:{key:"0380fd84da09775e405de9847c95a9569678bef6d3f90edf6a06cf8e8637a385"}} filters:{key:{name:"oracle.test.settlementF77D8" type:TYPE_INTEGER number_decimal_places:5} conditions:{operator:OPERATOR_GREATER_THAN_OR_EQUAL value:"0"}}}}}} data_source_spec_for_trading_termination:{id:"f8b050a626ddcc9a5986f009c688362a9a4d45c6b15d7dcac34298d49bbff5bd" data:{external:{oracle:{signers:{pub_key:{key:"0380fd84da09775e405de9847c95a9569678bef6d3f90edf6a06cf8e8637a385"}} filters:{key:{name:"oracle.test.terminationD62CB" type:TYPE_BOOLEAN} conditions:{operator:OPERATOR_EQUALS value:"True"}}}}}} data_source_spec_binding:{settlement_data_property:"oracle.test.settlementF77D8" trading_termination_property:"oracle.test.terminationD62CB"}}} margin_calculator:{scaling_factors:{search_level:1.1 initial_margin:1.2 collateral_release:1.4}} log_normal_risk_model:{risk_aversion_parameter:0.001 tau:0.00011407711613050422 params:{r:0.016 sigma:1.5}}} decimal_places:5 fees:{factors:{maker_fee:"0.004" infrastructure_fee:"0.001" liquidity_fee:"0.3"}} opening_auction:{duration:11} price_monitoring_settings:{parameters:{}} liquidity_monitoring_parameters:{target_stake_parameters:{time_window:10 scaling_factor:5} triggering_ratio:"0" auction_extension:1} trading_mode:TRADING_MODE_CONTINUOUS state:STATE_ACTIVE market_timestamps:{proposed:1676373665171877534 pending:1676373675225927677 open:1676373687726171649} position_decimal_places:5 lp_price_range:"1" 13:"0.1" 14:"0.1"

[2023-02-14T11:29:40.199Z] , datanodeResult=id:"9219b88bc203438cb5d4f3be485987e643c0b4ac84e9344fe82123e1c04c8d75" tradable_instrument:{instrument:{code:"CRYPTO:VEGA CORE SYS TEST" name:"Futures System-Tests Market" metadata:{tags:"asset_class:fx/crypto" tags:"product:futures"} future:{settlement_asset:"5628fb7220cf0f1efd0a87fac4f796224b10315458b5d662a74331a63fdfee04" quote_name:"USD" data_source_spec_for_settlement_data:{id:"f2504b6fde5e2212841bff25f55681c7418674d9b69eb59acf62960481045145" data:{external:{oracle:{signers:{pub_key:{key:"0380fd84da09775e405de9847c95a9569678bef6d3f90edf6a06cf8e8637a385"}} filters:{key:{name:"oracle.test.settlementF77D8" type:TYPE_INTEGER number_decimal_places:5} conditions:{operator:OPERATOR_GREATER_THAN_OR_EQUAL value:"0"}}}}}} data_source_spec_for_trading_termination:{id:"f8b050a626ddcc9a5986f009c688362a9a4d45c6b15d7dcac34298d49bbff5bd" data:{external:{oracle:{signers:{pub_key:{key:"0380fd84da09775e405de9847c95a9569678bef6d3f90edf6a06cf8e8637a385"}} filters:{key:{name:"oracle.test.terminationD62CB" type:TYPE_BOOLEAN} conditions:{operator:OPERATOR_EQUALS value:"True"}}}}}} data_source_spec_binding:{settlement_data_property:"oracle.test.settlementF77D8" trading_termination_property:"oracle.test.terminationD62CB"}}} margin_calculator:{scaling_factors:{search_level:1.1 initial_margin:1.2 collateral_release:1.4}} log_normal_risk_model:{risk_aversion_parameter:0.001 tau:0.00011407711613050422 params:{r:0.016 sigma:1.5}}} decimal_places:5 fees:{factors:{maker_fee:"0.004" infrastructure_fee:"0.001" liquidity_fee:"0.3"}} opening_auction:{duration:11} price_monitoring_settings:{parameters:{}} liquidity_monitoring_parameters:{target_stake_parameters:{time_window:10 scaling_factor:5} triggering_ratio:"0" auction_extension:1} trading_mode:TRADING_MODE_CONTINUOUS state:STATE_ACTIVE market_timestamps:{proposed:1676373665171877534 pending:1676373675225927677 open:1676373687726171649} position_decimal_places:5 lp_price_range:"1"

Jenkins Runs:

https://jenkins.ops.vega.xyz/blue/organizations/jenkins/common%2Fsystem-tests-wrapper/detail/system-tests-wrapper/50448/pipeline

https://jenkins.ops.vega.xyz/blue/organizations/jenkins/common%2Fsystem-tests-wrapper/detail/system-tests-wrapper/50465/pipeline/

System Tests Test Name - test_too_many_transactions_in_same_block_with_pow_difficulty_switched_off

[ze97286]

this is true but belongs in vegatools repo not here as it's an issue with the tool not with the core/datanode.