search

vegaprotocol / vega

A Go implementation of the Vega Protocol, a protocol for creating and trading derivatives on a fully decentralised network.
https://vega.xyz
GNU Affero General Public License v3.0
37 stars 22 forks source link

Core crashes when closing out traders in Monitoring Auction #2385

Closed ponthief closed 4 years ago

ponthief commented 4 years ago

Problem encountered

Core crashes when it needs to close out traders who are distressed. Market goes into Monitoring Auction and crashes.

System response

2020-10-13T09:20:54.182Z    ERROR   tm.abci.socket-server   server/socket_server.go:142 Connection error [err recovered from panic: Trade.size > order.remaining
goroutine 218 [running]:
github.com/tendermint/tendermint/abci/server.(*SocketServer).handleRequests.func1(0xc000582840, 0xc00058f2c0)
    /home/djordje/go/src/vega/vendor/github.com/tendermint/tendermint/abci/server/socket_server.go:165 +0xb5
panic(0x182e500, 0x1f438e0)
    /usr/local/go/src/runtime/panic.go:969 +0x166
code.vegaprotocol.io/vega/matching.(*PriceLevel).uncross(0xc01def4990, 0xc01df8c0c0, 0x2d91240, 0x1, 0xc01d9166e0, 0x1, 0x1, 0x1, 0x1, 0x0, ...)
    /home/djordje/go/src/vega/matching/pricelevel.go:121 +0x7b5
code.vegaprotocol.io/vega/matching.(*OrderBookSide).uncross(0xc01d5d7f00, 0xc01df8c0c0, 0xc01d9166e8, 0x1, 0x1, 0xc01d9166f0, 0x1, 0x1, 0x96, 0x0, ...)
    /home/djordje/go/src/vega/matching/side.go:494 +0x34b
code.vegaprotocol.io/vega/matching.(*OrderBook).uncrossBook(0xc000234f50, 0xc01d898b10, 0xc0004c6298, 0xa418fc, 0x1ad7480, 0xc01d898b10)
    /home/djordje/go/src/vega/matching/orderbook.go:408 +0x5d1
code.vegaprotocol.io/vega/matching.(*OrderBook).LeaveAuction(0xc000234f50, 0x1faf1c0, 0xc01d898990, 0x75dc2a9, 0xed7176873, 0x0, 0xc01d898b10, 0xc0004c6370, 0x128f998)
    /home/djordje/go/src/vega/matching/orderbook.go:200 +0x44
code.vegaprotocol.io/vega/execution.(*Market).LeaveAuction(0xc01d352680, 0x1faf1c0, 0xc01d898990, 0x75dc2a9, 0xed7176873, 0x0)
    /home/djordje/go/src/vega/execution/market.go:481 +0xfe
code.vegaprotocol.io/vega/execution.(*Market).OnChainTimeUpdate(0xc01d352680, 0x1faf1c0, 0xc01d898990, 0x75dc2a9, 0xed7176873, 0x0, 0x600)
    /home/djordje/go/src/vega/execution/market.go:372 +0x15ca
code.vegaprotocol.io/vega/execution.(*Engine).onChainTimeUpdate(0xc000186fc0, 0x1faf1c0, 0xc01d898990, 0x75dc2a9, 0xed7176873, 0x0)
    /home/djordje/go/src/vega/execution/engine.go:406 +0x534
code.vegaprotocol.io/vega/vegatime.(*Svc).notify(0xc000552000, 0x1faf1c0, 0xc01d898990, 0x75dc2a9, 0xed7176873, 0x0)
    /home/djordje/go/src/vega/vegatime/service.go:73 +0xc9
code.vegaprotocol.io/vega/vegatime.(*Svc).SetTimeNow(0xc000552000, 0x1faf1c0, 0xc01d898990, 0x75dc2a9, 0xed7176873, 0x0)
    /home/djordje/go/src/vega/vegatime/service.go:48 +0x170
code.vegaprotocol.io/vega/processor.(*App).OnBeginBlock(0xc01d7a0000, 0xc01d9809a0, 0x20, 0x20, 0xa, 0x0, 0xc01d067a60, 0xc, 0x20d, 0x75dc2a9, ...)
    /home/djordje/go/src/vega/processor/abci.go:197 +0x295
code.vegaprotocol.io/vega/blockchain/abci.(*App).BeginBlock(0xc0003f8bd0, 0xc01d9809a0, 0x20, 0x20, 0xa, 0x0, 0xc01d067a60, 0xc, 0x20d, 0x75dc2a9, ...)
    /home/djordje/go/src/vega/blockchain/abci/abci.go:44 +0xad
github.com/tendermint/tendermint/abci/server.(*SocketServer).handleRequest(0xc000582840, 0xc01d0a7a10, 0xc00058f320)
    /home/djordje/go/src/vega/vendor/github.com/tendermint/tendermint/abci/server/socket_server.go:222 +0xb4a
github.com/tendermint/tendermint/abci/server.(*SocketServer).handleRequests(0xc000582840, 0xc00058f2c0, 0x1f7da00, 0xc000010c28, 0xc00058f320)
    /home/djordje/go/src/vega/vendor/github.com/tendermint/tendermint/abci/server/socket_server.go:189 +0x173
created by github.com/tendermint/tendermint/abci/server.(*SocketServer).acceptConnectionsRoutine
    /home/djordje/go/src/vega/vendor/github.com/tendermint/tendermint/abci/server/socket_server.go:127 +0x2b3
 "data": {
        "markPrice": "75000",
        "bestBidPrice": "3000",
        "bestBidVolume": "3",
        "bestOfferPrice": "100",
        "midPrice": "1550",
        "openInterest": "11",
        "marketState": "MONITORING_AUCTION"
      }

System version: Specify the system version (0.26 - branch - 1962) Protocol: Specify the protocol

Component: Specify the components that might be related

Steps to reproduce

Automation

Execute all of the tests up to and including: https://github.com/vegaprotocol/system-tests/blob/master/grpcTesting/bvt/buildVerify_test.py#L551

edd commented 4 years ago

@3jtechtest does this block merging of #2229

ponthief commented 4 years ago

@edd Yes Also, I have just found out that this is also happening in the latest develop branch, so not an issue related to the pricemonitoring changes.