panic when getting Voucher during data transfer

[stuberman]

Checklist

• [X] This is not a question or a support request. If you have any boost related questions, please ask in the discussion forum.
• [X] This is not a new feature or enhancement request. If it is, please open a new idea discussion instead. New feature and enhancement requests would be entertained by the boost team after a thorough discussion only.
• [X] I have searched on the issue tracker and the discussion forum, and there is no existing related issue or discussion.
• [X] I am running the Latest release, or the most recent RC(release canadiate) for the upcoming release or the dev branch(master), or have an issue updating to any of these.
• [X] I did not make any code changes to boost.

Boost component

• [ ] boost daemon - storage providers
• [ ] boost client
• [ ] boost UI
• [X] boost data-transfer
• [ ] boost index-provider
• [ ] Other

Boost Version

boostd version 1.5.1-rc2+git.b5768c8
Daemon:  1.19.0+mainnet+git.64059ca87+api1.5.0
Local: lotus-miner version 1.19.0+mainnet+git.64059ca87

Describe the Bug

Deals from Estuary fail to download/transfer

Logging Information

2022-12-20T23:13:23.474Z    INFO    boost-storage-deal  logs/log.go:40  executing deal proposal received from network   {"id": "33316998-f823-4ada-920a-ba9cf5b442de", "peer": "12D3KooWCVXs8P7iq6ao4XhfAmKWrEeuKFWCJgqe9jGDMTqHYBjw"}
2022-12-20T23:13:23.510Z    INFO    boost-storage-deal  logs/log.go:40  processing deal acceptance request  {"id": "33316998-f823-4ada-920a-ba9cf5b442de"}
2022-12-20T23:13:24.215Z    INFO    boost-storage-deal  logs/log.go:40  tagged funds for deal   {"id": "33316998-f823-4ada-920a-ba9cf5b442de", "tagged for deal publish": "150000000000000000", "tagged for deal collateral": "4832551957390968", "total tagged for publish": "300000000000000000", "total tagged for collateral": "13536750656746547", "total available for publish": "49708386566633845217", "total available for collateral": "25188471048120140580"}
2022-12-20T23:13:24.226Z    INFO    boost-storage-deal  logs/log.go:40  created deal download staging file  {"id": "33316998-f823-4ada-920a-ba9cf5b442de", "path": "/market/.boost/incoming/33316998-f823-4ada-920a-ba9cf5b442de.download"}
2022-12-20T23:13:24.235Z    INFO    boost-storage-deal  logs/log.go:40  inserted deal into deals DB {"id": "33316998-f823-4ada-920a-ba9cf5b442de"}
2022-12-20T23:13:24.239Z    INFO    boost-storage-deal  logs/log.go:40  deal accepted and scheduled for execution   {"id": "33316998-f823-4ada-920a-ba9cf5b442de"}
2022-12-20T23:13:24.239Z    INFO    boost-storage-deal  logs/log.go:40  deal execution initiated    {"id": "33316998-f823-4ada-920a-ba9cf5b442de", "deal state": {"DealUuid":"33316998-f823-4ada-920a-ba9cf5b442de","CreatedAt":"2022-12-20T23:13:24.230808929Z","ClientDealProposal":{"Proposal":{"PieceCID":{"/":"baga6ea4seaqc7yeevk2lkx5ci3rc4keh7vhq6xg4kcifkfqdnoklzy3n4v3uyba"},"PieceSize":17179869184,"VerifiedDeal":true,"Client":"f3vnq2cmwig3qjisnx5hobxvsd4drn4f54xfxnv4tciw6vnjdsf5xipgafreprh5riwmgtcirpcdmi3urbg36a","Provider":"f01278","Label":"mAXGg5AIgQ60MPUJb1DOmMbtWkzpPuNSxPPjP1MemAvt+5n8guCo","StartEpoch":2462546,"EndEpoch":3957266,"StoragePricePerEpoch":"0","ProviderCollateral":"4832551957390968","ClientCollateral":"0"},"ClientSignature":{"Type":0,"Data":null}},"IsOffline":false,"ClientPeerID":"12D3KooWCVXs8P7iq6ao4XhfAmKWrEeuKFWCJgqe9jGDMTqHYBjw","DealDataRoot":{"/":"bafy2bzacebb22db5ijn5im5ggg5vnez2j64njmj47dh5jr5gal5x5zt7ec4cu"},"InboundFilePath":"/market/.boost/incoming/33316998-f823-4ada-920a-ba9cf5b442de.download","Transfer":{"Type":"libp2p","ClientID":"5354860","Params":null,"Size":9438568839},"ChainDealID":0,"PublishCID":null,"SectorID":0,"Offset":0,"Length":0,"Checkpoint":0,"CheckpointAt":"2022-12-20T23:13:24.23080915Z","Err":"","Retry":"auto","NBytesReceived":0,"FastRetrieval":false}}
2022-12-20T23:13:24.243Z    INFO    boost-prop  lp2pimpl/net.go:212 send deal proposal response {"id": "33316998-f823-4ada-920a-ba9cf5b442de", "accepted": true, "msg": "", "peer id": "12D3KooWCVXs8P7iq6ao4XhfAmKWrEeuKFWCJgqe9jGDMTqHYBjw", "client address": "f3vnq2cmwig3qjisnx5hobxvsd4drn4f54xfxnv4tciw6vnjdsf5xipgafreprh5riwmgtcirpcdmi3urbg36a", "provider address": "f01278", "piece cid": "baga6ea4seaqc7yeevk2lkx5ci3rc4keh7vhq6xg4kcifkfqdnoklzy3n4v3uyba", "piece size": 17179869184, "verified": true, "label": "mAXGg5AIgQ60MPUJb1DOmMbtWkzpPuNSxPPjP1MemAvt+5n8guCo", "start epoch": "2462546", "end epoch": "3957266", "price per epoch": "0"}
2022-12-20T23:13:24.252Z    INFO    boost-storage-deal  logs/log.go:40  size of downloaded file {"id": "33316998-f823-4ada-920a-ba9cf5b442de", "filepath": "/market/.boost/incoming/33316998-f823-4ada-920a-ba9cf5b442de.download", "size": 0}
2022-12-20T23:13:24.257Z    INFO    boost-storage-deal  logs/log.go:40  deal execution in progress  {"id": "33316998-f823-4ada-920a-ba9cf5b442de"}
2022-12-20T23:13:24.263Z    INFO    boost-storage-deal  logs/log.go:40  deal queued for transfer    {"id": "33316998-f823-4ada-920a-ba9cf5b442de", "transfer client id": "5354860"}
2022-12-20T23:13:52.006Z    INFO    boost-storage-deal  logs/log.go:40  start deal data transfer    {"id": "33316998-f823-4ada-920a-ba9cf5b442de", "transfer client id": "5354860"}
2022-12-20T23:19:24.582Z    ERROR   rpc go-jsonrpc@v0.0.0-20221118154859-f3f8e9b1cb11/handler.go:189    panic in rpc method 'Filecoin.MarketListDataTransfers': runtime error: invalid memory address or nil pointer dereference
github.com/filecoin-project/go-jsonrpc.doCall.func1
    /home/stuart/go/pkg/mod/github.com/nonsense/go-jsonrpc@v0.0.0-20221118154859-f3f8e9b1cb11/handler.go:189
runtime.gopanic
    /usr/local/go/src/runtime/panic.go:838
runtime.panicmem
    /usr/local/go/src/runtime/panic.go:220
runtime.sigpanic
    /usr/local/go/src/runtime/signal_unix.go:818
github.com/filecoin-project/go-data-transfer/channels.channelState.Voucher
    /home/stuart/go/pkg/mod/github.com/filecoin-project/go-data-transfer@v1.15.2/channels/channel_state.go:105
github.com/filecoin-project/lotus/api.NewDataTransferChannel
    /home/stuart/go/pkg/mod/github.com/filecoin-project/lotus@v1.18.0/api/types.go:108
github.com/filecoin-project/boost/node/impl.(*BoostAPI).MarketListDataTransfers
    /home/stuart/boost/node/impl/boost_legacy.go:33
reflect.Value.call
    /usr/local/go/src/reflect/value.go:556
reflect.Value.Call
    /usr/local/go/src/reflect/value.go:339
github.com/filecoin-project/boost/metrics/proxy.proxy.func1
    /home/stuart/boost/metrics/proxy/proxy.go:37
github.com/filecoin-project/boost/api.(*BoostStruct).MarketListDataTransfers
    /home/stuart/boost/api/proxy_gen.go:664
reflect.Value.call
    /usr/local/go/src/reflect/value.go:556
reflect.Value.Call
    /usr/local/go/src/reflect/value.go:339
github.com/filecoin-project/go-jsonrpc/auth.PermissionedProxy.func1
    /home/stuart/go/pkg/mod/github.com/nonsense/go-jsonrpc@v0.0.0-20221118154859-f3f8e9b1cb11/auth/auth.go:62
github.com/filecoin-project/boost/api.(*BoostStruct).MarketListDataTransfers
    /home/stuart/boost/api/proxy_gen.go:664
reflect.Value.call
    /usr/local/go/src/reflect/value.go:556
reflect.Value.Call
    /usr/local/go/src/reflect/value.go:339
github.com/filecoin-project/go-jsonrpc.doCall
    /home/stuart/go/pkg/mod/github.com/nonsense/go-jsonrpc@v0.0.0-20221118154859-f3f8e9b1cb11/handler.go:193
github.com/filecoin-project/go-jsonrpc.(*RPCServer).handle
    /home/stuart/go/pkg/mod/github.com/nonsense/go-jsonrpc@v0.0.0-20221118154859-f3f8e9b1cb11/handler.go:314
2022-12-20T23:19:24.583Z    ERROR   rpc go-jsonrpc@v0.0.0-20221118154859-f3f8e9b1cb11/server.go:97  RPC Error: fatal error calling 'Filecoin.MarketListDataTransfers': panic in rpc method 'Filecoin.MarketListDataTransfers': runtime error: invalid memory address or nil pointer dereference

Repo Steps

Boost log from storage deals UI

CreatedAt	2022-12-20 18:13:24.230 (15m ago)

f3vnq2cmwig3qjisnx5hobxvsd4drn4f54xfxnv4tciw6vnjdsf5xipgafreprh5riwmgtcirpcdmi3urbg36a 12D3KooWCVXs8P7iq6ao4XhfAmKWrEeuKFWCJgqe9jGDMTqHYBjw bafyreibli4jlspyon2oxvqgnnl7rdjheqxw7scvlnfyfd5slnbody2smem bafy2bzacebb22db5ijn5im5ggg5vnez2j64njmj47dh5jr5gal5x5zt7ec4cu Yes baga6ea4seaqc7yeevk2lkx5ci3rc4keh7vhq6xg4kcifkfqdnoklzy3n4v3uyba 16 GiB (17,179,869,184 bytes) 0 atto 0.005 FIL 0 atto 2,442,417 2,462,546 (in 7 days) 3,957,266 (in a year) 1,494,720 (519 days) 0 atto (Price per epoch x Piece Size x Duration) Online libp2p 8.8 GiB (9,438,568,839 bytes) URL: libp2p:///ip4/147.28.156.33/tcp/6745/p2p/12D3KooWQt6XqZ5XMkUiSqnfekep2X5o1GQxxKkKWj2mg1QYBXco 61 B (61 bytes) /market/.boost/incoming/33316998-f823-4ada-920a-ba9cf5b442de.download     Accepted (15m ago) Transfer stalled at 0%

CreatedAt 2022-12-20 18:13:24.230 (15m ago) Client Address f3vnq2cmwig3qjisnx5hobxvsd4drn4f54xfxnv4tciw6vnjdsf5xipgafreprh5riwmgtcirpcdmi3urbg36a Client Peer ID 12D3KooWCVXs8P7iq6ao4XhfAmKWrEeuKFWCJgqe9jGDMTqHYBjw Signed Proposal CID bafyreibli4jlspyon2oxvqgnnl7rdjheqxw7scvlnfyfd5slnbody2smem Deal Data Root CID bafy2bzacebb22db5ijn5im5ggg5vnez2j64njmj47dh5jr5gal5x5zt7ec4cu Verified Yes Piece CID baga6ea4seaqc7yeevk2lkx5ci3rc4keh7vhq6xg4kcifkfqdnoklzy3n4v3uyba Piece Size 16 GiB (17,179,869,184 bytes) Client Collateral 0 atto Provider Collateral 0.005 FIL Storage Price / epoch / GiB 0 atto Current Epoch 2,442,417 Start Epoch 2,462,546 (in 7 days) End Epoch 3,957,266 (in a year) Duration 1,494,720 (519 days) Storage Fee 0 atto (Price per epoch x Piece Size x Duration) Transfer Mode Online Transfer Type libp2p Transfer Size 8.8 GiB (9,438,568,839 bytes) Transfer Params URL: libp2p:///ip4/147.28.156.33/tcp/6745/p2p/12D3KooWQt6XqZ5XMkUiSqnfekep2X5o1GQxxKkKWj2mg1QYBXco Transferred 61 B (61 bytes) Inbound File Path /market/.boost/incoming/33316998-f823-4ada-920a-ba9cf5b442de.download Publish Message CID Chain Deal ID
Checkpoint Accepted (15m ago) Status Transfer stalled at 0% Cancel Transfer Deal Logs 2022-12-20 18:13:23.474
executing deal proposal received from network peer: 12D3KooWCVXs8P7iq6ao4XhfAmKWrEeuKFWCJgqe9jGDMTqHYBjw 2022-12-20 18:13:23.510 36ms
processing deal acceptance request 2022-12-20 18:13:24.215 705ms
tagged funds for deal tagged for deal collateral: 4,832,551,957,390,968 tagged for deal publish: 150,000,000,000,000,000 total available for collateral: 25,188,471,048,120,140,580 total available for publish: 49,708,386,566,633,845,217 total tagged for collateral: 13,536,750,656,746,547 total tagged for publish: 300,000,000,000,000,000 2022-12-20 18:13:24.226 11ms
created deal download staging file path: /market/.boost/incoming/33316998-f823-4ada-920a-ba9cf5b442de.download 2022-12-20 18:13:24.235 9ms inserted deal into deals DB 2022-12-20 18:13:24.239 4ms deal accepted and scheduled for execution 2022-12-20 18:13:24.239 0ms deal execution initiated deal state: 2022-12-20 18:13:24.252 13ms
size of downloaded file filepath: /market/.boost/incoming/33316998-f823-4ada-920a-ba9cf5b442de.download size: 0 2022-12-20 18:13:24.257 5ms deal execution in progress 2022-12-20 18:13:24.263 6ms deal queued for transfer transfer client id: 5,354,860 2022-12-20 18:13:52.006 28s start deal data transfer transfer client id: 5,354,860 2022-12-20 18:13:52.008 2ms http-transport: execute transfer deal size: 9,438,568,839 output file: /market/.boost/incoming/33316998-f823-4ada-920a-ba9cf5b442de.download time before context deadline: 23h59m59.999992106s 2022-12-20 18:13:52.013 5ms http-transport: existing file size deal size: 9,438,568,839 file size: 0 2022-12-20 18:13:52.017 4ms http-transport: libp2p-http url multiaddr: /ip4/147.28.156.33/tcp/6745 peer id: 12D3KooWQt6XqZ5XMkUiSqnfekep2X5o1GQxxKkKWj2mg1QYBXco url: libp2p://12D3KooWQt6XqZ5XMkUiSqnfekep2X5o1GQxxKkKWj2mg1QYBXco 2022-12-20 18:13:52.020 3ms http-transport: started async http transfer 2022-12-20 18:13:52.021 1ms http-transport: sending http request range-rq: bytes=0- received: 0 remaining: 9,438,568,839 2022-12-20 18:13:53.838 2s
http-transport: http request error http code: 0 outputErr: error reading from http response stream: unexpected EOF 2022-12-20 18:13:53.845 7ms http-transport: some data was transferred before connection error, so resetting backoff to zero transferred: 61 2022-12-20 18:13:53.847 2ms http-transport: backing off before retrying http request attempts: 1 backoff time: 5s 2022-12-20 18:13:58.848 5s
http-transport: back-off complete, retrying http request backoff time: 5s 2022-12-20 18:13:58.855 7ms http-transport: sending http request range-rq: bytes=61- received: 61 remaining: 9,438,568,778 2022-12-20 18:13:59.524 669ms
http-transport: http request error http code: 0 outputErr: error reading from http response stream: unexpected EOF 2022-12-20 18:13:59.530 6ms http-transport: backing off before retrying http request attempts: 2 backoff time: 7.233891231s 2022-12-20 18:14:06.764 7s
http-transport: back-off complete, retrying http request backoff time: 7.233891231s 2022-12-20 18:14:06.771 7ms http-transport: sending http request range-rq: bytes=61- received: 61 remaining: 9,438,568,778 2022-12-20 18:14:07.438 667ms
http-transport: http request error http code: 0 outputErr: error reading from http response stream: unexpected EOF 2022-12-20 18:14:07.445 7ms http-transport: backing off before retrying http request attempts: 3 backoff time: 7.147743309s 2022-12-20 18:14:14.593 7s
http-transport: back-off complete, retrying http request backoff time: 7.147743309s 2022-12-20 18:14:14.600 7ms http-transport: sending http request range-rq: bytes=61- received: 61 remaining: 9,438,568,778 2022-12-20 18:14:15.267 667ms
http-transport: http request error http code: 0 outputErr: error reading from http response stream: unexpected EOF 2022-12-20 18:14:15.274 7ms http-transport: backing off before retrying http request attempts: 4 backoff time: 10.766629557s 2022-12-20 18:14:26.041 11s http-transport: back-off complete, retrying http request backoff time: 10.766629557s 2022-12-20 18:14:26.048 7ms http-transport: sending http request range-rq: bytes=61- received: 61 remaining: 9,438,568,778 2022-12-20 18:14:26.716 668ms
http-transport: http request error http code: 0 outputErr: error reading from http response stream: unexpected EOF 2022-12-20 18:14:26.723 7ms http-transport: backing off before retrying http request attempts: 5 backoff time: 15.716676867s 2022-12-20 18:14:42.440 16s http-transport: back-off complete, retrying http request backoff time: 15.716676867s 2022-12-20 18:14:42.447 7ms http-transport: sending http request range-rq: bytes=61- received: 61 remaining: 9,438,568,778 2022-12-20 18:14:43.115 668ms
http-transport: http request error http code: 0 outputErr: error reading from http response stream: unexpected EOF 2022-12-20 18:14:43.121 6ms http-transport: backing off before retrying http request attempts: 6 backoff time: 21.43935264s 2022-12-20 18:15:04.561 21s http-transport: back-off complete, retrying http request backoff time: 21.43935264s 2022-12-20 18:15:04.567 6ms http-transport: sending http request range-rq: bytes=61- received: 61 remaining: 9,438,568,778 2022-12-20 18:15:15.824 11s http-transport: http request error http code: 0 outputErr: error reading from http response stream: unexpected EOF 2022-12-20 18:15:15.830 6ms http-transport: backing off before retrying http request attempts: 7 backoff time: 5.772288846s 2022-12-20 18:15:21.603 6s
http-transport: back-off complete, retrying http request backoff time: 5.772288846s 2022-12-20 18:15:21.610 7ms http-transport: sending http request range-rq: bytes=61- received: 61 remaining: 9,438,568,778 2022-12-20 18:15:22.277 667ms
http-transport: http request error http code: 0 outputErr: error reading from http response stream: unexpected EOF 2022-12-20 18:15:22.284 7ms http-transport: backing off before retrying http request attempts: 8 backoff time: 1m15.717418996s 2022-12-20 18:16:38.001 1m
http-transport: back-off complete, retrying http request backoff time: 1m15.717418996s 2022-12-20 18:16:38.008 7ms http-transport: sending http request range-rq: bytes=61- received: 61 remaining: 9,438,568,778 2022-12-20 18:16:51.952 14s http-transport: http request error http code: 0 outputErr: error reading from http response stream: unexpected EOF 2022-12-20 18:16:51.956 4ms http-transport: backing off before retrying http request attempts: 9 backoff time: 1m52.829222054s 2022-12-20 18:18:44.785 2m
http-transport: back-off complete, retrying http request backoff time: 1m52.829222054s 2022-12-20 18:18:44.792 7ms http-transport: sending http request range-rq: bytes=61- received: 61 remaining: 9,438,568,778 2022-12-20 18:18:45.928 1s
http-transport: http request error http code: 0 outputErr: error reading from http response stream: unexpected EOF 2022-12-20 18:18:45.932 4ms http-transport: backing off before retrying http request attempts: 10 backoff time: 1m38.411623696s 2022-12-20 18:20:24.344 2m
http-transport: back-off complete, retrying http request backoff time: 1m38.411623696s 2022-12-20 18:20:24.352 8ms http-transport: sending http request range-rq: bytes=61- received: 61 remaining: 9,438,568,778 2022-12-20 18:20:25.027 675ms
http-transport: http request error http code: 0 outputErr: error reading from http response stream: unexpected EOF 2022-12-20 18:20:25.034 7ms http-transport: backing off before retrying http request attempts: 11 backoff time: 4m47.704750756s 2022-12-20 18:25:12.739 5m
http-transport: back-off complete, retrying http request backoff time: 4m47.704750756s 2022-12-20 18:25:12.746 7ms http-transport: sending http request range-rq: bytes=61- received: 61 remaining: 9,438,568,778 2022-12-20 18:25:13.871 1s
http-transport: http request error http code: 0 outputErr: error reading from http response stream: unexpected EOF 2022-12-20 18:25:13.878 7ms http-transport: backing off before retrying http request attempts: 12 backoff time: 3m33.781663682s

[stuberman]

Additional logs after another panic running -vv flag

2022-12-20T23:47:35.048Z DEBUG dagstore dagstore/wrapper.go:169 trace {"shard-key": "baga6ea4seaqcpzadudc7i3bard5ihiqi7j4qbm6mzadsunkg7gxo3cykhlhoeia", "op-type": "OpShardRelease", "after": "ShardStateAvailable"} unexpected fault address 0x0 fatal error: fault [signal SIGSEGV: segmentation violation code=0x80 addr=0x0 pc=0x1cb99fd]

goroutine 914290 [running]: runtime.throw({0x3ba206f?, 0x38abdc0?}) /usr/local/go/src/runtime/panic.go:992 +0x71 fp=0xc00e2d7328 sp=0xc00e2d72f8 pc=0x6afcd1 runtime.sigpanic() /usr/local/go/src/runtime/signal_unix.go:825 +0x305 fp=0xc00e2d7378 sp=0xc00e2d7328 pc=0x6c61a5 github.com/filecoin-project/go-data-transfer/channels.channelState.Voucher({{0xc00f5e0b70, 0x26}, 0x0, {{0x6, 0x1}}, 0x40000, {0x40000, 0x0}, {0x0, 0x1}, ...}) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-data-transfer@v1.15.2/channels/channel_state.go:104 +0x3d fp=0xc00e2d73b0 sp=0xc00e2d7378 pc=0x1cb99fd github.com/filecoin-project/go-data-transfer/channels.(*channelState).Voucher(0x3897d80?)

:1 +0x78 fp=0xc00e2d75a0 sp=0xc00e2d73b0 pc=0x1cc3c78 github.com/filecoin-project/lotus/api.NewDataTransferChannel({0xc000a1ffb0, 0x26}, {0x44078c0, 0xc0b5e34400}) /home/stuart/go/pkg/mod/github.com/filecoin-project/lotus@v1.18.0/api/types.go:108 +0x1b6 fp=0xc00e2d7680 sp=0xc00e2d75a0 pc=0x12db996 github.com/filecoin-project/boost/node/impl.(*BoostAPI).MarketListDataTransfers(0xc000b024e0, {0x43f0250?, 0xc025224300?}) /home/stuart/boost/node/impl/boost_legacy.go:33 +0x1a5 fp=0xc00e2d7818 sp=0xc00e2d7680 pc=0x1e42f85 runtime.call32(0xc024e0ff50, 0xc096677140, 0x0, 0x0, 0x0, 0x18, 0xc00e2d7d58) /usr/local/go/src/runtime/asm_amd64.s:702 +0x49 fp=0xc00e2d7848 sp=0xc00e2d7818 pc=0x6e19e9 runtime.reflectcall(0x393acc0?, 0xc025224300?, 0x4?, 0x3cb9bc4?, 0x0?, 0x12?, 0x38c5aa0?) :1 +0x3c fp=0xc00e2d7888 sp=0xc00e2d7848 pc=0x6e5edc reflect.Value.call({0x3b6a060?, 0xc000b024e0?, 0x38?}, {0x3b934b0, 0x4}, {0xc024d4db30, 0x1, 0x0?}) /usr/local/go/src/reflect/value.go:556 +0x845 fp=0xc00e2d7e78 sp=0xc00e2d7888 pc=0x712505 reflect.Value.Call({0x3b6a060?, 0xc000b024e0?, 0xc000984de8?}, {0xc024d4db30, 0x1, 0x1}) /usr/local/go/src/reflect/value.go:339 +0xbf fp=0xc00e2d7ef0 sp=0xc00e2d7e78 pc=0x711a7f github.com/filecoin-project/boost/metrics/proxy.proxy.func1({0xc024d4db30, 0x1, 0x1}) /home/stuart/boost/metrics/proxy/proxy.go:37 +0x25b fp=0xc00e2d7fc0 sp=0xc00e2d7ef0 pc=0x1e6f07b reflect.callReflect(0xc0247d11d0, 0xc00e2d8520, 0xc00e2d83f8, 0xc00e2d8400) /usr/local/go/src/reflect/value.go:742 +0x56a fp=0xc00e2d83a8 sp=0xc00e2d7fc0 pc=0x713d8a reflect.callReflect(0xc0247d11d0, 0xc01be6e520, 0xc01be6e3f8, 0xc01be6e400) :1 +0x4b fp=0xc00e2d83d8 sp=0xc00e2d83a8 pc=0x71faeb reflect.makeFuncStub() /usr/local/go/src/reflect/asm_amd64.s:47 +0x7a fp=0xc00e2d8520 sp=0xc00e2d83d8 pc=0x71f95a github.com/filecoin-project/boost/api.(*BoostStruct).MarketListDataTransfers(0x8?, {0x43f0250?, 0xc025224210?}) /home/stuart/boost/api/proxy_gen.go:664 +0x35 fp=0xc00e2d8540 sp=0xc00e2d8520 pc=0x1978f35 runtime.call32(0xc024e0fe90, 0xc096677128, 0x0, 0x0, 0x0, 0x18, 0xc00e2d8a80) /usr/local/go/src/runtime/asm_amd64.s:702 +0x49 fp=0xc00e2d8570 sp=0xc00e2d8540 pc=0x6e19e9 runtime.reflectcall(0x38c5aa0?, 0xc04d8c1170?, 0x4?, 0x3cb9bc4?, 0x0?, 0x12?, 0x38c5aa0?) :1 +0x3c fp=0xc00e2d85b0 sp=0xc00e2d8570 pc=0x6e5edc reflect.Value.call({0x3b680c0?, 0xc01453ea00?, 0x3755ee0?}, {0x3b934b0, 0x4}, {0xc024d4db00, 0x1, 0x38c5aa0?}) /usr/local/go/src/reflect/value.go:556 +0x845 fp=0xc00e2d8ba0 sp=0xc00e2d85b0 pc=0x712505 reflect.Value.Call({0x3b680c0?, 0xc01453ea00?, 0x5c28080?}, {0xc024d4db00, 0x1, 0x1}) /usr/local/go/src/reflect/value.go:339 +0xbf fp=0xc00e2d8c18 sp=0xc00e2d8ba0 pc=0x711a7f github.com/filecoin-project/go-jsonrpc/auth.PermissionedProxy.func1({0xc024d4db00, 0x1, 0x1}) /home/stuart/go/pkg/mod/github.com/nonsense/go-jsonrpc@v0.0.0-20221118154859-f3f8e9b1cb11/auth/auth.go:62 +0x425 fp=0xc00e2d8d00 sp=0xc00e2d8c18 pc=0xbc9845 reflect.callReflect(0xc02489af00, 0xc00e2d9260, 0xc00e2d9138, 0xc00e2d9140) /usr/local/go/src/reflect/value.go:742 +0x56a fp=0xc00e2d90e8 sp=0xc00e2d8d00 pc=0x713d8a reflect.callReflect(0xc02489af00, 0xc01f9db260, 0xc01f9db138, 0xc01f9db140) :1 +0x4b fp=0xc00e2d9118 sp=0xc00e2d90e8 pc=0x71faeb reflect.makeFuncStub() /usr/local/go/src/reflect/asm_amd64.s:47 +0x7a fp=0xc00e2d9260 sp=0xc00e2d9118 pc=0x71f95a github.com/filecoin-project/boost/api.(*BoostStruct).MarketListDataTransfers(0x8?, {0x43f0250?, 0xc025224210?}) /home/stuart/boost/api/proxy_gen.go:664 +0x35 fp=0xc00e2d9280 sp=0xc00e2d9260 pc=0x1978f35 runtime.call32(0xc024e0fe30, 0xc02bedefd8, 0x0, 0x0, 0x0, 0x18, 0xc00e2d97c0) /usr/local/go/src/runtime/asm_amd64.s:702 +0x49 fp=0xc00e2d92b0 sp=0xc00e2d9280 pc=0x6e19e9 runtime.reflectcall(0x393acc0?, 0xc025224210?, 0x4?, 0x3cb9bc4?, 0x0?, 0x12?, 0x38c5aa0?) :1 +0x3c fp=0xc00e2d92f0 sp=0xc00e2d92b0 pc=0x6e5edc reflect.Value.call({0xc024de01e0?, 0xc02bedefd8?, 0x13?}, {0x3b934b0, 0x4}, {0xc025224240, 0x2, 0x2?}) /usr/local/go/src/reflect/value.go:556 +0x845 fp=0xc00e2d98e0 sp=0xc00e2d92f0 pc=0x712505 reflect.Value.Call({0xc024de01e0?, 0xc02bedefd8?, 0xc0121f8740?}, {0xc025224240, 0x2, 0x2}) /usr/local/go/src/reflect/value.go:339 +0xbf fp=0xc00e2d9958 sp=0xc00e2d98e0 pc=0x711a7f github.com/filecoin-project/go-jsonrpc.doCall({0xc0c31f9980?, 0xc02bedefd8?}, {0xc024de01e0?, 0xc02bedefd8?, 0x0?}, {0xc025224240?, 0x0?, 0x0?}) /home/stuart/go/pkg/mod/github.com/nonsense/go-jsonrpc@v0.0.0-20221118154859-f3f8e9b1cb11/handler.go:193 +0xd0 fp=0xc00e2d99f0 sp=0xc00e2d9958 pc=0xa808f0 github.com/filecoin-project/go-jsonrpc.(*RPCServer).handle(0xc02489bd40, {0x43f01a8, 0xc02570b100}, {{0xc019a59748, 0x3}, {0x37406a0, 0xc019a59750}, {0xc0c31f9980, 0x20}, {0x5ff3640, ...}, ...}, ...) /home/stuart/go/pkg/mod/github.com/nonsense/go-jsonrpc@v0.0.0-20221118154859-f3f8e9b1cb11/handler.go:314 +0x1249 fp=0xc00e2d9f48 sp=0xc00e2d99f0 pc=0xa825e9 github.com/filecoin-project/go-jsonrpc.(*wsConn).handleCall.func4() /home/stuart/go/pkg/mod/github.com/nonsense/go-jsonrpc@v0.0.0-20221118154859-f3f8e9b1cb11/websocket.go:424 +0x7e fp=0xc00e2d9fe0 sp=0xc00e2d9f48 pc=0xa877fe runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1571 +0x1 fp=0xc00e2d9fe8 sp=0xc00e2d9fe0 pc=0x6e3621 created by github.com/filecoin-project/go-jsonrpc.(*wsConn).handleCall /home/stuart/go/pkg/mod/github.com/nonsense/go-jsonrpc@v0.0.0-20221118154859-f3f8e9b1cb11/websocket.go:424 +0x4cf goroutine 1 [chan receive, 6 minutes]: main.glob..func36(0x5c60800?) /home/stuart/boost/cmd/boostd/run.go:152 +0x1057 github.com/urfave/cli/v2.(*Command).Run(0x5c60800, 0xc000adf8c0) /home/stuart/go/pkg/mod/github.com/urfave/cli/v2@v2.8.1/command.go:169 +0x631 github.com/urfave/cli/v2.(*App).RunContext(0xc000b02680, {0x43f01e0?, 0xc0000520b8}, {0xc00004e080, 0x4, 0x4}) /home/stuart/go/pkg/mod/github.com/urfave/cli/v2@v2.8.1/app.go:341 +0xbc8 github.com/urfave/cli/v2.(*App).Run(...) /home/stuart/go/pkg/mod/github.com/urfave/cli/v2@v2.8.1/app.go:247 main.main() /home/stuart/boost/cmd/boostd/main.go:57 +0x4b2 goroutine 6 [select]: go.opencensus.io/stats/view.(*worker).start(0xc0000de300) /home/stuart/go/pkg/mod/go.opencensus.io@v0.23.0/stats/view/worker.go:276 +0xad created by go.opencensus.io/stats/view.init.0 /home/stuart/go/pkg/mod/go.opencensus.io@v0.23.0/stats/view/worker.go:34 +0x8d goroutine 7 [select, 6 minutes]: github.com/ipfs/go-log/writer.(*MirrorWriter).logRoutine(0xc0000c5950) /home/stuart/go/pkg/mod/github.com/ipfs/go-log@v1.0.5/writer/writer.go:71 +0x11f created by github.com/ipfs/go-log/writer.NewMirrorWriter /home/stuart/go/pkg/mod/github.com/ipfs/go-log@v1.0.5/writer/writer.go:36 +0xca goroutine 706408 [select, 1 minutes]: github.com/libp2p/go-libp2p-pubsub.(*PubSub).handleSendingMessages(0xc0a0fea810?, {0x43f01a8, 0xc000bd9640}, {0x4403a50?, 0xc0203eaf20}, 0xc025082d80) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/comm.go:164 +0x305 created by github.com/libp2p/go-libp2p-pubsub.(*PubSub).handleNewPeer /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/comm.go:117 +0x334 goroutine 39 [select]: github.com/filecoin-project/go-jsonrpc.(*wsConn).setupPings.func3() /home/stuart/go/pkg/mod/github.com/nonsense/go-jsonrpc@v0.0.0-20221118154859-f3f8e9b1cb11/websocket.go:494 +0x90 created by github.com/filecoin-project/go-jsonrpc.(*wsConn).setupPings /home/stuart/go/pkg/mod/github.com/nonsense/go-jsonrpc@v0.0.0-20221118154859-f3f8e9b1cb11/websocket.go:492 +0x10d goroutine 32 [chan receive]: github.com/golang/glog.(*loggingT).flushDaemon(0xc0009338c0?) /home/stuart/go/pkg/mod/github.com/golang/glog@v1.0.0/glog.go:882 +0x6a created by github.com/golang/glog.init.0 /home/stuart/go/pkg/mod/github.com/golang/glog@v1.0.0/glog.go:410 +0x1bf goroutine 114 [syscall, 6 minutes]: os/signal.signal_recv() /usr/local/go/src/runtime/sigqueue.go:151 +0x2f os/signal.loop() /usr/local/go/src/os/signal/signal_unix.go:23 +0x19 created by os/signal.Notify.func1.1 /usr/local/go/src/os/signal/signal.go:151 +0x2a goroutine 66 [IO wait, 6 minutes]: internal/poll.runtime_pollWait(0x7f9a35129ef8, 0x72) /usr/local/go/src/runtime/netpoll.go:302 +0x89 internal/poll.(*pollDesc).wait(0xc0012b0000?, 0x0?, 0x0) /usr/local/go/src/internal/poll/fd_poll_runtime.go:83 +0x32 internal/poll.(*pollDesc).waitRead(...) /usr/local/go/src/internal/poll/fd_poll_runtime.go:88 internal/poll.(*FD).Accept(0xc0012b0000) /usr/local/go/src/internal/poll/fd_unix.go:614 +0x22c net.(*netFD).accept(0xc0012b0000) /usr/local/go/src/net/fd_unix.go:172 +0x35 net.(*TCPListener).accept(0xc000cf0030) /usr/local/go/src/net/tcpsock_posix.go:139 +0x28 net.(*TCPListener).Accept(0xc000cf0030) /usr/local/go/src/net/tcpsock.go:288 +0x3d net/http.(*Server).Serve(0xc000e38000, {0x43edab8, 0xc000cf0030}) /usr/local/go/src/net/http/server.go:3039 +0x385 net/http.(*Server).ListenAndServe(0xc000e38000) /usr/local/go/src/net/http/server.go:2968 +0x7d net/http.ListenAndServe(...) /usr/local/go/src/net/http/server.go:3222 main.glob..func36.1() /home/stuart/boost/cmd/boostd/run.go:39 +0x45 created by main.glob..func36 /home/stuart/boost/cmd/boostd/run.go:38 +0x70 goroutine 55 [select, 1 minutes]: github.com/filecoin-project/lotus/journal/fsjournal.(*fsJournal).runLoop(0xc0015883c0) /home/stuart/go/pkg/mod/github.com/filecoin-project/lotus@v1.18.0/journal/fsjournal/fs.go:144 +0xda created by github.com/filecoin-project/lotus/journal/fsjournal.OpenFSJournal /home/stuart/go/pkg/mod/github.com/filecoin-project/lotus@v1.18.0/journal/fsjournal/fs.go:58 +0x36a goroutine 505 [sync.Cond.Wait, 6 minutes]: sync.runtime_notifyListWait(0xc01107c6d0, 0x0) /usr/local/go/src/runtime/sema.go:513 +0x13d sync.(*Cond).Wait(0x2?) /usr/local/go/src/sync/cond.go:56 +0x8c github.com/ipfs/go-graphsync/notifications.(*publisher).dequeue(0xc01107c700) /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/notifications/publisher.go:226 +0x6e github.com/ipfs/go-graphsync/notifications.(*publisher).start(0xc013f34d80?) /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/notifications/publisher.go:122 +0x125 created by github.com/ipfs/go-graphsync/notifications.(*publisher).Startup /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/notifications/publisher.go:46 +0x56 goroutine 889 [select, 6 minutes]: github.com/filecoin-project/go-statemachine.(*StateMachine).run(0xc000d5c8c0) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/machine.go:53 +0x130 created by github.com/filecoin-project/go-statemachine.(*StateGroup).loadOrCreate /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/group.go:132 +0x570 goroutine 98 [select]: github.com/filecoin-project/go-jsonrpc.(*wsConn).handleWsConn(0xc00025e3c0, {0x43f01e0, 0xc0000520b8}) /home/stuart/go/pkg/mod/github.com/nonsense/go-jsonrpc@v0.0.0-20221118154859-f3f8e9b1cb11/websocket.go:603 +0x546 created by github.com/filecoin-project/go-jsonrpc.websocketClient /home/stuart/go/pkg/mod/github.com/nonsense/go-jsonrpc@v0.0.0-20221118154859-f3f8e9b1cb11/client.go:271 +0x3e5 goroutine 202 [select]: reflect.rselect({0xc000e05d58, 0x2, 0x7521a0?}) /usr/local/go/src/runtime/select.go:590 +0x23e reflect.Select({0xc000e05f60?, 0x2, 0x2?}) /usr/local/go/src/reflect/value.go:2870 +0xd2 github.com/filecoin-project/go-jsonrpc.(*client).makeOutChan.func1.1() /home/stuart/go/pkg/mod/github.com/nonsense/go-jsonrpc@v0.0.0-20221118154859-f3f8e9b1cb11/client.go:356 +0x41f created by github.com/filecoin-project/go-jsonrpc.(*client).makeOutChan.func1 /home/stuart/go/pkg/mod/github.com/nonsense/go-jsonrpc@v0.0.0-20221118154859-f3f8e9b1cb11/client.go:331 +0x1ca goroutine 100 [chan receive, 6 minutes]: github.com/filecoin-project/lotus/cli/util.ReqContext.func1() /home/stuart/go/pkg/mod/github.com/filecoin-project/lotus@v1.18.0/cli/util/api.go:342 +0x28 created by github.com/filecoin-project/lotus/cli/util.ReqContext /home/stuart/go/pkg/mod/github.com/filecoin-project/lotus@v1.18.0/cli/util/api.go:341 +0xf1 goroutine 56 [select, 6 minutes]: github.com/libp2p/go-libp2p/p2p/host/peerstore/pstoremem.(*memoryAddrBook).background(0xc0009a2d00, {0x43f01a8, 0xc001582480}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/host/peerstore/pstoremem/addr_book.go:113 +0x12c created by github.com/libp2p/go-libp2p/p2p/host/peerstore/pstoremem.NewAddrBook /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/host/peerstore/pstoremem/addr_book.go:93 +0x26a goroutine 57 [select, 6 minutes]: github.com/dgraph-io/badger/v2/y.(*WaterMark).process(0xc0015a40c0, 0xc0015a4090) /home/stuart/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.3/y/watermark.go:212 +0x26c created by github.com/dgraph-io/badger/v2/y.(*WaterMark).Init /home/stuart/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.3/y/watermark.go:70 +0xaa goroutine 58 [select, 6 minutes]: github.com/dgraph-io/badger/v2/y.(*WaterMark).process(0xc0015a40f0, 0xc0015a4090) /home/stuart/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.3/y/watermark.go:212 +0x26c created by github.com/dgraph-io/badger/v2/y.(*WaterMark).Init /home/stuart/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.3/y/watermark.go:70 +0xaa goroutine 59 [select]: github.com/dgraph-io/badger/v2.(*DB).updateSize(0xc000b01800, 0xc0015a4240) /home/stuart/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.3/db.go:1201 +0x13a created by github.com/dgraph-io/badger/v2.Open /home/stuart/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.3/db.go:361 +0x112a goroutine 41 [select]: github.com/dgraph-io/badger/v2.(*levelsController).runCompactor(0xc000e18620, 0x0, 0xc00029b020) /home/stuart/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.3/levels.go:390 +0x209 created by github.com/dgraph-io/badger/v2.(*levelsController).startCompact /home/stuart/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.3/levels.go:371 +0x50 goroutine 42 [select]: github.com/dgraph-io/badger/v2.(*levelsController).runCompactor(0xc000e18620, 0x1, 0xc00029b020) /home/stuart/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.3/levels.go:390 +0x209 created by github.com/dgraph-io/badger/v2.(*levelsController).startCompact /home/stuart/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.3/levels.go:371 +0x50 goroutine 43 [chan receive, 6 minutes]: github.com/dgraph-io/badger/v2.(*DB).flushMemtable(0xc000b01800, 0x0?) /home/stuart/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.3/db.go:1108 +0xaf github.com/dgraph-io/badger/v2.Open.func5() /home/stuart/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.3/db.go:378 +0x25 created by github.com/dgraph-io/badger/v2.Open /home/stuart/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.3/db.go:377 +0x13aa goroutine 46 [select, 6 minutes]: github.com/dgraph-io/badger/v2.(*DB).doWrites(0xc000b01800, 0xc00029b170) /home/stuart/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.3/db.go:863 +0x209 created by github.com/dgraph-io/badger/v2.Open /home/stuart/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.3/db.go:412 +0x1865 goroutine 45 [select, 6 minutes]: github.com/dgraph-io/badger/v2.(*valueLog).flushDiscardStats(0xc000b019d0) /home/stuart/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.3/value.go:1955 +0x155 created by github.com/dgraph-io/badger/v2.(*valueLog).open /home/stuart/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.3/value.go:1114 +0x9d goroutine 47 [chan receive, 6 minutes]: github.com/dgraph-io/badger/v2.(*valueLog).waitOnGC(0xc000b019d0, 0x0?) /home/stuart/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.3/value.go:1858 +0x6f created by github.com/dgraph-io/badger/v2.Open /home/stuart/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.3/db.go:416 +0x198a goroutine 48 [select, 6 minutes]: github.com/dgraph-io/badger/v2.(*publisher).listenForUpdates(0xc0015a4180, 0xc00029b1d0) /home/stuart/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.3/publisher.go:67 +0x139 created by github.com/dgraph-io/badger/v2.Open /home/stuart/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.3/db.go:420 +0x1aa5 goroutine 49 [select, 6 minutes]: github.com/ipfs/go-ds-badger2.(*Datastore).periodicGC(0xc0015ae600) /home/stuart/go/pkg/mod/github.com/ipfs/go-ds-badger2@v0.1.2/datastore.go:182 +0xe5 created by github.com/ipfs/go-ds-badger2.NewDatastore /home/stuart/go/pkg/mod/github.com/ipfs/go-ds-badger2@v0.1.2/datastore.go:170 +0x605 goroutine 130 [select, 1 minutes]: github.com/syndtr/goleveldb/leveldb.(*session).refLoop(0xc0009304b0) /home/stuart/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20210819022825-2ae1ddf74ef7/leveldb/session_util.go:189 +0x59b created by github.com/syndtr/goleveldb/leveldb.newSession /home/stuart/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20210819022825-2ae1ddf74ef7/leveldb/session.go:93 +0x2d9 goroutine 131 [select, 1 minutes]: github.com/syndtr/goleveldb/leveldb.(*DB).compactionError(0xc0011808c0) /home/stuart/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20210819022825-2ae1ddf74ef7/leveldb/db_compaction.go:91 +0x158 created by github.com/syndtr/goleveldb/leveldb.openDB /home/stuart/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20210819022825-2ae1ddf74ef7/leveldb/db.go:148 +0x4ea goroutine 132 [select]: github.com/syndtr/goleveldb/leveldb.(*DB).mpoolDrain(0xc0011808c0) /home/stuart/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20210819022825-2ae1ddf74ef7/leveldb/db_state.go:101 +0xa8 created by github.com/syndtr/goleveldb/leveldb.openDB /home/stuart/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20210819022825-2ae1ddf74ef7/leveldb/db.go:149 +0x52a goroutine 133 [select, 1 minutes]: github.com/syndtr/goleveldb/leveldb.(*DB).tCompaction(0xc0011808c0) /home/stuart/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20210819022825-2ae1ddf74ef7/leveldb/db_compaction.go:836 +0x657 created by github.com/syndtr/goleveldb/leveldb.openDB /home/stuart/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20210819022825-2ae1ddf74ef7/leveldb/db.go:155 +0x598 goroutine 134 [select, 1 minutes]: github.com/syndtr/goleveldb/leveldb.(*DB).mCompaction(0xc0011808c0) /home/stuart/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20210819022825-2ae1ddf74ef7/leveldb/db_compaction.go:773 +0x113 created by github.com/syndtr/goleveldb/leveldb.openDB /home/stuart/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20210819022825-2ae1ddf74ef7/leveldb/db.go:156 +0x5d6 goroutine 135 [select, 6 minutes]: github.com/dgraph-io/badger/v2/y.(*WaterMark).process(0xc0088f2d20, 0xc0088f2cf0) /home/stuart/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.3/y/watermark.go:212 +0x26c created by github.com/dgraph-io/badger/v2/y.(*WaterMark).Init /home/stuart/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.3/y/watermark.go:70 +0xaa goroutine 136 [select, 6 minutes]: github.com/dgraph-io/badger/v2/y.(*WaterMark).process(0xc0088f2d50, 0xc0088f2cf0) /home/stuart/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.3/y/watermark.go:212 +0x26c created by github.com/dgraph-io/badger/v2/y.(*WaterMark).Init /home/stuart/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.3/y/watermark.go:70 +0xaa goroutine 137 [select]: github.com/dgraph-io/badger/v2.(*DB).updateSize(0xc000c00800, 0xc0088f2ea0) /home/stuart/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.3/db.go:1201 +0x13a created by github.com/dgraph-io/badger/v2.Open /home/stuart/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.3/db.go:361 +0x112a goroutine 138 [select]: github.com/dgraph-io/badger/v2.(*levelsController).runCompactor(0xc0001d7a40, 0x0, 0xc0088f2060) /home/stuart/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.3/levels.go:390 +0x209 created by github.com/dgraph-io/badger/v2.(*levelsController).startCompact /home/stuart/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.3/levels.go:371 +0x50 goroutine 139 [select]: github.com/dgraph-io/badger/v2.(*levelsController).runCompactor(0xc0001d7a40, 0x1, 0xc0088f2060) /home/stuart/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.3/levels.go:390 +0x209 created by github.com/dgraph-io/badger/v2.(*levelsController).startCompact /home/stuart/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.3/levels.go:371 +0x50 goroutine 140 [chan receive, 6 minutes]: github.com/dgraph-io/badger/v2.(*DB).flushMemtable(0xc000c00800, 0xc0011808c0?) /home/stuart/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.3/db.go:1108 +0xaf github.com/dgraph-io/badger/v2.Open.func5() /home/stuart/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.3/db.go:378 +0x25 created by github.com/dgraph-io/badger/v2.Open /home/stuart/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.3/db.go:377 +0x13aa goroutine 146 [select, 6 minutes]: github.com/dgraph-io/badger/v2.(*DB).doWrites(0xc000c00800, 0xc0089ed500) /home/stuart/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.3/db.go:863 +0x209 created by github.com/dgraph-io/badger/v2.Open /home/stuart/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.3/db.go:412 +0x1865 goroutine 142 [select, 6 minutes]: github.com/dgraph-io/badger/v2.(*valueLog).flushDiscardStats(0xc000c009d0) /home/stuart/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.3/value.go:1955 +0x155 created by github.com/dgraph-io/badger/v2.(*valueLog).open /home/stuart/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.3/value.go:1114 +0x9d goroutine 147 [chan receive, 6 minutes]: github.com/dgraph-io/badger/v2.(*valueLog).waitOnGC(0xc000c009d0, 0x0?) /home/stuart/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.3/value.go:1858 +0x6f created by github.com/dgraph-io/badger/v2.Open /home/stuart/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.3/db.go:416 +0x198a goroutine 148 [select, 6 minutes]: github.com/dgraph-io/badger/v2.(*publisher).listenForUpdates(0xc0088f2de0, 0xc0089ed560) /home/stuart/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.3/publisher.go:67 +0x139 created by github.com/dgraph-io/badger/v2.Open /home/stuart/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.2007.3/db.go:420 +0x1aa5 goroutine 149 [select, 6 minutes]: github.com/ipfs/go-ds-badger2.(*Datastore).periodicGC(0xc000b96000) /home/stuart/go/pkg/mod/github.com/ipfs/go-ds-badger2@v0.1.2/datastore.go:182 +0xe5 created by github.com/ipfs/go-ds-badger2.NewDatastore /home/stuart/go/pkg/mod/github.com/ipfs/go-ds-badger2@v0.1.2/datastore.go:170 +0x605 goroutine 172 [select]: github.com/libp2p/go-libp2p/p2p/net/connmgr.(*decayer).process(0xc0012b1580) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/net/connmgr/decay.go:164 +0x245 created by github.com/libp2p/go-libp2p/p2p/net/connmgr.NewDecayer /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/net/connmgr/decay.go:95 +0x265 goroutine 175 [select, 1 minutes]: github.com/libp2p/go-libp2p/p2p/net/swarm.(*DialBackoff).background(0xd82090?, {0x43f01a8, 0xc0002cd600}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/net/swarm/swarm_dial.go:123 +0xea created by github.com/libp2p/go-libp2p/p2p/net/swarm.(*DialBackoff).init /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/net/swarm/swarm_dial.go:116 +0xc5 goroutine 171 [select]: github.com/libp2p/go-libp2p/p2p/host/resource-manager.(*resourceManager).background(0xc000b1c0b0) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/host/resource-manager/rcmgr.go:348 +0x11c created by github.com/libp2p/go-libp2p/p2p/host/resource-manager.NewResourceManager /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/host/resource-manager/rcmgr.go:161 +0x605 goroutine 173 [select]: github.com/libp2p/go-libp2p/p2p/net/connmgr.(*BasicConnMgr).background(0xc0009a1200) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/net/connmgr/connmgr.go:317 +0x137 created by github.com/libp2p/go-libp2p/p2p/net/connmgr.NewConnManager /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/net/connmgr/connmgr.go:140 +0x377 goroutine 78720 [IO wait]: internal/poll.runtime_pollWait(0x7f99ef978ff0, 0x72) /usr/local/go/src/runtime/netpoll.go:302 +0x89 internal/poll.(*pollDesc).wait(0xc027141980?, 0xc01cfd4000?, 0x0) /usr/local/go/src/internal/poll/fd_poll_runtime.go:83 +0x32 internal/poll.(*pollDesc).waitRead(...) /usr/local/go/src/internal/poll/fd_poll_runtime.go:88 internal/poll.(*FD).Read(0xc027141980, {0xc01cfd4000, 0x1000, 0x1000}) /usr/local/go/src/internal/poll/fd_unix.go:167 +0x25a net.(*netFD).Read(0xc027141980, {0xc01cfd4000?, 0xc0d1b9ea00?, 0x1c?}) /usr/local/go/src/net/fd_posix.go:55 +0x29 net.(*conn).Read(0xc01951a708, {0xc01cfd4000?, 0x80?, 0xc01f9d9d48?}) /usr/local/go/src/net/net.go:183 +0x45 bufio.(*Reader).Read(0xc01dd180c0, {0xc01cfd6090, 0x2, 0x6fa62e?}) /usr/local/go/src/bufio/bufio.go:236 +0x1b4 io.ReadAtLeast({0x43d7260, 0xc01dd180c0}, {0xc01cfd6090, 0x2, 0x2}, 0x2) /usr/local/go/src/io/io.go:331 +0x9a io.ReadFull(...) /usr/local/go/src/io/io.go:350 github.com/libp2p/go-libp2p/p2p/security/noise.(*secureSession).readNextInsecureMsgLen(0xc01cfd6000) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/security/noise/rw.go:133 +0x3b github.com/libp2p/go-libp2p/p2p/security/noise.(*secureSession).Read(0xc01cfd6000, {0xc006c61490, 0xc, 0xc}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/security/noise/rw.go:52 +0x22f io.ReadAtLeast({0x7f9a049c0680, 0xc01cfd6000}, {0xc006c61490, 0xc, 0xc}, 0xc) /usr/local/go/src/io/io.go:331 +0x9a io.ReadFull(...) /usr/local/go/src/io/io.go:350 github.com/libp2p/go-yamux/v3.(*Session).recvLoop(0xc00e227200) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:644 +0xf7 github.com/libp2p/go-yamux/v3.(*Session).recv(0x10?) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:616 +0x1e created by github.com/libp2p/go-yamux/v3.newSession /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:158 +0x66a goroutine 177 [select]: github.com/libp2p/go-libp2p/p2p/protocol/identify.(*ObservedAddrManager).worker(0xc001591ba0) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/protocol/identify/obsaddr.go:258 +0x1d3 created by github.com/libp2p/go-libp2p/p2p/protocol/identify.NewObservedAddrManager /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/protocol/identify/obsaddr.go:158 +0x3d6 goroutine 178 [select]: github.com/libp2p/go-libp2p/p2p/protocol/identify.(*idService).loop(0xc001591ad0) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/protocol/identify/id.go:216 +0x4e5 created by github.com/libp2p/go-libp2p/p2p/protocol/identify.NewIDService /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/protocol/identify/id.go:163 +0x356 goroutine 113982 [select, 5 minutes]: github.com/libp2p/go-libp2p-pubsub.(*PubSub).handleSendingMessages(0x0?, {0x43f01a8, 0xc000bd9640}, {0x4403a50?, 0xc080a63100}, 0xc0267ed4a0) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/comm.go:164 +0x305 created by github.com/libp2p/go-libp2p-pubsub.(*PubSub).handleNewPeer /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/comm.go:117 +0x334 goroutine 394161 [IO wait]: internal/poll.runtime_pollWait(0x7f99eee3c290, 0x72) /usr/local/go/src/runtime/netpoll.go:302 +0x89 internal/poll.(*pollDesc).wait(0xc0c2b2ea00?, 0xc0c2b48000?, 0x0) /usr/local/go/src/internal/poll/fd_poll_runtime.go:83 +0x32 internal/poll.(*pollDesc).waitRead(...) /usr/local/go/src/internal/poll/fd_poll_runtime.go:88 internal/poll.(*FD).Read(0xc0c2b2ea00, {0xc0c2b48000, 0x1000, 0x1000}) /usr/local/go/src/internal/poll/fd_unix.go:167 +0x25a net.(*netFD).Read(0xc0c2b2ea00, {0xc0c2b48000?, 0xc0c31f8be0?, 0x1c?}) /usr/local/go/src/net/fd_posix.go:55 +0x29 net.(*conn).Read(0xc08dc08040, {0xc0c2b48000?, 0xc0d4cb72c0?, 0xc00e939d48?}) /usr/local/go/src/net/net.go:183 +0x45 bufio.(*Reader).Read(0xc01d15ae40, {0xc012d26390, 0x2, 0x6fa62e?}) /usr/local/go/src/bufio/bufio.go:236 +0x1b4 io.ReadAtLeast({0x43d7260, 0xc01d15ae40}, {0xc012d26390, 0x2, 0x2}, 0x2) /usr/local/go/src/io/io.go:331 +0x9a io.ReadFull(...) /usr/local/go/src/io/io.go:350 github.com/libp2p/go-libp2p/p2p/security/noise.(*secureSession).readNextInsecureMsgLen(0xc012d26300) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/security/noise/rw.go:133 +0x3b github.com/libp2p/go-libp2p/p2p/security/noise.(*secureSession).Read(0xc012d26300, {0xc05b82dee0, 0xc, 0xc}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/security/noise/rw.go:52 +0x22f io.ReadAtLeast({0x7f9a049c0680, 0xc012d26300}, {0xc05b82dee0, 0xc, 0xc}, 0xc) /usr/local/go/src/io/io.go:331 +0x9a io.ReadFull(...) /usr/local/go/src/io/io.go:350 github.com/libp2p/go-yamux/v3.(*Session).recvLoop(0xc0c490ab40) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:644 +0xf7 github.com/libp2p/go-yamux/v3.(*Session).recv(0xc02a939fb8?) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:616 +0x1e created by github.com/libp2p/go-yamux/v3.newSession /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:158 +0x66a goroutine 109264 [IO wait]: internal/poll.runtime_pollWait(0x7f99eed517f0, 0x72) /usr/local/go/src/runtime/netpoll.go:302 +0x89 internal/poll.(*pollDesc).wait(0xc01cf63280?, 0xc018690000?, 0x0) /usr/local/go/src/internal/poll/fd_poll_runtime.go:83 +0x32 internal/poll.(*pollDesc).waitRead(...) /usr/local/go/src/internal/poll/fd_poll_runtime.go:88 internal/poll.(*FD).Read(0xc01cf63280, {0xc018690000, 0x1000, 0x1000}) /usr/local/go/src/internal/poll/fd_unix.go:167 +0x25a net.(*netFD).Read(0xc01cf63280, {0xc018690000?, 0xc0d1b9e480?, 0x1c?}) /usr/local/go/src/net/fd_posix.go:55 +0x29 net.(*conn).Read(0xc027fa2158, {0xc018690000?, 0xc01c497798?, 0xc01f12fd48?}) /usr/local/go/src/net/net.go:183 +0x45 bufio.(*Reader).Read(0xc025ad7860, {0xc02b186750, 0x2, 0x6fa62e?}) /usr/local/go/src/bufio/bufio.go:236 +0x1b4 io.ReadAtLeast({0x43d7260, 0xc025ad7860}, {0xc02b186750, 0x2, 0x2}, 0x2) /usr/local/go/src/io/io.go:331 +0x9a io.ReadFull(...) /usr/local/go/src/io/io.go:350 github.com/libp2p/go-libp2p/p2p/security/noise.(*secureSession).readNextInsecureMsgLen(0xc02b1866c0) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/security/noise/rw.go:133 +0x3b github.com/libp2p/go-libp2p/p2p/security/noise.(*secureSession).Read(0xc02b1866c0, {0xc01ac26010, 0xc, 0xc}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/security/noise/rw.go:52 +0x22f io.ReadAtLeast({0x7f9a049c0680, 0xc02b1866c0}, {0xc01ac26010, 0xc, 0xc}, 0xc) /usr/local/go/src/io/io.go:331 +0x9a io.ReadFull(...) /usr/local/go/src/io/io.go:350 github.com/libp2p/go-yamux/v3.(*Session).recvLoop(0xc01c078240) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:644 +0xf7 github.com/libp2p/go-yamux/v3.(*Session).recv(0xc00025e480?) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:616 +0x1e created by github.com/libp2p/go-yamux/v3.newSession /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:158 +0x66a goroutine 68737 [select, 6 minutes]: github.com/libp2p/go-libp2p-pubsub.(*PubSub).handleSendingMessages(0xc022959020?, {0x43f01a8, 0xc000bd9640}, {0x4403a50?, 0xc0168e4ca0}, 0xc011782ae0) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/comm.go:164 +0x305 created by github.com/libp2p/go-libp2p-pubsub.(*PubSub).handleNewPeer /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/comm.go:117 +0x334 goroutine 482102 [sync.Cond.Wait, 3 minutes]: sync.runtime_notifyListWait(0xc020544c90, 0x0) /usr/local/go/src/runtime/sema.go:513 +0x13d sync.(*Cond).Wait(0xc01b1eda80?) /usr/local/go/src/sync/cond.go:56 +0x8c github.com/ipfs/go-graphsync/notifications.(*publisher).dequeue(0xc020544cc0) /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/notifications/publisher.go:226 +0x6e github.com/ipfs/go-graphsync/notifications.(*publisher).start(0xc02c845fb8?) /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/notifications/publisher.go:122 +0x125 created by github.com/ipfs/go-graphsync/notifications.(*publisher).Startup /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/notifications/publisher.go:46 +0x56 goroutine 492151 [sync.Cond.Wait, 3 minutes]: sync.runtime_notifyListWait(0xc014297b50, 0x0) /usr/local/go/src/runtime/sema.go:513 +0x13d sync.(*Cond).Wait(0x0?) /usr/local/go/src/sync/cond.go:56 +0x8c github.com/ipfs/go-graphsync/notifications.(*publisher).dequeue(0xc014297b80) /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/notifications/publisher.go:226 +0x6e github.com/ipfs/go-graphsync/notifications.(*publisher).start(0xc012164080?) /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/notifications/publisher.go:122 +0x125 created by github.com/ipfs/go-graphsync/notifications.(*publisher).Startup /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/notifications/publisher.go:46 +0x56 goroutine 180 [select]: github.com/libp2p/go-libp2p/p2p/transport/quic.(*reuse).gc(0xc000bd87c0) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/transport/quic/reuse.go:93 +0x13e created by github.com/libp2p/go-libp2p/p2p/transport/quic.newReuse /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/transport/quic/reuse.go:71 +0x110 goroutine 181 [select]: github.com/libp2p/go-libp2p/p2p/transport/quic.(*reuse).gc(0xc000bd8800) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/transport/quic/reuse.go:93 +0x13e created by github.com/libp2p/go-libp2p/p2p/transport/quic.newReuse /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/transport/quic/reuse.go:71 +0x110 goroutine 182 [select, 6 minutes]: github.com/libp2p/go-libp2p/p2p/host/peerstore/pstoremem.(*memoryAddrBook).background(0xc0009a1b00, {0x43f01a8, 0xc000bd8840}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/host/peerstore/pstoremem/addr_book.go:113 +0x12c created by github.com/libp2p/go-libp2p/p2p/host/peerstore/pstoremem.NewAddrBook /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/host/peerstore/pstoremem/addr_book.go:93 +0x26a goroutine 183 [select, 1 minutes]: github.com/libp2p/go-libp2p/p2p/net/swarm.(*DialBackoff).background(0x0?, {0x43f01a8, 0xc000bd8880}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/net/swarm/swarm_dial.go:123 +0xea created by github.com/libp2p/go-libp2p/p2p/net/swarm.(*DialBackoff).init /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/net/swarm/swarm_dial.go:116 +0xc5 goroutine 184 [select]: github.com/libp2p/go-libp2p/p2p/transport/quic.(*reuse).gc(0xc000bd8a80) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/transport/quic/reuse.go:93 +0x13e created by github.com/libp2p/go-libp2p/p2p/transport/quic.newReuse /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/transport/quic/reuse.go:71 +0x110 goroutine 185 [select]: github.com/libp2p/go-libp2p/p2p/transport/quic.(*reuse).gc(0xc000bd8ac0) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/transport/quic/reuse.go:93 +0x13e created by github.com/libp2p/go-libp2p/p2p/transport/quic.newReuse /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/transport/quic/reuse.go:71 +0x110 goroutine 186 [select]: github.com/libp2p/go-libp2p/p2p/host/autonat.(*autoNATService).background(0xc000bd8b00, {0x43f01a8, 0xc000bd8b40}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/host/autonat/svc.go:270 +0x165 created by github.com/libp2p/go-libp2p/p2p/host/autonat.(*autoNATService).Enable /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/host/autonat/svc.go:248 +0x1c5 goroutine 416 [select, 6 minutes]: github.com/filecoin-project/go-statemachine.(*StateMachine).run(0xc0087a4a50) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/machine.go:53 +0x130 created by github.com/filecoin-project/go-statemachine.(*StateGroup).loadOrCreate /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/group.go:132 +0x570 goroutine 201 [select]: github.com/filecoin-project/go-jsonrpc.websocketClient.func2({0x43f0250, 0xc0bfadcdb0}, {{{0x3b910d4, 0x3}, {0x37406a0, 0xc0aae04540}, {0xc0b23ccfa8, 0x15}, {0xc0bfadcd80, 0x1, ...}, ...}, ...}) /home/stuart/go/pkg/mod/github.com/nonsense/go-jsonrpc@v0.0.0-20221118154859-f3f8e9b1cb11/client.go:242 +0x27b github.com/filecoin-project/go-jsonrpc.(*client).sendRequest(0xc000cd3f40, {0x43f0250, 0xc0bfadcdb0}, {{0x3b910d4, 0x3}, {0x37406a0, 0xc0aae04540}, {0xc0b23ccfa8, 0x15}, {0xc0bfadcd80, ...}, ...}, ...) /home/stuart/go/pkg/mod/github.com/nonsense/go-jsonrpc@v0.0.0-20221118154859-f3f8e9b1cb11/client.go:423 +0x12a github.com/filecoin-project/go-jsonrpc.(*rpcFunc).handleRpcCall(0xc0011aa870, {0xc0bfadcd50, 0x2, 0x2}) /home/stuart/go/pkg/mod/github.com/nonsense/go-jsonrpc@v0.0.0-20221118154859-f3f8e9b1cb11/client.go:545 +0x757 github.com/filecoin-project/lotus/api.(*FullNodeStruct).ChainReadObj(0xc0aae04530?, {0x43f01e0?, 0xc0000520d0?}, {{0xc04f446570?, 0x0?}}) /home/stuart/go/pkg/mod/github.com/filecoin-project/lotus@v1.18.0/api/proxy_gen.go:1422 +0x43 github.com/filecoin-project/lotus/blockstore.(*apiBlockstore).Get(0x0?, {0x43f01e0?, 0xc0000520d0?}, {{0xc04f446570?, 0xae3a40?}}) /home/stuart/go/pkg/mod/github.com/filecoin-project/lotus@v1.18.0/blockstore/api.go:38 +0x3a github.com/ipfs/go-ipld-cbor.(*BasicIpldStore).Get(0xc001482c00, {0x43f01e0?, 0xc0000520d0?}, {{0xc04f446570?, 0x48?}}, {0x3842c80?, 0xc0bfa96c30?}) /home/stuart/go/pkg/mod/github.com/ipfs/go-ipld-cbor@v0.0.6/store.go:63 +0xf6 github.com/filecoin-project/go-amt-ipld/v4.(*link).load(0xc04f4520e0, {0x43f01e0, 0xc0000520d0}, {0x7f99efd24308, 0xc01d571b60}, 0x7f99efd24308?, 0x0) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-amt-ipld/v4@v4.0.0/link.go:21 +0xca github.com/filecoin-project/go-amt-ipld/v4.(*node).get(0xc0000520d0?, {0x43f01e0, 0xc0000520d0}, {0x7f99efd24308, 0xc01d571b60}, 0x7f98dece3fff?, 0x400?, 0x400?, {0x7f99ef9ace30, 0xc0d1c8f6c0}) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-amt-ipld/v4@v4.0.0/node.go:225 +0x206 github.com/filecoin-project/go-amt-ipld/v4.(*node).get(0xc0000520d0?, {0x43f01e0, 0xc0000520d0}, {0x7f99efd24308, 0xc01d571b60}, 0x68f0cb?, 0x7f98dece3fff?, 0x6a94d0?, {0x7f99ef9ace30, 0xc0d1c8f6c0}) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-amt-ipld/v4@v4.0.0/node.go:235 +0x26a github.com/filecoin-project/go-amt-ipld/v4.(*node).get(0xc0000520d0?, {0x43f01e0, 0xc0000520d0}, {0x7f99efd24308, 0xc01d571b60}, 0x6a0445?, 0x68b8c5?, 0x400?, {0x7f99ef9ace30, 0xc0d1c8f6c0}) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-amt-ipld/v4@v4.0.0/node.go:235 +0x26a github.com/filecoin-project/go-amt-ipld/v4.(*node).get(0x2?, {0x43f01e0, 0xc0000520d0}, {0x7f99efd24308, 0xc01d571b60}, 0xc032cf9308?, 0x6829b6?, 0x3878e00?, {0x7f99ef9ace30, 0xc0d1c8f6c0}) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-amt-ipld/v4@v4.0.0/node.go:235 +0x26a github.com/filecoin-project/go-amt-ipld/v4.(*Root).Get(0x684c05?, {0x43f01e0?, 0xc0000520d0?}, 0xc032cf93a8?, {0x7f99ef9ace30?, 0xc0d1c8f6c0?}) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-amt-ipld/v4@v4.0.0/amt.go:225 +0xa9 github.com/filecoin-project/go-state-types/builtin/v9/util/adt.(*Array).Get(0xc065539b78, 0x1?, {0x43d8a00?, 0xc0d1c8f6c0}) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-state-types@v0.9.8/builtin/v9/util/adt/array.go:130 +0xa9 github.com/filecoin-project/lotus/chain/actors/builtin/market.(*dealStates9).Get(0xc01e34f470, 0xc0c3bfb980?) /home/stuart/go/pkg/mod/github.com/filecoin-project/lotus@v1.18.0/chain/actors/builtin/market/v9.go:154 +0x57 github.com/filecoin-project/lotus/chain/events/state.(*StatePredicates).DealStateChangedForIDs.func1({0xc01145bc40?, 0xc0bf2cda20?}, {0x43f0de8, 0xc01e34f3b0}, {0x43f0de8, 0xc01e34f470}) /home/stuart/go/pkg/mod/github.com/filecoin-project/lotus@v1.18.0/chain/events/state/predicates.go:239 +0xcd github.com/filecoin-project/lotus/markets/storageadapter.(*dealStateMatcher).matcher.func1(0xc015a63e40, 0xc01145b980) /home/stuart/go/pkg/mod/github.com/filecoin-project/lotus@v1.18.0/markets/storageadapter/dealstatematcher.go:54 +0x2f7 github.com/filecoin-project/lotus/chain/events.(*watcherEvents).checkStateChanges(0xc001150498, 0xc0013f4ba0?, 0xc03cc82180?) /home/stuart/go/pkg/mod/github.com/filecoin-project/lotus@v1.18.0/chain/events/events_called.go:352 +0x151 github.com/filecoin-project/lotus/chain/events.(*hcEventsObserver).Apply(0xc001150410, {0x43f0250, 0xc0766537d0}, 0xc015a63e40, 0xc01145b980) /home/stuart/go/pkg/mod/github.com/filecoin-project/lotus@v1.18.0/chain/events/events_called.go:127 +0x145 github.com/filecoin-project/lotus/chain/events.(*observer).headChange(0xc000d5c460, {0x43f01a8?, 0xc000d332c0?}, {0xc0cb104de0, 0x1, 0xc0002ce660?}, {0xc0cb104de8, 0x1, 0x3?}) /home/stuart/go/pkg/mod/github.com/filecoin-project/lotus@v1.18.0/chain/events/observer.go:234 +0xc09 github.com/filecoin-project/lotus/chain/events.(*observer).applyChanges(0xc0009db0e0?, {0x43f01a8, 0xc000d332c0}, {0xc05c89f180, 0x2, 0x1?}) /home/stuart/go/pkg/mod/github.com/filecoin-project/lotus@v1.18.0/chain/events/observer.go:153 +0x385 github.com/filecoin-project/lotus/chain/events.(*observer).listenHeadChangesOnce(0xc000d5c460, {0x43f01a8?, 0xc000d33280?}) /home/stuart/go/pkg/mod/github.com/filecoin-project/lotus@v1.18.0/chain/events/observer.go:127 +0x7f2 github.com/filecoin-project/lotus/chain/events.(*observer).listenHeadChanges(0x13027c0?, {0x43f01a8, 0xc000d33280}) /home/stuart/go/pkg/mod/github.com/filecoin-project/lotus@v1.18.0/chain/events/observer.go:58 +0xa5 created by github.com/filecoin-project/lotus/chain/events.(*observer).start /home/stuart/go/pkg/mod/github.com/filecoin-project/lotus@v1.18.0/chain/events/observer.go:45 +0xa5 goroutine 189 [select]: github.com/libp2p/go-libp2p/p2p/host/autonat.(*AmbientAutoNAT).background(0xc00025c2a0) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/host/autonat/autonat.go:180 +0x319 created by github.com/libp2p/go-libp2p/p2p/host/autonat.New /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/host/autonat/autonat.go:129 +0x78a goroutine 203 [chan receive, 6 minutes]: github.com/filecoin-project/go-jsonrpc.(*wsConn).handleCtxAsync(0x0?, {0x43f0250?, 0xc0013f4d80?}, {0x37406a0, 0xc000fccc58}) /home/stuart/go/pkg/mod/github.com/nonsense/go-jsonrpc@v0.0.0-20221118154859-f3f8e9b1cb11/websocket.go:283 +0x5e created by github.com/filecoin-project/go-jsonrpc.(*wsConn).handleResponse /home/stuart/go/pkg/mod/github.com/nonsense/go-jsonrpc@v0.0.0-20221118154859-f3f8e9b1cb11/websocket.go:370 +0x305 goroutine 191 [select]: github.com/libp2p/go-libp2p/p2p/host/pstoremanager.(*PeerstoreManager).background(0xc000a490e0, {0x43f01a8, 0xc000bd8c00}, {0x43e5e68?, 0xc0013dadb0}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/host/pstoremanager/pstoremanager.go:95 +0x2de created by github.com/libp2p/go-libp2p/p2p/host/pstoremanager.(*PeerstoreManager).Start /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/host/pstoremanager/pstoremanager.go:77 +0x1d5 goroutine 192 [select]: github.com/libp2p/go-libp2p/p2p/host/basic.(*BasicHost).background(0xc00034b980) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/host/basic/basic_host.go:521 +0x20b created by github.com/libp2p/go-libp2p/p2p/host/basic.(*BasicHost).Start /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/host/basic/basic_host.go:362 +0x76 goroutine 193 [select, 6 minutes]: github.com/jbenet/goprocess/context.CloseAfterContext.func1() /home/stuart/go/pkg/mod/github.com/jbenet/goprocess@v0.1.4/context/context.go:65 +0x86 created by github.com/jbenet/goprocess/context.CloseAfterContext /home/stuart/go/pkg/mod/github.com/jbenet/goprocess@v0.1.4/context/context.go:64 +0xf5 goroutine 763 [select, 6 minutes]: github.com/filecoin-project/go-statemachine.(*StateMachine).run(0xc0143825f0) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/machine.go:53 +0x130 created by github.com/filecoin-project/go-statemachine.(*StateGroup).loadOrCreate /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/group.go:132 +0x570 goroutine 243 [select, 6 minutes]: github.com/jbenet/goprocess/context.CloseAfterContext.func1() /home/stuart/go/pkg/mod/github.com/jbenet/goprocess@v0.1.4/context/context.go:65 +0x86 created by github.com/jbenet/goprocess/context.CloseAfterContext /home/stuart/go/pkg/mod/github.com/jbenet/goprocess@v0.1.4/context/context.go:64 +0xf5 goroutine 4761 [select, 6 minutes]: github.com/filecoin-project/go-statemachine.(*StateMachine).run(0xc0248b0730) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/machine.go:53 +0x130 created by github.com/filecoin-project/go-statemachine.(*StateGroup).loadOrCreate /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/group.go:132 +0x570 goroutine 200 [select]: github.com/filecoin-project/go-jsonrpc.(*wsConn).handleWsConn(0xc00025e480, {0x43f01a8, 0xc000bd93c0}) /home/stuart/go/pkg/mod/github.com/nonsense/go-jsonrpc@v0.0.0-20221118154859-f3f8e9b1cb11/websocket.go:603 +0x546 created by github.com/filecoin-project/go-jsonrpc.websocketClient /home/stuart/go/pkg/mod/github.com/nonsense/go-jsonrpc@v0.0.0-20221118154859-f3f8e9b1cb11/client.go:271 +0x3e5 goroutine 246 [select, 6 minutes]: github.com/libp2p/go-libp2p-kad-dht/providers.(*ProviderManager).run(0xc000b97c20, {0x43f01a8, 0xc000bd8d80}, {0x43fe190, 0xc000b97ce0}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-kad-dht@v0.18.0/providers/providers_manager.go:150 +0x2bf github.com/libp2p/go-libp2p-kad-dht/providers.NewProviderManager.func1({0x43fe190?, 0xc000b97ce0?}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-kad-dht@v0.18.0/providers/providers_manager.go:120 +0x3b github.com/jbenet/goprocess.(*process).Go.func1() /home/stuart/go/pkg/mod/github.com/jbenet/goprocess@v0.1.4/impl-mutex.go:134 +0x36 created by github.com/jbenet/goprocess.(*process).Go /home/stuart/go/pkg/mod/github.com/jbenet/goprocess@v0.1.4/impl-mutex.go:133 +0x238 goroutine 762 [select, 6 minutes]: github.com/filecoin-project/go-statemachine.(*StateMachine).run(0xc0143825a0) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/machine.go:53 +0x130 created by github.com/filecoin-project/go-statemachine.(*StateGroup).loadOrCreate /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/group.go:132 +0x570 goroutine 1154 [select, 6 minutes]: github.com/filecoin-project/go-statemachine.(*StateMachine).run(0xc014020d20) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/machine.go:53 +0x130 created by github.com/filecoin-project/go-statemachine.(*StateGroup).loadOrCreate /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/group.go:132 +0x570 goroutine 2000 [select, 6 minutes]: github.com/filecoin-project/go-statemachine.(*StateMachine).run(0xc01a968000) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/machine.go:53 +0x130 created by github.com/filecoin-project/go-statemachine.(*StateGroup).loadOrCreate /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/group.go:132 +0x570 goroutine 1321 [select, 6 minutes]: github.com/filecoin-project/go-statemachine.(*StateMachine).run(0xc012d8d860) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/machine.go:53 +0x130 created by github.com/filecoin-project/go-statemachine.(*StateGroup).loadOrCreate /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/group.go:132 +0x570 goroutine 759 [select, 6 minutes]: github.com/filecoin-project/go-statemachine.(*StateMachine).run(0xc0143824b0) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/machine.go:53 +0x130 created by github.com/filecoin-project/go-statemachine.(*StateGroup).loadOrCreate /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/group.go:132 +0x570 goroutine 755 [select, 6 minutes]: github.com/filecoin-project/go-statemachine.(*StateMachine).run(0xc0143823c0) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/machine.go:53 +0x130 created by github.com/filecoin-project/go-statemachine.(*StateGroup).loadOrCreate /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/group.go:132 +0x570 goroutine 253 [select]: github.com/libp2p/go-libp2p-kad-dht.(*subscriberNotifee).subscribe(0xc000be21c8, {0x43fe190, 0xc000b97e60}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-kad-dht@v0.18.0/subscriber_notifee.go:68 +0x1be github.com/jbenet/goprocess.(*process).Go.func1() /home/stuart/go/pkg/mod/github.com/jbenet/goprocess@v0.1.4/impl-mutex.go:134 +0x36 created by github.com/jbenet/goprocess.(*process).Go /home/stuart/go/pkg/mod/github.com/jbenet/goprocess@v0.1.4/impl-mutex.go:133 +0x238 goroutine 757 [select, 6 minutes]: github.com/filecoin-project/go-statemachine.(*StateMachine).run(0xc014382460) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/machine.go:53 +0x130 created by github.com/filecoin-project/go-statemachine.(*StateGroup).loadOrCreate /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/group.go:132 +0x570 goroutine 255 [select, 6 minutes]: github.com/libp2p/go-libp2p-kad-dht.(*IpfsDHT).persistRTPeersInPeerStore(0xc00096aa80) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-kad-dht@v0.18.0/dht.go:536 +0x105 created by github.com/libp2p/go-libp2p-kad-dht.New /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-kad-dht@v0.18.0/dht.go:230 +0x694 goroutine 634 [select, 5 minutes]: github.com/libp2p/go-yamux/v3.(*Session).AcceptStream(0xc0004a6120) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:262 +0xb3 github.com/libp2p/go-libp2p/p2p/muxer/yamux.(*conn).AcceptStream(0x13882a5?) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/muxer/yamux/conn.go:43 +0x19 github.com/libp2p/go-libp2p/p2p/net/swarm.(*Conn).start.func1() /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/net/swarm/swarm_conn.go:110 +0xae created by github.com/libp2p/go-libp2p/p2p/net/swarm.(*Conn).start /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/net/swarm/swarm_conn.go:105 +0x56 goroutine 633 [select, 6 minutes]: github.com/ipfs/go-graphsync/messagequeue.(*MessageQueue).runQueue(0xc000c3a0c0) /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/messagequeue/messagequeue.go:157 +0x12c created by github.com/ipfs/go-graphsync/messagequeue.(*MessageQueue).Startup /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/messagequeue/messagequeue.go:142 +0x56 goroutine 258 [select]: github.com/libp2p/go-libp2p-kad-dht.(*IpfsDHT).rtPeerLoop(0xc00096aa80, {0x43fe190, 0xc000b97f20}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-kad-dht@v0.18.0/dht.go:587 +0x132 github.com/jbenet/goprocess.(*process).Go.func1() /home/stuart/go/pkg/mod/github.com/jbenet/goprocess@v0.1.4/impl-mutex.go:134 +0x36 created by github.com/jbenet/goprocess.(*process).Go /home/stuart/go/pkg/mod/github.com/jbenet/goprocess@v0.1.4/impl-mutex.go:133 +0x238 goroutine 213 [select, 6 minutes]: github.com/filecoin-project/dagstore.RecoverImmediately({0x43f01a8, 0xc012202580}, 0xc001308060?, 0xc000c205a0, 0x1, 0xc0121ed640) /home/stuart/go/pkg/mod/github.com/filecoin-project/dagstore@v0.5.5/handlers.go:27 +0x1bf created by github.com/filecoin-project/lotus/markets/dagstore.(*Wrapper).Start /home/stuart/go/pkg/mod/github.com/filecoin-project/lotus@v1.18.0/markets/dagstore/wrapper.go:156 +0x218 goroutine 212 [select, 1 minutes]: github.com/filecoin-project/lotus/markets/dagstore.(*Wrapper).traceLoop(0xc000d4a6e0) /home/stuart/go/pkg/mod/github.com/filecoin-project/lotus@v1.18.0/markets/dagstore/wrapper.go:166 +0x292 created by github.com/filecoin-project/lotus/markets/dagstore.(*Wrapper).Start /home/stuart/go/pkg/mod/github.com/filecoin-project/lotus@v1.18.0/markets/dagstore/wrapper.go:151 +0xfb goroutine 360632 [select, 4 minutes]: github.com/libp2p/go-libp2p-pubsub.(*PubSub).handleSendingMessages(0xc062b3d710?, {0x43f01a8, 0xc000bd9640}, {0x4403a50?, 0xc070e99540}, 0xc018cc8900) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/comm.go:164 +0x305 created by github.com/libp2p/go-libp2p-pubsub.(*PubSub).handleNewPeer /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/comm.go:117 +0x334 goroutine 262 [IO wait]: internal/poll.runtime_pollWait(0x7f9a35129a48, 0x72) /usr/local/go/src/runtime/netpoll.go:302 +0x89 internal/poll.(*pollDesc).wait(0xc00098b500?, 0xc09b9e1d60?, 0x0) /usr/local/go/src/internal/poll/fd_poll_runtime.go:83 +0x32 internal/poll.(*pollDesc).waitRead(...) /usr/local/go/src/internal/poll/fd_poll_runtime.go:88 internal/poll.(*FD).Accept(0xc00098b500) /usr/local/go/src/internal/poll/fd_unix.go:614 +0x22c net.(*netFD).accept(0xc00098b500) /usr/local/go/src/net/fd_unix.go:172 +0x35 net.(*TCPListener).accept(0xc000be2228) /usr/local/go/src/net/tcpsock_posix.go:139 +0x28 net.(*TCPListener).Accept(0xc000be2228) /usr/local/go/src/net/tcpsock.go:288 +0x3d github.com/multiformats/go-multiaddr/net.(*maListener).Accept(0x7f9a04615470?) /home/stuart/go/pkg/mod/github.com/multiformats/go-multiaddr@v0.6.0/net/net.go:243 +0x33 github.com/libp2p/go-libp2p/p2p/transport/tcp.(*tcpListener).Accept(0xc000be2288) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/transport/tcp/tcp.go:80 +0x2d github.com/libp2p/go-libp2p/p2p/transport/tcp.(*tracingListener).Accept(0x0?) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/transport/tcp/metrics.go:260 +0x22 github.com/libp2p/go-libp2p/p2p/net/upgrader.(*listener).handleIncoming(0xc000284620) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/net/upgrader/listener.go:74 +0x104 created by github.com/libp2p/go-libp2p/p2p/net/upgrader.(*upgrader).UpgradeListener /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/net/upgrader/upgrader.go:110 +0x225 goroutine 263 [chan receive]: github.com/libp2p/go-libp2p/p2p/net/upgrader.(*listener).Accept(0xc000284620) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/net/upgrader/listener.go:162 +0x45 github.com/libp2p/go-libp2p/p2p/net/swarm.(*Swarm).AddListenAddr.func2() /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/net/swarm/swarm_listen.go:124 +0xf5 created by github.com/libp2p/go-libp2p/p2p/net/swarm.(*Swarm).AddListenAddr /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/net/swarm/swarm_listen.go:102 +0x25c goroutine 70 [select, 5 minutes]: github.com/libp2p/go-libp2p-kad-dht/rtrefresh.(*RtRefreshManager).loop(0xc0003c8be0) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-kad-dht@v0.18.0/rtrefresh/rt_refresh_manager.go:146 +0x239 created by github.com/libp2p/go-libp2p-kad-dht/rtrefresh.(*RtRefreshManager).Start /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-kad-dht@v0.18.0/rtrefresh/rt_refresh_manager.go:88 +0x6a goroutine 211 [select]: github.com/filecoin-project/lotus/markets/dagstore.(*Wrapper).gcLoop(0xc000d4a6e0) /home/stuart/go/pkg/mod/github.com/filecoin-project/lotus@v1.18.0/markets/dagstore/wrapper.go:187 +0x14a created by github.com/filecoin-project/lotus/markets/dagstore.(*Wrapper).Start /home/stuart/go/pkg/mod/github.com/filecoin-project/lotus@v1.18.0/markets/dagstore/wrapper.go:147 +0xaf goroutine 9337 [select, 6 minutes]: github.com/filecoin-project/go-statemachine.(*StateMachine).run(0xc0233f4140) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/machine.go:53 +0x130 created by github.com/filecoin-project/go-statemachine.(*StateGroup).loadOrCreate /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/group.go:132 +0x570 goroutine 73 [select]: github.com/libp2p/go-libp2p-kad-dht.(*IpfsDHT).fixLowPeersRoutine(0xc00096aa80, {0x43fe190, 0xc0002ce240}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-kad-dht@v0.18.0/dht.go:458 +0x136 github.com/jbenet/goprocess.(*process).Go.func1() /home/stuart/go/pkg/mod/github.com/jbenet/goprocess@v0.1.4/impl-mutex.go:134 +0x36 created by github.com/jbenet/goprocess.(*process).Go /home/stuart/go/pkg/mod/github.com/jbenet/goprocess@v0.1.4/impl-mutex.go:133 +0x238 goroutine 389048 [select]: github.com/libp2p/go-yamux/v3.(*Session).sendLoop(0xc0be6c37a0) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:559 +0x5c5 github.com/libp2p/go-yamux/v3.(*Session).send(0xc01daeca80?) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:488 +0x1e created by github.com/libp2p/go-yamux/v3.newSession /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:159 +0x6aa goroutine 877699 [select]: github.com/libp2p/go-yamux/v3.(*Stream).Read(0xc0d89de0e0, {0xc00e4d8000, 0x1000, 0x1000}) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/stream.go:111 +0x1a6 github.com/libp2p/go-libp2p/p2p/muxer/yamux.(*stream).Read(0xc01e0fb6b0?, {0xc00e4d8000?, 0x6fa3dc?, 0x0?}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/muxer/yamux/stream.go:17 +0x1e github.com/libp2p/go-libp2p/p2p/net/swarm.(*Stream).Read(0xc0e8642200, {0xc00e4d8000?, 0x59?, 0xc028c3fdc0?}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/net/swarm/swarm_stream.go:55 +0x36 github.com/multiformats/go-multistream.(*lazyClientConn).Read(0xc05fcdbb90?, {0xc00e4d8000?, 0x1000?, 0x1000?}) /home/stuart/go/pkg/mod/github.com/multiformats/go-multistream@v0.3.3/lazyClient.go:68 +0xa3 github.com/libp2p/go-libp2p/p2p/host/basic.(*streamWrapper).Read(0x681aad?, {0xc00e4d8000?, 0xc09ff21f10?, 0x7f9a5c6ea1d8?}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/host/basic/basic_host.go:1079 +0x27 bufio.(*Reader).fill(0xc014c51b00) /usr/local/go/src/bufio/bufio.go:106 +0x103 bufio.(*Reader).ReadByte(0xc014c51b00) /usr/local/go/src/bufio/bufio.go:264 +0x2c github.com/multiformats/go-varint.ReadUvarint({0x43d7240, 0xc014c51b00}) /home/stuart/go/pkg/mod/github.com/multiformats/go-varint@v0.0.6/varint.go:80 +0x74 github.com/libp2p/go-msgio/protoio.(*uvarintReader).ReadMsg(0xc0debe5a00, {0x43ebb38, 0xc09ff21f10}) /home/stuart/go/pkg/mod/github.com/libp2p/go-msgio@v0.2.0/protoio/uvarint_reader.go:70 +0x85 github.com/libp2p/go-libp2p-pubsub.(*PubSub).handlePeerEOF(0xc00031da00, {0x0?, 0x0?}, {0x4403a50, 0xc0233a70c0}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/comm.go:139 +0x144 created by github.com/libp2p/go-libp2p-pubsub.(*PubSub).handleNewPeer /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/comm.go:118 +0x3db goroutine 76 [select]: github.com/filecoin-project/go-jsonrpc.(*wsConn).setupPings.func3() /home/stuart/go/pkg/mod/github.com/nonsense/go-jsonrpc@v0.0.0-20221118154859-f3f8e9b1cb11/websocket.go:494 +0x90 created by github.com/filecoin-project/go-jsonrpc.(*wsConn).setupPings /home/stuart/go/pkg/mod/github.com/nonsense/go-jsonrpc@v0.0.0-20221118154859-f3f8e9b1cb11/websocket.go:492 +0x10d goroutine 584 [select, 6 minutes]: github.com/filecoin-project/go-statemachine.(*StateMachine).run(0xc013f040a0) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/machine.go:53 +0x130 created by github.com/filecoin-project/go-statemachine.(*StateGroup).loadOrCreate /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/group.go:132 +0x570 goroutine 206 [select, 1 minutes]: github.com/ipfs/go-graphsync/requestmanager.(*RequestManager).run(0xc000ae73b0) /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/requestmanager/server.go:48 +0xf2 created by github.com/ipfs/go-graphsync/requestmanager.(*RequestManager).Startup /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/requestmanager/client.go:366 +0x56 goroutine 207 [select]: github.com/ipfs/go-graphsync/taskqueue.(*WorkerTaskQueue).worker(0xc000d5c6e0, {0x43dc060, 0xc00dd3f780}) /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/taskqueue/taskqueue.go:121 +0x18c created by github.com/ipfs/go-graphsync/taskqueue.(*WorkerTaskQueue).Startup /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/taskqueue/taskqueue.go:97 +0x3a goroutine 208 [select]: github.com/ipfs/go-graphsync/taskqueue.(*WorkerTaskQueue).worker(0xc000d5c6e0, {0x43dc060, 0xc00dd3f780}) /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/taskqueue/taskqueue.go:121 +0x18c created by github.com/ipfs/go-graphsync/taskqueue.(*WorkerTaskQueue).Startup /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/taskqueue/taskqueue.go:97 +0x3a goroutine 209 [select]: github.com/ipfs/go-graphsync/taskqueue.(*WorkerTaskQueue).worker(0xc000d5c6e0, {0x43dc060, 0xc00dd3f780}) /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/taskqueue/taskqueue.go:121 +0x18c created by github.com/ipfs/go-graphsync/taskqueue.(*WorkerTaskQueue).Startup /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/taskqueue/taskqueue.go:97 +0x3a goroutine 274 [select]: github.com/ipfs/go-graphsync/taskqueue.(*WorkerTaskQueue).worker(0xc000d5c6e0, {0x43dc060, 0xc00dd3f780}) /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/taskqueue/taskqueue.go:121 +0x18c created by github.com/ipfs/go-graphsync/taskqueue.(*WorkerTaskQueue).Startup /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/taskqueue/taskqueue.go:97 +0x3a goroutine 275 [select]: github.com/ipfs/go-graphsync/taskqueue.(*WorkerTaskQueue).worker(0xc000d5c6e0, {0x43dc060, 0xc00dd3f780}) /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/taskqueue/taskqueue.go:121 +0x18c created by github.com/ipfs/go-graphsync/taskqueue.(*WorkerTaskQueue).Startup /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/taskqueue/taskqueue.go:97 +0x3a goroutine 276 [select, 1 minutes]: github.com/ipfs/go-graphsync/responsemanager.(*ResponseManager).run(0xc000454120) /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/responsemanager/server.go:39 +0xf0 created by github.com/ipfs/go-graphsync/responsemanager.(*ResponseManager).Startup /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/responsemanager/client.go:276 +0x56 goroutine 277 [select]: github.com/ipfs/go-graphsync/taskqueue.(*WorkerTaskQueue).worker(0xc000d5c780, {0x43dc2e0, 0xc000d33e00}) /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/taskqueue/taskqueue.go:121 +0x18c created by github.com/ipfs/go-graphsync/taskqueue.(*WorkerTaskQueue).Startup /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/taskqueue/taskqueue.go:97 +0x3a goroutine 266 [select]: github.com/libp2p/go-libp2p-pubsub.(*peerGater).background(0xc001308780, {0x43f01a8, 0xc000bd9640}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/peer_gater.go:210 +0x105 created by github.com/libp2p/go-libp2p-pubsub.newPeerGater /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/peer_gater.go:200 +0x14d goroutine 80 [select, 6 minutes]: github.com/libp2p/go-yamux/v3.(*Stream).Read(0xc0011942a0, {0xc006c16000, 0x1000, 0x1000}) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/stream.go:111 +0x1a6 github.com/libp2p/go-libp2p/p2p/muxer/yamux.(*stream).Read(0xc00034c8b0?, {0xc006c16000?, 0x6fa3dc?, 0x0?}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/muxer/yamux/stream.go:17 +0x1e github.com/libp2p/go-libp2p/p2p/net/swarm.(*Stream).Read(0xc011fb3c00, {0xc006c16000?, 0x59?, 0x37482a0?}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/net/swarm/swarm_stream.go:55 +0x36 github.com/multiformats/go-multistream.(*lazyClientConn).Read(0xc000104700?, {0xc006c16000?, 0x1000?, 0x1000?}) /home/stuart/go/pkg/mod/github.com/multiformats/go-multistream@v0.3.3/lazyClient.go:68 +0xa3 github.com/libp2p/go-libp2p/p2p/host/basic.(*streamWrapper).Read(0x681b93?, {0xc006c16000?, 0xc000104770?, 0x7f9a5c6e93c8?}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/host/basic/basic_host.go:1079 +0x27 bufio.(*Reader).fill(0xc0002cef00) /usr/local/go/src/bufio/bufio.go:106 +0x103 bufio.(*Reader).ReadByte(0xc0002cef00) /usr/local/go/src/bufio/bufio.go:264 +0x2c github.com/multiformats/go-varint.ReadUvarint({0x43d7240, 0xc0002cef00}) /home/stuart/go/pkg/mod/github.com/multiformats/go-varint@v0.0.6/varint.go:80 +0x74 github.com/libp2p/go-msgio/protoio.(*uvarintReader).ReadMsg(0xc0002d9140, {0x43ebb38, 0xc000104770}) /home/stuart/go/pkg/mod/github.com/libp2p/go-msgio@v0.2.0/protoio/uvarint_reader.go:70 +0x85 github.com/libp2p/go-libp2p-pubsub.(*PubSub).handlePeerEOF(0xc00031da00, {0x12?, 0x0?}, {0x4403a50, 0xc00046d280}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/comm.go:139 +0x144 created by github.com/libp2p/go-libp2p-pubsub.(*PubSub).handleNewPeer /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/comm.go:118 +0x3db goroutine 267 [select]: github.com/libp2p/go-libp2p-pubsub.(*peerScore).background(0xc001308720, {0x43f01a8, 0xc000bd9640}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/score.go:428 +0x2b2 created by github.com/libp2p/go-libp2p-pubsub.(*peerScore).Start /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/score.go:251 +0xfa goroutine 265 [select]: github.com/libp2p/go-libp2p-pubsub.(*backoff).cleanupLoop(0xc0013dfb90, {0x43f01a8, 0xc000bd9640}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/backoff.go:100 +0xe7 created by github.com/libp2p/go-libp2p-pubsub.newBackoff /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/backoff.go:47 +0x165 goroutine 268 [select]: github.com/libp2p/go-libp2p-pubsub.(*GossipSubRouter).heartbeatTimer(0xc0003b0780) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/gossipsub.go:1324 +0x1f5 created by github.com/libp2p/go-libp2p-pubsub.(*GossipSubRouter).Attach /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/gossipsub.go:497 +0x1a5 goroutine 269 [select, 6 minutes]: github.com/libp2p/go-libp2p-pubsub.(*GossipSubRouter).connector(0xc0003b0780) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/gossipsub.go:939 +0xd1 created by github.com/libp2p/go-libp2p-pubsub.(*GossipSubRouter).Attach /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/gossipsub.go:501 +0x1ae goroutine 270 [select, 6 minutes]: github.com/libp2p/go-libp2p-pubsub.(*GossipSubRouter).connector(0xc0003b0780) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/gossipsub.go:939 +0xd1 created by github.com/libp2p/go-libp2p-pubsub.(*GossipSubRouter).Attach /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/gossipsub.go:501 +0x1ae goroutine 271 [select, 6 minutes]: github.com/libp2p/go-libp2p-pubsub.(*GossipSubRouter).connector(0xc0003b0780) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/gossipsub.go:939 +0xd1 created by github.com/libp2p/go-libp2p-pubsub.(*GossipSubRouter).Attach /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/gossipsub.go:501 +0x1ae goroutine 272 [select, 6 minutes]: github.com/libp2p/go-libp2p-pubsub.(*GossipSubRouter).connector(0xc0003b0780) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/gossipsub.go:939 +0xd1 created by github.com/libp2p/go-libp2p-pubsub.(*GossipSubRouter).Attach /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/gossipsub.go:501 +0x1ae goroutine 273 [select, 6 minutes]: github.com/libp2p/go-libp2p-pubsub.(*GossipSubRouter).connector(0xc0003b0780) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/gossipsub.go:939 +0xd1 created by github.com/libp2p/go-libp2p-pubsub.(*GossipSubRouter).Attach /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/gossipsub.go:501 +0x1ae goroutine 290 [select, 6 minutes]: github.com/libp2p/go-libp2p-pubsub.(*GossipSubRouter).connector(0xc0003b0780) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/gossipsub.go:939 +0xd1 created by github.com/libp2p/go-libp2p-pubsub.(*GossipSubRouter).Attach /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/gossipsub.go:501 +0x1ae goroutine 291 [select, 6 minutes]: github.com/libp2p/go-libp2p-pubsub.(*GossipSubRouter).connector(0xc0003b0780) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/gossipsub.go:939 +0xd1 created by github.com/libp2p/go-libp2p-pubsub.(*GossipSubRouter).Attach /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/gossipsub.go:501 +0x1ae goroutine 292 [select, 6 minutes]: github.com/libp2p/go-libp2p-pubsub.(*GossipSubRouter).connector(0xc0003b0780) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/gossipsub.go:939 +0xd1 created by github.com/libp2p/go-libp2p-pubsub.(*GossipSubRouter).Attach /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/gossipsub.go:501 +0x1ae goroutine 293 [select, 6 minutes]: github.com/libp2p/go-libp2p-pubsub.(*validation).validateWorker(0xc000bd96c0) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/validation.go:264 +0xc5 created by github.com/libp2p/go-libp2p-pubsub.(*validation).Start /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/validation.go:133 +0x5a goroutine 294 [select, 6 minutes]: github.com/libp2p/go-libp2p-pubsub.(*validation).validateWorker(0xc000bd96c0) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/validation.go:264 +0xc5 created by github.com/libp2p/go-libp2p-pubsub.(*validation).Start /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/validation.go:133 +0x5a goroutine 295 [select, 6 minutes]: github.com/libp2p/go-libp2p-pubsub.(*validation).validateWorker(0xc000bd96c0) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/validation.go:264 +0xc5 created by github.com/libp2p/go-libp2p-pubsub.(*validation).Start /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/validation.go:133 +0x5a goroutine 296 [select, 6 minutes]: github.com/libp2p/go-libp2p-pubsub.(*validation).validateWorker(0xc000bd96c0) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/validation.go:264 +0xc5 created by github.com/libp2p/go-libp2p-pubsub.(*validation).Start /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/validation.go:133 +0x5a goroutine 297 [select, 6 minutes]: github.com/libp2p/go-libp2p-pubsub.(*validation).validateWorker(0xc000bd96c0) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/validation.go:264 +0xc5 created by github.com/libp2p/go-libp2p-pubsub.(*validation).Start /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/validation.go:133 +0x5a goroutine 298 [select, 6 minutes]: github.com/libp2p/go-libp2p-pubsub.(*validation).validateWorker(0xc000bd96c0) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/validation.go:264 +0xc5 created by github.com/libp2p/go-libp2p-pubsub.(*validation).Start /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/validation.go:133 +0x5a goroutine 299 [select, 6 minutes]: github.com/libp2p/go-libp2p-pubsub.(*validation).validateWorker(0xc000bd96c0) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/validation.go:264 +0xc5 created by github.com/libp2p/go-libp2p-pubsub.(*validation).Start /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/validation.go:133 +0x5a goroutine 300 [select, 6 minutes]: github.com/libp2p/go-libp2p-pubsub.(*validation).validateWorker(0xc000bd96c0) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/validation.go:264 +0xc5 created by github.com/libp2p/go-libp2p-pubsub.(*validation).Start /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/validation.go:133 +0x5a goroutine 301 [select, 6 minutes]: github.com/libp2p/go-libp2p-pubsub.(*validation).validateWorker(0xc000bd96c0) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/validation.go:264 +0xc5 created by github.com/libp2p/go-libp2p-pubsub.(*validation).Start /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/validation.go:133 +0x5a goroutine 302 [select, 6 minutes]: github.com/libp2p/go-libp2p-pubsub.(*validation).validateWorker(0xc000bd96c0) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/validation.go:264 +0xc5 created by github.com/libp2p/go-libp2p-pubsub.(*validation).Start /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/validation.go:133 +0x5a goroutine 303 [select, 6 minutes]: github.com/libp2p/go-libp2p-pubsub.(*validation).validateWorker(0xc000bd96c0) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/validation.go:264 +0xc5 created by github.com/libp2p/go-libp2p-pubsub.(*validation).Start /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/validation.go:133 +0x5a goroutine 304 [select, 6 minutes]: github.com/libp2p/go-libp2p-pubsub.(*validation).validateWorker(0xc000bd96c0) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/validation.go:264 +0xc5 created by github.com/libp2p/go-libp2p-pubsub.(*validation).Start /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/validation.go:133 +0x5a goroutine 305 [select]: github.com/libp2p/go-libp2p-pubsub.(*PubSub).processLoop(0xc00031da00, {0x43f01a8, 0xc000bd9640}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/pubsub.go:542 +0x4df created by github.com/libp2p/go-libp2p-pubsub.NewPubSub /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/pubsub.go:323 +0xdea goroutine 322 [select]: github.com/filecoin-project/go-jsonrpc.(*wsConn).handleWsConn(0xc000c3a000, {0x43f01a8, 0xc000bd9900}) /home/stuart/go/pkg/mod/github.com/nonsense/go-jsonrpc@v0.0.0-20221118154859-f3f8e9b1cb11/websocket.go:603 +0x546 created by github.com/filecoin-project/go-jsonrpc.websocketClient /home/stuart/go/pkg/mod/github.com/nonsense/go-jsonrpc@v0.0.0-20221118154859-f3f8e9b1cb11/client.go:271 +0x3e5 goroutine 307 [select]: github.com/filecoin-project/go-jsonrpc.(*wsConn).setupPings.func3() /home/stuart/go/pkg/mod/github.com/nonsense/go-jsonrpc@v0.0.0-20221118154859-f3f8e9b1cb11/websocket.go:494 +0x90 created by github.com/filecoin-project/go-jsonrpc.(*wsConn).setupPings /home/stuart/go/pkg/mod/github.com/nonsense/go-jsonrpc@v0.0.0-20221118154859-f3f8e9b1cb11/websocket.go:492 +0x10d goroutine 693505 [IO wait]: internal/poll.runtime_pollWait(0x7f9a35129b38, 0x72) /usr/local/go/src/runtime/netpoll.go:302 +0x89 internal/poll.(*pollDesc).wait(0xc00132a180?, 0xc000c26000?, 0x0) /usr/local/go/src/internal/poll/fd_poll_runtime.go:83 +0x32 internal/poll.(*pollDesc).waitRead(...) /usr/local/go/src/internal/poll/fd_poll_runtime.go:88 internal/poll.(*FD).Read(0xc00132a180, {0xc000c26000, 0x1000, 0x1000}) /usr/local/go/src/internal/poll/fd_unix.go:167 +0x25a net.(*netFD).Read(0xc00132a180, {0xc000c26000?, 0xc001327d98?, 0x67d28d?}) /usr/local/go/src/net/fd_posix.go:55 +0x29 net.(*conn).Read(0xc000c24000, {0xc000c26000?, 0x6e37e7?, 0xc0013950d8?}) /usr/local/go/src/net/net.go:183 +0x45 bufio.(*Reader).fill(0xc000c20060) /usr/local/go/src/bufio/bufio.go:106 +0x103 bufio.(*Reader).Peek(0xc000c20060, 0x2) /usr/local/go/src/bufio/bufio.go:144 +0x5d github.com/gorilla/websocket.(*Conn).read(0xc000156000, 0x0?) /home/stuart/go/pkg/mod/github.com/gorilla/websocket@v1.5.0/conn.go:371 +0x2c github.com/gorilla/websocket.(*Conn).advanceFrame(0xc000156000) /home/stuart/go/pkg/mod/github.com/gorilla/websocket@v1.5.0/conn.go:809 +0x7b github.com/gorilla/websocket.(*Conn).NextReader(0xc000156000) /home/stuart/go/pkg/mod/github.com/gorilla/websocket@v1.5.0/conn.go:1009 +0xc5 github.com/filecoin-project/go-jsonrpc.(*wsConn).nextMessage(0xc000c3a000) /home/stuart/go/pkg/mod/github.com/nonsense/go-jsonrpc@v0.0.0-20221118154859-f3f8e9b1cb11/websocket.go:100 +0x10e created by github.com/filecoin-project/go-jsonrpc.(*wsConn).handleWsConn /home/stuart/go/pkg/mod/github.com/nonsense/go-jsonrpc@v0.0.0-20221118154859-f3f8e9b1cb11/websocket.go:615 +0xd33 goroutine 323 [select]: github.com/filecoin-project/lotus/storage/paths.(*Local).reportHealth(0x0?, {0x43f01a8, 0xc000c300c0}) /home/stuart/go/pkg/mod/github.com/filecoin-project/lotus@v1.18.0/storage/paths/local.go:426 +0xf3 created by github.com/filecoin-project/lotus/storage/paths.(*Local).open /home/stuart/go/pkg/mod/github.com/filecoin-project/lotus@v1.18.0/storage/paths/local.go:297 +0x12a goroutine 324 [select, 1 minutes]: github.com/syndtr/goleveldb/leveldb.(*session).refLoop(0xc000c74000) /home/stuart/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20210819022825-2ae1ddf74ef7/leveldb/session_util.go:189 +0x59b created by github.com/syndtr/goleveldb/leveldb.newSession /home/stuart/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20210819022825-2ae1ddf74ef7/leveldb/session.go:93 +0x2d9 goroutine 284 [select, 1 minutes]: github.com/syndtr/goleveldb/leveldb.(*DB).compactionError(0xc000acc000) /home/stuart/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20210819022825-2ae1ddf74ef7/leveldb/db_compaction.go:91 +0x158 created by github.com/syndtr/goleveldb/leveldb.openDB /home/stuart/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20210819022825-2ae1ddf74ef7/leveldb/db.go:148 +0x4ea goroutine 285 [select]: github.com/syndtr/goleveldb/leveldb.(*DB).mpoolDrain(0xc000acc000) /home/stuart/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20210819022825-2ae1ddf74ef7/leveldb/db_state.go:101 +0xa8 created by github.com/syndtr/goleveldb/leveldb.openDB /home/stuart/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20210819022825-2ae1ddf74ef7/leveldb/db.go:149 +0x52a goroutine 286 [select, 1 minutes]: github.com/syndtr/goleveldb/leveldb.(*DB).tCompaction(0xc000acc000) /home/stuart/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20210819022825-2ae1ddf74ef7/leveldb/db_compaction.go:836 +0x657 created by github.com/syndtr/goleveldb/leveldb.openDB /home/stuart/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20210819022825-2ae1ddf74ef7/leveldb/db.go:155 +0x598 goroutine 287 [select, 1 minutes]: github.com/syndtr/goleveldb/leveldb.(*DB).mCompaction(0xc000acc000) /home/stuart/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20210819022825-2ae1ddf74ef7/leveldb/db_compaction.go:773 +0x113 created by github.com/syndtr/goleveldb/leveldb.openDB /home/stuart/go/pkg/mod/github.com/syndtr/goleveldb@v1.0.1-0.20210819022825-2ae1ddf74ef7/leveldb/db.go:156 +0x5d6 goroutine 288 [select, 6 minutes]: database/sql.(*DB).connectionOpener(0xc0013b0a90, {0x43f01a8, 0xc006ab28c0}) /usr/local/go/src/database/sql/sql.go:1226 +0x8d created by database/sql.OpenDB /usr/local/go/src/database/sql/sql.go:794 +0x18d goroutine 338 [select, 6 minutes]: database/sql.(*DB).connectionOpener(0xc0013b1e10, {0x43f01a8, 0xc006ab2d80}) /usr/local/go/src/database/sql/sql.go:1226 +0x8d created by database/sql.OpenDB /usr/local/go/src/database/sql/sql.go:794 +0x18d goroutine 339 [select, 6 minutes]: database/sql.(*DB).connectionOpener(0xc0013c21a0, {0x43f01a8, 0xc006ab2e40}) /usr/local/go/src/database/sql/sql.go:1226 +0x8d created by database/sql.OpenDB /usr/local/go/src/database/sql/sql.go:794 +0x18d goroutine 409180 [select, 3 minutes]: github.com/libp2p/go-libp2p/p2p/protocol/identify.(*peerHandler).loop(0xc0bca079a0, {0x43f01a8, 0xc00f81c840}, 0xc011ead600?) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/protocol/identify/peer_loop.go:89 +0x105 created by github.com/libp2p/go-libp2p/p2p/protocol/identify.(*peerHandler).start /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/protocol/identify/peer_loop.go:71 +0xf0 goroutine 15 [select]: github.com/filecoin-project/lotus/node/modules.graphsyncStats.func1.1() /home/stuart/go/pkg/mod/github.com/filecoin-project/lotus@v1.18.0/node/modules/graphsync.go:71 +0x108 created by github.com/filecoin-project/lotus/node/modules.graphsyncStats.func1 /home/stuart/go/pkg/mod/github.com/filecoin-project/lotus@v1.18.0/node/modules/graphsync.go:68 +0xcd goroutine 353 [select, 6 minutes]: github.com/filecoin-project/dagstore.(*DAGStore).dispatcher(0xc0009669a0, 0xc006ad14a0) /home/stuart/go/pkg/mod/github.com/filecoin-project/dagstore@v0.5.5/dispatcher.go:11 +0x108 created by github.com/filecoin-project/dagstore.(*DAGStore).Start /home/stuart/go/pkg/mod/github.com/filecoin-project/dagstore@v0.5.5/dagstore.go:320 +0x970 goroutine 85923 [IO wait]: internal/poll.runtime_pollWait(0x7f99ef9793b0, 0x72) /usr/local/go/src/runtime/netpoll.go:302 +0x89 internal/poll.(*pollDesc).wait(0xc00f22e080?, 0xc011536000?, 0x0) /usr/local/go/src/internal/poll/fd_poll_runtime.go:83 +0x32 internal/poll.(*pollDesc).waitRead(...) /usr/local/go/src/internal/poll/fd_poll_runtime.go:88 internal/poll.(*FD).Read(0xc00f22e080, {0xc011536000, 0x1000, 0x1000}) /usr/local/go/src/internal/poll/fd_unix.go:167 +0x25a net.(*netFD).Read(0xc00f22e080, {0xc011536000?, 0xc0237e4700?, 0x1c?}) /usr/local/go/src/net/fd_posix.go:55 +0x29 net.(*conn).Read(0xc02247ae08, {0xc011536000?, 0xc014f11d50?, 0xc014f11d48?}) /usr/local/go/src/net/net.go:183 +0x45 bufio.(*Reader).Read(0xc022648e40, {0xc00f776750, 0x2, 0x6fa62e?}) /usr/local/go/src/bufio/bufio.go:236 +0x1b4 io.ReadAtLeast({0x43d7260, 0xc022648e40}, {0xc00f776750, 0x2, 0x2}, 0x2) /usr/local/go/src/io/io.go:331 +0x9a io.ReadFull(...) /usr/local/go/src/io/io.go:350 github.com/libp2p/go-libp2p/p2p/security/noise.(*secureSession).readNextInsecureMsgLen(0xc00f7766c0) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/security/noise/rw.go:133 +0x3b github.com/libp2p/go-libp2p/p2p/security/noise.(*secureSession).Read(0xc00f7766c0, {0xc010699c00, 0xc, 0xc}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/security/noise/rw.go:52 +0x22f io.ReadAtLeast({0x7f9a049c0680, 0xc00f7766c0}, {0xc010699c00, 0xc, 0xc}, 0xc) /usr/local/go/src/io/io.go:331 +0x9a io.ReadFull(...) /usr/local/go/src/io/io.go:350 github.com/libp2p/go-yamux/v3.(*Session).recvLoop(0xc016ecafc0) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:644 +0xf7 github.com/libp2p/go-yamux/v3.(*Session).recv(0xc0247d0180?) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:616 +0x1e created by github.com/libp2p/go-yamux/v3.newSession /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:158 +0x66a goroutine 351 [select]: github.com/filecoin-project/dagstore.(*DAGStore).consumeNext(0xc0009669a0) /home/stuart/go/pkg/mod/github.com/filecoin-project/dagstore@v0.5.5/dagstore_control.go:356 +0x205 github.com/filecoin-project/dagstore.(*DAGStore).control(0xc0009669a0) /home/stuart/go/pkg/mod/github.com/filecoin-project/dagstore@v0.5.5/dagstore_control.go:47 +0x105 created by github.com/filecoin-project/dagstore.(*DAGStore).Start /home/stuart/go/pkg/mod/github.com/filecoin-project/dagstore@v0.5.5/dagstore.go:309 +0x805 goroutine 78 [select, 6 minutes]: github.com/libp2p/go-libp2p/p2p/protocol/identify.(*peerHandler).loop(0xc0002da640, {0x43f01a8, 0xc0002d8900}, 0xc0003b0780?) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/protocol/identify/peer_loop.go:89 +0x105 created by github.com/libp2p/go-libp2p/p2p/protocol/identify.(*peerHandler).start /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/protocol/identify/peer_loop.go:71 +0xf0 goroutine 379 [select, 6 minutes]: github.com/ipfs/go-graphsync/messagequeue.(*MessageQueue).runQueue(0xc00025efc0) /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/messagequeue/messagequeue.go:157 +0x12c created by github.com/ipfs/go-graphsync/messagequeue.(*MessageQueue).Startup /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/messagequeue/messagequeue.go:142 +0x56 goroutine 352 [select, 1 minutes]: github.com/filecoin-project/dagstore.(*DAGStore).dispatcher(0xc0009669a0, 0xc006ad0b40) /home/stuart/go/pkg/mod/github.com/filecoin-project/dagstore@v0.5.5/dispatcher.go:11 +0x108 created by github.com/filecoin-project/dagstore.(*DAGStore).Start /home/stuart/go/pkg/mod/github.com/filecoin-project/dagstore@v0.5.5/dagstore.go:314 +0x88e goroutine 2613 [select, 6 minutes]: github.com/filecoin-project/go-statemachine.(*StateMachine).run(0xc01c9dcb90) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/machine.go:53 +0x130 created by github.com/filecoin-project/go-statemachine.(*StateGroup).loadOrCreate /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/group.go:132 +0x570 goroutine 326 [select, 5 minutes]: github.com/libp2p/go-libp2p-gostream.(*listener).Accept(0xc0122aa7c0) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-gostream@v0.5.0/listener.go:27 +0x91 net/http.(*Server).Serve(0xc0011941c0, {0x43eb8f8, 0xc0122aa7c0}) /usr/local/go/src/net/http/server.go:3039 +0x385 github.com/filecoin-project/go-legs/p2p/protocol/head.(*Publisher).Serve(0xc012027080, {0x4402ea0, 0xc000cfd520}, {0xc00090cca0?, 0xc006ab9700?}) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-legs@v0.4.9/p2p/protocol/head/head.go:53 +0x34e github.com/filecoin-project/go-legs/dtsync.startHeadPublisher.func1() /home/stuart/go/pkg/mod/github.com/filecoin-project/go-legs@v0.4.9/dtsync/publisher.go:86 +0x185 created by github.com/filecoin-project/go-legs/dtsync.startHeadPublisher /home/stuart/go/pkg/mod/github.com/filecoin-project/go-legs@v0.4.9/dtsync/publisher.go:84 +0xc5 goroutine 316364 [select, 4 minutes]: github.com/libp2p/go-libp2p/p2p/protocol/identify.(*peerHandler).loop(0xc0d809ab90, {0x43f01a8, 0xc011925480}, 0xc00dd6cd00?) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/protocol/identify/peer_loop.go:89 +0x105 created by github.com/libp2p/go-libp2p/p2p/protocol/identify.(*peerHandler).start /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/protocol/identify/peer_loop.go:71 +0xf0 goroutine 334 [select]: github.com/filecoin-project/go-fil-markets/storagemarket/impl.(*Provider).Start.func2() /home/stuart/go/pkg/mod/github.com/filecoin-project/go-fil-markets@v1.25.1/storagemarket/impl/provider.go:221 +0xb0 created by github.com/filecoin-project/go-fil-markets/storagemarket/impl.(*Provider).Start /home/stuart/go/pkg/mod/github.com/filecoin-project/go-fil-markets@v1.25.1/storagemarket/impl/provider.go:219 +0x1bb goroutine 392 [select]: github.com/filecoin-project/boost/retrievalmarket/rtvllog.(*RetrievalLog).gcUpdateMap(0xc011dbd6e0, {0x43f01a8, 0xc006ab2c80}) /home/stuart/boost/retrievalmarket/rtvllog/retrieval_log.go:193 +0x11c created by github.com/filecoin-project/boost/retrievalmarket/rtvllog.(*RetrievalLog).Start /home/stuart/boost/retrievalmarket/rtvllog/retrieval_log.go:34 +0xb5 goroutine 376 [IO wait]: internal/poll.runtime_pollWait(0x7f9a35129688, 0x72) /usr/local/go/src/runtime/netpoll.go:302 +0x89 internal/poll.(*pollDesc).wait(0xc0122fc580?, 0xc013f26000?, 0x0) /usr/local/go/src/internal/poll/fd_poll_runtime.go:83 +0x32 internal/poll.(*pollDesc).waitRead(...) /usr/local/go/src/internal/poll/fd_poll_runtime.go:88 internal/poll.(*FD).Read(0xc0122fc580, {0xc013f26000, 0x1000, 0x1000}) /usr/local/go/src/internal/poll/fd_unix.go:167 +0x25a net.(*netFD).Read(0xc0122fc580, {0xc013f26000?, 0xc0c31f8be0?, 0x1c?}) /usr/local/go/src/net/fd_posix.go:55 +0x29 net.(*conn).Read(0xc000c24360, {0xc013f26000?, 0xc011938598?, 0xc000d1dd48?}) /usr/local/go/src/net/net.go:183 +0x45 bufio.(*Reader).Read(0xc006ad1620, {0xc00025eed0, 0x2, 0x6fa62e?}) /usr/local/go/src/bufio/bufio.go:236 +0x1b4 io.ReadAtLeast({0x43d7260, 0xc006ad1620}, {0xc00025eed0, 0x2, 0x2}, 0x2) /usr/local/go/src/io/io.go:331 +0x9a io.ReadFull(...) /usr/local/go/src/io/io.go:350 github.com/libp2p/go-libp2p/p2p/security/noise.(*secureSession).readNextInsecureMsgLen(0xc00025ee40) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/security/noise/rw.go:133 +0x3b github.com/libp2p/go-libp2p/p2p/security/noise.(*secureSession).Read(0xc00025ee40, {0xc00f83d540, 0xc, 0xc}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/security/noise/rw.go:52 +0x22f io.ReadAtLeast({0x7f9a049c0680, 0xc00025ee40}, {0xc00f83d540, 0xc, 0xc}, 0xc) /usr/local/go/src/io/io.go:331 +0x9a io.ReadFull(...) /usr/local/go/src/io/io.go:350 github.com/libp2p/go-yamux/v3.(*Session).recvLoop(0xc000455e60) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:644 +0xf7 github.com/libp2p/go-yamux/v3.(*Session).recv(0x10?) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:616 +0x1e created by github.com/libp2p/go-yamux/v3.newSession /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:158 +0x66a goroutine 377 [select]: github.com/libp2p/go-yamux/v3.(*Session).sendLoop(0xc000455e60) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:559 +0x5c5 github.com/libp2p/go-yamux/v3.(*Session).send(0xc001307fb8?) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:488 +0x1e created by github.com/libp2p/go-yamux/v3.newSession /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:159 +0x6aa goroutine 866220 [IO wait]: internal/poll.runtime_pollWait(0x7f99ef9791d0, 0x72) /usr/local/go/src/runtime/netpoll.go:302 +0x89 internal/poll.(*pollDesc).wait(0xc01cd22100?, 0xc01bd38000?, 0x0) /usr/local/go/src/internal/poll/fd_poll_runtime.go:83 +0x32 internal/poll.(*pollDesc).waitRead(...) /usr/local/go/src/internal/poll/fd_poll_runtime.go:88 internal/poll.(*FD).Read(0xc01cd22100, {0xc01bd38000, 0x1000, 0x1000}) /usr/local/go/src/internal/poll/fd_unix.go:167 +0x25a net.(*netFD).Read(0xc01cd22100, {0xc01bd38000?, 0xc02504c680?, 0x1c?}) /usr/local/go/src/net/fd_posix.go:55 +0x29 net.(*conn).Read(0xc038975d70, {0xc01bd38000?, 0xc01897e0e8?, 0xc014f17d48?}) /usr/local/go/src/net/net.go:183 +0x45 bufio.(*Reader).Read(0xc01a697620, {0xc0d5afd110, 0x2, 0x6fa62e?}) /usr/local/go/src/bufio/bufio.go:236 +0x1b4 io.ReadAtLeast({0x43d7260, 0xc01a697620}, {0xc0d5afd110, 0x2, 0x2}, 0x2) /usr/local/go/src/io/io.go:331 +0x9a io.ReadFull(...) /usr/local/go/src/io/io.go:350 github.com/libp2p/go-libp2p/p2p/security/noise.(*secureSession).readNextInsecureMsgLen(0xc0d5afd080) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/security/noise/rw.go:133 +0x3b github.com/libp2p/go-libp2p/p2p/security/noise.(*secureSession).Read(0xc0d5afd080, {0xc010a5caa0, 0xc, 0xc}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/security/noise/rw.go:52 +0x22f io.ReadAtLeast({0x7f9a049c0680, 0xc0d5afd080}, {0xc010a5caa0, 0xc, 0xc}, 0xc) /usr/local/go/src/io/io.go:331 +0x9a io.ReadFull(...) /usr/local/go/src/io/io.go:350 github.com/libp2p/go-yamux/v3.(*Session).recvLoop(0xc01bcdc900) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:644 +0xf7 github.com/libp2p/go-yamux/v3.(*Session).recv(0xc000d387b8?) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:616 +0x1e created by github.com/libp2p/go-yamux/v3.newSession /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:158 +0x66a goroutine 364 [sync.Cond.Wait, 6 minutes]: sync.runtime_notifyListWait(0xc013db5f90, 0x0) /usr/local/go/src/runtime/sema.go:513 +0x13d sync.(*Cond).Wait(0x0?) /usr/local/go/src/sync/cond.go:56 +0x8c github.com/ipfs/go-graphsync/notifications.(*publisher).dequeue(0xc013db5fc0) /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/notifications/publisher.go:226 +0x6e github.com/ipfs/go-graphsync/notifications.(*publisher).start(0xc000455e60?) /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/notifications/publisher.go:122 +0x125 created by github.com/ipfs/go-graphsync/notifications.(*publisher).Startup /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/notifications/publisher.go:46 +0x56 goroutine 910420 [select]: github.com/ipfs/go-graphsync/messagequeue.(*MessageQueue).runQueue(0xc00860a300) /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/messagequeue/messagequeue.go:157 +0x12c created by github.com/ipfs/go-graphsync/messagequeue.(*MessageQueue).Startup /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/messagequeue/messagequeue.go:142 +0x56 goroutine 382 [select, 5 minutes]: github.com/libp2p/go-yamux/v3.(*Session).AcceptStream(0xc000455e60) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:262 +0xb3 github.com/libp2p/go-libp2p/p2p/muxer/yamux.(*conn).AcceptStream(0x13882a5?) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/muxer/yamux/conn.go:43 +0x19 github.com/libp2p/go-libp2p/p2p/net/swarm.(*Conn).start.func1() /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/net/swarm/swarm_conn.go:110 +0xae created by github.com/libp2p/go-libp2p/p2p/net/swarm.(*Conn).start /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/net/swarm/swarm_conn.go:105 +0x56 goroutine 333 [select]: github.com/libp2p/go-flow-metrics.(*sweeper).runActive(0x5fbdfa0) /home/stuart/go/pkg/mod/github.com/libp2p/go-flow-metrics@v0.1.0/sweeper.go:78 +0x1ed github.com/libp2p/go-flow-metrics.(*sweeper).run(0x5fbdfa0) /home/stuart/go/pkg/mod/github.com/libp2p/go-flow-metrics@v0.1.0/sweeper.go:52 +0x36 created by github.com/libp2p/go-flow-metrics.(*sweeper).start /home/stuart/go/pkg/mod/github.com/libp2p/go-flow-metrics@v0.1.0/sweeper.go:46 +0x8a goroutine 846063 [sync.Cond.Wait]: sync.runtime_notifyListWait(0xc010c97110, 0x0) /usr/local/go/src/runtime/sema.go:513 +0x13d sync.(*Cond).Wait(0x0?) /usr/local/go/src/sync/cond.go:56 +0x8c github.com/ipfs/go-graphsync/notifications.(*publisher).dequeue(0xc010c97140) /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/notifications/publisher.go:226 +0x6e github.com/ipfs/go-graphsync/notifications.(*publisher).start(0xc020a0ba00?) /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/notifications/publisher.go:122 +0x125 created by github.com/ipfs/go-graphsync/notifications.(*publisher).Startup /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/notifications/publisher.go:46 +0x56 goroutine 79 [select, 6 minutes]: github.com/libp2p/go-libp2p-pubsub.(*PubSub).handleSendingMessages(0x61?, {0x43f01a8, 0xc000bd9640}, {0x4403a50?, 0xc00046d280}, 0xc001309440) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/comm.go:164 +0x305 created by github.com/libp2p/go-libp2p-pubsub.(*PubSub).handleNewPeer /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/comm.go:117 +0x334 goroutine 912448 [select]: github.com/libp2p/go-yamux/v3.(*Session).AcceptStream(0xc0e9c12d80) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:262 +0xb3 github.com/libp2p/go-libp2p/p2p/muxer/yamux.(*conn).AcceptStream(0x13882a5?) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/muxer/yamux/conn.go:43 +0x19 github.com/libp2p/go-libp2p/p2p/net/swarm.(*Conn).start.func1() /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/net/swarm/swarm_conn.go:110 +0xae created by github.com/libp2p/go-libp2p/p2p/net/swarm.(*Conn).start /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/net/swarm/swarm_conn.go:105 +0x56 goroutine 310 [select, 6 minutes]: github.com/libp2p/go-yamux/v3.(*Stream).Read(0xc0012ac0e0, {0xc014002000, 0x1000, 0x1000}) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/stream.go:111 +0x1a6 github.com/libp2p/go-libp2p/p2p/muxer/yamux.(*stream).Read(0x6b2927?, {0xc014002000?, 0x6b2960?, 0xc0004c2680?}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/muxer/yamux/stream.go:17 +0x1e github.com/libp2p/go-libp2p/p2p/net/swarm.(*Stream).Read(0xc011eadd00, {0xc014002000?, 0xc000285ea0?, 0xc000285e30?}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/net/swarm/swarm_stream.go:55 +0x36 bufio.(*Reader).fill(0xc001309500) /usr/local/go/src/bufio/bufio.go:106 +0x103 bufio.(*Reader).ReadByte(0xc001309500) /usr/local/go/src/bufio/bufio.go:264 +0x2c github.com/multiformats/go-varint.ReadUvarint({0x43d7240, 0xc001309500}) /home/stuart/go/pkg/mod/github.com/multiformats/go-varint@v0.0.6/varint.go:80 +0x74 github.com/libp2p/go-msgio/protoio.(*uvarintReader).ReadMsg(0xc000bd9cc0, {0x43ebb38, 0xc000285ea0}) /home/stuart/go/pkg/mod/github.com/libp2p/go-msgio@v0.2.0/protoio/uvarint_reader.go:70 +0x85 github.com/libp2p/go-libp2p-pubsub.(*PubSub).handleNewStream(0xc00031da00, {0x4403ae0, 0xc011eadd00}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/comm.go:66 +0x30d github.com/libp2p/go-libp2p/p2p/host/basic.(*BasicHost).SetStreamHandler.func1({0xc0013381e0, 0xe}, {0x7f9a04a2e390?, 0xc011eadd00}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/host/basic/basic_host.go:569 +0x76 created by github.com/libp2p/go-libp2p/p2p/host/basic.(*BasicHost).newStreamHandler /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/host/basic/basic_host.go:410 +0x715 goroutine 393 [select, 6 minutes]: github.com/filecoin-project/boost/retrievalmarket/rtvllog.(*RetrievalLog).gcDatabase(0xc011dbd6e0, {0x43f01a8, 0xc006ab2c80}) /home/stuart/boost/retrievalmarket/rtvllog/retrieval_log.go:214 +0x119 created by github.com/filecoin-project/boost/retrievalmarket/rtvllog.(*RetrievalLog).Start /home/stuart/boost/retrievalmarket/rtvllog/retrieval_log.go:35 +0x125 goroutine 4845 [select, 6 minutes]: github.com/filecoin-project/go-statemachine.(*StateMachine).run(0xc02442e870) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/machine.go:53 +0x130 created by github.com/filecoin-project/go-statemachine.(*StateGroup).loadOrCreate /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/group.go:132 +0x570 goroutine 756419 [select, 1 minutes]: github.com/libp2p/go-yamux/v3.(*Stream).Read(0xc0d89df260, {0xc0ba0b3000, 0x1000, 0x1000}) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/stream.go:111 +0x1a6 github.com/libp2p/go-libp2p/p2p/muxer/yamux.(*stream).Read(0xc0ba0e2060?, {0xc0ba0b3000?, 0x6fa3dc?, 0x0?}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/muxer/yamux/stream.go:17 +0x1e github.com/libp2p/go-libp2p/p2p/net/swarm.(*Stream).Read(0xc011b23c00, {0xc0ba0b3000?, 0x13f435e?, 0x3?}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/net/swarm/swarm_stream.go:55 +0x36 github.com/multiformats/go-multistream.(*lazyClientConn).Read(0xc0bac92690?, {0xc0ba0b3000?, 0x1000?, 0x1000?}) /home/stuart/go/pkg/mod/github.com/multiformats/go-multistream@v0.3.3/lazyClient.go:68 +0xa3 github.com/libp2p/go-libp2p/p2p/host/basic.(*streamWrapper).Read(0x681aad?, {0xc0ba0b3000?, 0xc0b942f810?, 0x7f9a5c6eab38?}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/host/basic/basic_host.go:1079 +0x27 bufio.(*Reader).fill(0xc025727200) /usr/local/go/src/bufio/bufio.go:106 +0x103 bufio.(*Reader).ReadByte(0xc025727200) /usr/local/go/src/bufio/bufio.go:264 +0x2c github.com/multiformats/go-varint.ReadUvarint({0x43d7240, 0xc025727200}) /home/stuart/go/pkg/mod/github.com/multiformats/go-varint@v0.0.6/varint.go:80 +0x74 github.com/libp2p/go-msgio/protoio.(*uvarintReader).ReadMsg(0xc010ead2c0, {0x43ebb38, 0xc0b942f810}) /home/stuart/go/pkg/mod/github.com/libp2p/go-msgio@v0.2.0/protoio/uvarint_reader.go:70 +0x85 github.com/libp2p/go-libp2p-pubsub.(*PubSub).handlePeerEOF(0xc00031da00, {0xc0df5ff100?, 0xc02524d8c0?}, {0x4403a50, 0xc0bacbe020}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/comm.go:139 +0x144 created by github.com/libp2p/go-libp2p-pubsub.(*PubSub).handleNewPeer /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/comm.go:118 +0x3db goroutine 202871 [sync.Cond.Wait, 5 minutes]: sync.runtime_notifyListWait(0xc021a5d4d0, 0x0) /usr/local/go/src/runtime/sema.go:513 +0x13d sync.(*Cond).Wait(0xc019864af0?) /usr/local/go/src/sync/cond.go:56 +0x8c github.com/ipfs/go-graphsync/notifications.(*publisher).dequeue(0xc021a5d500) /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/notifications/publisher.go:226 +0x6e github.com/ipfs/go-graphsync/notifications.(*publisher).start(0xc017f11880?) /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/notifications/publisher.go:122 +0x125 created by github.com/ipfs/go-graphsync/notifications.(*publisher).Startup /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/notifications/publisher.go:46 +0x56 goroutine 1696 [select, 6 minutes]: github.com/filecoin-project/go-statemachine.(*StateMachine).run(0xc019790910) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/machine.go:53 +0x130 created by github.com/filecoin-project/go-statemachine.(*StateGroup).loadOrCreate /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/group.go:132 +0x570 goroutine 1442 [select, 6 minutes]: github.com/filecoin-project/go-statemachine.(*StateMachine).run(0xc013f058b0) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/machine.go:53 +0x130 created by github.com/filecoin-project/go-statemachine.(*StateGroup).loadOrCreate /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/group.go:132 +0x570 goroutine 68069 [IO wait]: internal/poll.runtime_pollWait(0x7f99ef0f1c48, 0x72) /usr/local/go/src/runtime/netpoll.go:302 +0x89 internal/poll.(*pollDesc).wait(0xc019470100?, 0xc01684a000?, 0x0) /usr/local/go/src/internal/poll/fd_poll_runtime.go:83 +0x32 internal/poll.(*pollDesc).waitRead(...) /usr/local/go/src/internal/poll/fd_poll_runtime.go:88 internal/poll.(*FD).Read(0xc019470100, {0xc01684a000, 0x1000, 0x1000}) /usr/local/go/src/internal/poll/fd_unix.go:167 +0x25a net.(*netFD).Read(0xc019470100, {0xc01684a000?, 0x7511e6?, 0xc08be63958?}) /usr/local/go/src/net/fd_posix.go:55 +0x29 net.(*conn).Read(0xc026f18688, {0xc01684a000?, 0x0?, 0x0?}) /usr/local/go/src/net/net.go:183 +0x45 net/http.(*connReader).Read(0xc0224bec00, {0xc01684a000, 0x1000, 0x1000}) /usr/local/go/src/net/http/server.go:780 +0x16d bufio.(*Reader).fill(0xc01488e360) /usr/local/go/src/bufio/bufio.go:106 +0x103 bufio.(*Reader).Peek(0xc01488e360, 0x4) /usr/local/go/src/bufio/bufio.go:144 +0x5d net/http.(*conn).readRequest(0xc018012f00, {0x43f01a8, 0xc010561680}) /usr/local/go/src/net/http/server.go:985 +0x1f8 net/http.(*conn).serve(0xc018012f00, {0x43f0250, 0xc0247d0180}) /usr/local/go/src/net/http/server.go:1891 +0x32b created by net/http.(*Server).Serve /usr/local/go/src/net/http/server.go:3071 +0x4db goroutine 8614 [select, 6 minutes]: github.com/filecoin-project/go-statemachine.(*StateMachine).run(0xc01e4404b0) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/machine.go:53 +0x130 created by github.com/filecoin-project/go-statemachine.(*StateGroup).loadOrCreate /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/group.go:132 +0x570 goroutine 113975 [sync.Cond.Wait, 5 minutes]: sync.runtime_notifyListWait(0xc011669a10, 0x0) /usr/local/go/src/runtime/sema.go:513 +0x13d sync.(*Cond).Wait(0x0?) /usr/local/go/src/sync/cond.go:56 +0x8c github.com/ipfs/go-graphsync/notifications.(*publisher).dequeue(0xc011669a40) /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/notifications/publisher.go:226 +0x6e github.com/ipfs/go-graphsync/notifications.(*publisher).start(0x10?) /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/notifications/publisher.go:122 +0x125 created by github.com/ipfs/go-graphsync/notifications.(*publisher).Startup /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/notifications/publisher.go:46 +0x56 goroutine 922157 [select]: github.com/libp2p/go-yamux/v3.(*Stream).Read(0xc01e91e460, {0xc01aad3000, 0x1000, 0x1000}) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/stream.go:111 +0x1a6 github.com/libp2p/go-libp2p/p2p/muxer/yamux.(*stream).Read(0xc01d054df0?, {0xc01aad3000?, 0x6fa3dc?, 0x0?}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/muxer/yamux/stream.go:17 +0x1e github.com/libp2p/go-libp2p/p2p/net/swarm.(*Stream).Read(0xc0d1e49d00, {0xc01aad3000?, 0x155?, 0x140?}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/net/swarm/swarm_stream.go:55 +0x36 github.com/multiformats/go-multistream.(*lazyClientConn).Read(0xc0117cce00?, {0xc01aad3000?, 0x1000?, 0x1000?}) /home/stuart/go/pkg/mod/github.com/multiformats/go-multistream@v0.3.3/lazyClient.go:68 +0xa3 github.com/libp2p/go-libp2p/p2p/host/basic.(*streamWrapper).Read(0x681aad?, {0xc01aad3000?, 0xc0117cce70?, 0x7f9a5c6ea1d8?}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/host/basic/basic_host.go:1079 +0x27 bufio.(*Reader).fill(0xc01dd18960) /usr/local/go/src/bufio/bufio.go:106 +0x103 bufio.(*Reader).ReadByte(0xc01dd18960) /usr/local/go/src/bufio/bufio.go:264 +0x2c github.com/multiformats/go-varint.ReadUvarint({0x43d7240, 0xc01dd18960}) /home/stuart/go/pkg/mod/github.com/multiformats/go-varint@v0.0.6/varint.go:80 +0x74 github.com/libp2p/go-msgio/protoio.(*uvarintReader).ReadMsg(0xc0155725c0, {0x43ebb38, 0xc0117cce70}) /home/stuart/go/pkg/mod/github.com/libp2p/go-msgio@v0.2.0/protoio/uvarint_reader.go:70 +0x85 github.com/libp2p/go-libp2p-pubsub.(*PubSub).handlePeerEOF(0xc00031da00, {0xc026f99f78?, 0xc026f99fd0?}, {0x4403a50, 0xc0260e2200}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/comm.go:139 +0x144 created by github.com/libp2p/go-libp2p-pubsub.(*PubSub).handleNewPeer /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/comm.go:118 +0x3db goroutine 249858 [select, 4 minutes]: github.com/libp2p/go-yamux/v3.(*Stream).Read(0xc01ec20c40, {0xc01d386000, 0x1000, 0x1000}) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/stream.go:111 +0x1a6 github.com/libp2p/go-libp2p/p2p/muxer/yamux.(*stream).Read(0xc011eb45d0?, {0xc01d386000?, 0x6fa3dc?, 0x0?}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/muxer/yamux/stream.go:17 +0x1e github.com/libp2p/go-libp2p/p2p/net/swarm.(*Stream).Read(0xc01d01f400, {0xc01d386000?, 0xc028c3d559?, 0x3?}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/net/swarm/swarm_stream.go:55 +0x36 github.com/multiformats/go-multistream.(*lazyClientConn).Read(0xc00043db20?, {0xc01d386000?, 0x1000?, 0x1000?}) /home/stuart/go/pkg/mod/github.com/multiformats/go-multistream@v0.3.3/lazyClient.go:68 +0xa3 github.com/libp2p/go-libp2p/p2p/host/basic.(*streamWrapper).Read(0x681aad?, {0xc01d386000?, 0xc00e28a8c0?, 0x7f9a5c6ea688?}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/host/basic/basic_host.go:1079 +0x27 bufio.(*Reader).fill(0xc016e1a3c0) /usr/local/go/src/bufio/bufio.go:106 +0x103 bufio.(*Reader).ReadByte(0xc016e1a3c0) /usr/local/go/src/bufio/bufio.go:264 +0x2c github.com/multiformats/go-varint.ReadUvarint({0x43d7240, 0xc016e1a3c0}) /home/stuart/go/pkg/mod/github.com/multiformats/go-varint@v0.0.6/varint.go:80 +0x74 github.com/libp2p/go-msgio/protoio.(*uvarintReader).ReadMsg(0xc010cdff40, {0x43ebb38, 0xc00e28a8c0}) /home/stuart/go/pkg/mod/github.com/libp2p/go-msgio@v0.2.0/protoio/uvarint_reader.go:70 +0x85 github.com/libp2p/go-libp2p-pubsub.(*PubSub).handlePeerEOF(0xc00031da00, {0xc028c3d778?, 0xc028c3d7d0?}, {0x4403a50, 0xc01c5e9c40}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/comm.go:139 +0x144 created by github.com/libp2p/go-libp2p-pubsub.(*PubSub).handleNewPeer /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/comm.go:118 +0x3db goroutine 6729 [select, 6 minutes]: github.com/filecoin-project/go-statemachine.(*StateMachine).run(0xc026dd3270) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/machine.go:53 +0x130 created by github.com/filecoin-project/go-statemachine.(*StateGroup).loadOrCreate /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/group.go:132 +0x570 goroutine 431 [select, 6 minutes]: github.com/filecoin-project/go-statemachine.(*StateMachine).run(0xc014020410) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/machine.go:53 +0x130 created by github.com/filecoin-project/go-statemachine.(*StateGroup).loadOrCreate /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/group.go:132 +0x570 goroutine 67926 [select]: github.com/libp2p/go-yamux/v3.(*Session).sendLoop(0xc015c99c20) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:559 +0x5c5 github.com/libp2p/go-yamux/v3.(*Session).send(0xc0157f7100?) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:488 +0x1e created by github.com/libp2p/go-yamux/v3.newSession /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:159 +0x6aa goroutine 613 [sync.Cond.Wait, 6 minutes]: sync.runtime_notifyListWait(0xc01a31f410, 0x0) /usr/local/go/src/runtime/sema.go:513 +0x13d sync.(*Cond).Wait(0xc01521aaf0?) /usr/local/go/src/sync/cond.go:56 +0x8c github.com/ipfs/go-graphsync/notifications.(*publisher).dequeue(0xc01a31f440) /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/notifications/publisher.go:226 +0x6e github.com/ipfs/go-graphsync/notifications.(*publisher).start(0xc013f34d00?) /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/notifications/publisher.go:122 +0x125 created by github.com/ipfs/go-graphsync/notifications.(*publisher).Startup /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/notifications/publisher.go:46 +0x56 goroutine 467575 [select, 3 minutes]: github.com/libp2p/go-yamux/v3.(*Session).AcceptStream(0xc0ae2ffe60) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:262 +0xb3 github.com/libp2p/go-libp2p/p2p/muxer/yamux.(*conn).AcceptStream(0x13882a5?) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/muxer/yamux/conn.go:43 +0x19 github.com/libp2p/go-libp2p/p2p/net/swarm.(*Conn).start.func1() /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/net/swarm/swarm_conn.go:110 +0xae created by github.com/libp2p/go-libp2p/p2p/net/swarm.(*Conn).start /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/net/swarm/swarm_conn.go:105 +0x56 goroutine 585 [select, 6 minutes]: github.com/filecoin-project/go-statemachine.(*StateMachine).run(0xc013f040f0) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/machine.go:53 +0x130 created by github.com/filecoin-project/go-statemachine.(*StateGroup).loadOrCreate /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/group.go:132 +0x570 goroutine 636 [select, 6 minutes]: github.com/filecoin-project/go-statemachine.(*StateMachine).run(0xc014382050) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/machine.go:53 +0x130 created by github.com/filecoin-project/go-statemachine.(*StateGroup).loadOrCreate /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/group.go:132 +0x570 goroutine 331532 [select, 4 minutes]: github.com/libp2p/go-libp2p-pubsub.(*PubSub).handleSendingMessages(0xc0227a02d0?, {0x43f01a8, 0xc000bd9640}, {0x4403a50?, 0xc06e0c9500}, 0xc02050bb60) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/comm.go:164 +0x305 created by github.com/libp2p/go-libp2p-pubsub.(*PubSub).handleNewPeer /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/comm.go:117 +0x334 goroutine 911385 [select]: github.com/libp2p/go-libp2p/p2p/protocol/identify.(*peerHandler).loop(0xc0e88d5ea0, {0x43f01a8, 0xc0d2b969c0}, 0xc016ecf980?) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/protocol/identify/peer_loop.go:89 +0x105 created by github.com/libp2p/go-libp2p/p2p/protocol/identify.(*peerHandler).start /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/protocol/identify/peer_loop.go:71 +0xf0 goroutine 413 [select, 6 minutes]: github.com/libp2p/go-libp2p-pubsub.(*PubSub).handleSendingMessages(0x10000c000aeae40?, {0x43f01a8, 0xc000bd9640}, {0x4403a50?, 0xc019a70560}, 0xc000b336e0) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/comm.go:164 +0x305 created by github.com/libp2p/go-libp2p-pubsub.(*PubSub).handleNewPeer /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/comm.go:117 +0x334 goroutine 122113 [select, 5 minutes]: github.com/ipfs/go-graphsync/messagequeue.(*MessageQueue).runQueue(0xc0b0fc03c0) /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/messagequeue/messagequeue.go:157 +0x12c created by github.com/ipfs/go-graphsync/messagequeue.(*MessageQueue).Startup /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/messagequeue/messagequeue.go:142 +0x56 goroutine 689270 [select, 1 minutes]: github.com/libp2p/go-libp2p-pubsub.(*PubSub).handleSendingMessages(0xc08cfa9620?, {0x43f01a8, 0xc000bd9640}, {0x4403a50?, 0xc029894640}, 0xc023af4ae0) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/comm.go:164 +0x305 created by github.com/libp2p/go-libp2p-pubsub.(*PubSub).handleNewPeer /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/comm.go:117 +0x334 goroutine 10424 [select, 6 minutes]: github.com/filecoin-project/go-statemachine.(*StateMachine).run(0xc0248b1680) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/machine.go:53 +0x130 created by github.com/filecoin-project/go-statemachine.(*StateGroup).loadOrCreate /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/group.go:132 +0x570 goroutine 554 [sync.Cond.Wait, 6 minutes]: sync.runtime_notifyListWait(0xc011e54510, 0x0) /usr/local/go/src/runtime/sema.go:513 +0x13d sync.(*Cond).Wait(0xc0013ee640?) /usr/local/go/src/sync/cond.go:56 +0x8c github.com/ipfs/go-graphsync/notifications.(*publisher).dequeue(0xc011e54640) /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/notifications/publisher.go:226 +0x6e github.com/ipfs/go-graphsync/notifications.(*publisher).start(0xc000bd8a80?) /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/notifications/publisher.go:122 +0x125 created by github.com/ipfs/go-graphsync/notifications.(*publisher).Startup /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/notifications/publisher.go:46 +0x56 goroutine 7469 [select, 6 minutes]: github.com/filecoin-project/go-statemachine.(*StateMachine).run(0xc000bdeb90) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/machine.go:53 +0x130 created by github.com/filecoin-project/go-statemachine.(*StateGroup).loadOrCreate /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/group.go:132 +0x570 goroutine 501 [select, 6 minutes]: github.com/ipfs/go-graphsync/messagequeue.(*MessageQueue).runQueue(0xc000aeaa80) /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/messagequeue/messagequeue.go:157 +0x12c created by github.com/ipfs/go-graphsync/messagequeue.(*MessageQueue).Startup /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/messagequeue/messagequeue.go:142 +0x56 goroutine 932791 [select]: github.com/ipfs/go-graphsync/messagequeue.(*MessageQueue).runQueue(0xc0fdc40840) /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/messagequeue/messagequeue.go:157 +0x12c created by github.com/ipfs/go-graphsync/messagequeue.(*MessageQueue).Startup /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/messagequeue/messagequeue.go:142 +0x56 goroutine 546 [IO wait]: internal/poll.runtime_pollWait(0x7f9a24702e58, 0x72) /usr/local/go/src/runtime/netpoll.go:302 +0x89 internal/poll.(*pollDesc).wait(0xc014124400?, 0xc0002df000?, 0x0) /usr/local/go/src/internal/poll/fd_poll_runtime.go:83 +0x32 internal/poll.(*pollDesc).waitRead(...) /usr/local/go/src/internal/poll/fd_poll_runtime.go:88 internal/poll.(*FD).Read(0xc014124400, {0xc0002df000, 0x1000, 0x1000}) /usr/local/go/src/internal/poll/fd_unix.go:167 +0x25a net.(*netFD).Read(0xc014124400, {0xc0002df000?, 0xc01a8ad2e0?, 0x1c?}) /usr/local/go/src/net/fd_posix.go:55 +0x29 net.(*conn).Read(0xc01427e010, {0xc0002df000?, 0xc0122b1418?, 0xc00019fd48?}) /usr/local/go/src/net/net.go:183 +0x45 bufio.(*Reader).Read(0xc000a4d2c0, {0xc00025ebd0, 0x2, 0x6fa62e?}) /usr/local/go/src/bufio/bufio.go:236 +0x1b4 io.ReadAtLeast({0x43d7260, 0xc000a4d2c0}, {0xc00025ebd0, 0x2, 0x2}, 0x2) /usr/local/go/src/io/io.go:331 +0x9a io.ReadFull(...) /usr/local/go/src/io/io.go:350 github.com/libp2p/go-libp2p/p2p/security/noise.(*secureSession).readNextInsecureMsgLen(0xc00025eb40) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/security/noise/rw.go:133 +0x3b github.com/libp2p/go-libp2p/p2p/security/noise.(*secureSession).Read(0xc00025eb40, {0xc0075b6010, 0xc, 0xc}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/security/noise/rw.go:52 +0x22f io.ReadAtLeast({0x7f9a049c0680, 0xc00025eb40}, {0xc0075b6010, 0xc, 0xc}, 0xc) /usr/local/go/src/io/io.go:331 +0x9a io.ReadFull(...) /usr/local/go/src/io/io.go:350 github.com/libp2p/go-yamux/v3.(*Session).recvLoop(0xc0003cf440) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:644 +0xf7 github.com/libp2p/go-yamux/v3.(*Session).recv(0x0?) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:616 +0x1e created by github.com/libp2p/go-yamux/v3.newSession /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:158 +0x66a goroutine 200073 [select, 5 minutes]: github.com/libp2p/go-yamux/v3.(*Session).AcceptStream(0xc0143fec60) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:262 +0xb3 github.com/libp2p/go-libp2p/p2p/muxer/yamux.(*conn).AcceptStream(0x13882a5?) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/muxer/yamux/conn.go:43 +0x19 github.com/libp2p/go-libp2p/p2p/net/swarm.(*Conn).start.func1() /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/net/swarm/swarm_conn.go:110 +0xae created by github.com/libp2p/go-libp2p/p2p/net/swarm.(*Conn).start /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/net/swarm/swarm_conn.go:105 +0x56 goroutine 415 [select, 6 minutes]: github.com/libp2p/go-yamux/v3.(*Stream).Read(0xc0174160e0, {0xc010441000, 0x1000, 0x1000}) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/stream.go:111 +0x1a6 github.com/libp2p/go-libp2p/p2p/muxer/yamux.(*stream).Read(0x6b2927?, {0xc010441000?, 0x6b2960?, 0xc0004c2680?}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/muxer/yamux/stream.go:17 +0x1e github.com/libp2p/go-libp2p/p2p/net/swarm.(*Stream).Read(0xc014124580, {0xc010441000?, 0xc0187b40e0?, 0xc0187b4070?}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/net/swarm/swarm_stream.go:55 +0x36 bufio.(*Reader).fill(0xc011ed88a0) /usr/local/go/src/bufio/bufio.go:106 +0x103 bufio.(*Reader).ReadByte(0xc011ed88a0) /usr/local/go/src/bufio/bufio.go:264 +0x2c github.com/multiformats/go-varint.ReadUvarint({0x43d7240, 0xc011ed88a0}) /home/stuart/go/pkg/mod/github.com/multiformats/go-varint@v0.0.6/varint.go:80 +0x74 github.com/libp2p/go-msgio/protoio.(*uvarintReader).ReadMsg(0xc010623200, {0x43ebb38, 0xc0187b40e0}) /home/stuart/go/pkg/mod/github.com/libp2p/go-msgio@v0.2.0/protoio/uvarint_reader.go:70 +0x85 github.com/libp2p/go-libp2p-pubsub.(*PubSub).handleNewStream(0xc00031da00, {0x4403ae0, 0xc014124580}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/comm.go:66 +0x30d github.com/libp2p/go-libp2p/p2p/host/basic.(*BasicHost).SetStreamHandler.func1({0xc00782e480, 0xe}, {0x7f9a04a2e390?, 0xc014124580}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/host/basic/basic_host.go:569 +0x76 created by github.com/libp2p/go-libp2p/p2p/host/basic.(*BasicHost).newStreamHandler /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/host/basic/basic_host.go:410 +0x715 goroutine 11754 [select, 6 minutes]: github.com/filecoin-project/go-statemachine.(*StateMachine).run(0xc01ce67d10) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/machine.go:53 +0x130 created by github.com/filecoin-project/go-statemachine.(*StateGroup).loadOrCreate /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/group.go:132 +0x570 goroutine 494214 [sync.Cond.Wait, 3 minutes]: sync.runtime_notifyListWait(0xc010d8c510, 0x0) /usr/local/go/src/runtime/sema.go:513 +0x13d sync.(*Cond).Wait(0xc015d3db80?) /usr/local/go/src/sync/cond.go:56 +0x8c github.com/ipfs/go-graphsync/notifications.(*publisher).dequeue(0xc010d8c540) /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/notifications/publisher.go:226 +0x6e github.com/ipfs/go-graphsync/notifications.(*publisher).start(0xc00025e480?) /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/notifications/publisher.go:122 +0x125 created by github.com/ipfs/go-graphsync/notifications.(*publisher).Startup /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/notifications/publisher.go:46 +0x56 goroutine 4841 [select, 6 minutes]: github.com/filecoin-project/go-statemachine.(*StateMachine).run(0xc02442e730) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/machine.go:53 +0x130 created by github.com/filecoin-project/go-statemachine.(*StateGroup).loadOrCreate /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/group.go:132 +0x570 goroutine 480 [select, 6 minutes]: github.com/filecoin-project/go-statemachine.(*StateMachine).run(0xc000daa050) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/machine.go:53 +0x130 created by github.com/filecoin-project/go-statemachine.(*StateGroup).loadOrCreate /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/group.go:132 +0x570 goroutine 410 [select, 6 minutes]: github.com/libp2p/go-yamux/v3.(*Stream).Read(0xc011dc2460, {0xc00130b000, 0x1000, 0x1000}) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/stream.go:111 +0x1a6 github.com/libp2p/go-libp2p/p2p/muxer/yamux.(*stream).Read(0xc012794000?, {0xc00130b000?, 0x6fa3dc?, 0x0?}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/muxer/yamux/stream.go:17 +0x1e github.com/libp2p/go-libp2p/p2p/net/swarm.(*Stream).Read(0xc000a76200, {0xc00130b000?, 0x1928?, 0xc0009bb5d8?}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/net/swarm/swarm_stream.go:55 +0x36 github.com/multiformats/go-multistream.(*lazyClientConn).Read(0xc0001d7ea0?, {0xc00130b000?, 0x1000?, 0x1000?}) /home/stuart/go/pkg/mod/github.com/multiformats/go-multistream@v0.3.3/lazyClient.go:68 +0xa3 github.com/libp2p/go-libp2p/p2p/host/basic.(*streamWrapper).Read(0xc0009bb610?, {0xc00130b000?, 0xc000100000?, 0x7f9a5c6e9878?}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/host/basic/basic_host.go:1079 +0x27 bufio.(*Reader).fill(0xc000b32ba0) /usr/local/go/src/bufio/bufio.go:106 +0x103 bufio.(*Reader).ReadByte(0xc000b32ba0) /usr/local/go/src/bufio/bufio.go:264 +0x2c github.com/multiformats/go-varint.ReadUvarint({0x43d7240, 0xc000b32ba0}) /home/stuart/go/pkg/mod/github.com/multiformats/go-varint@v0.0.6/varint.go:80 +0x74 github.com/libp2p/go-msgio/protoio.(*uvarintReader).ReadMsg(0xc014b60200, {0x43ebb38, 0xc000100000}) /home/stuart/go/pkg/mod/github.com/libp2p/go-msgio@v0.2.0/protoio/uvarint_reader.go:70 +0x85 github.com/libp2p/go-libp2p-pubsub.(*PubSub).handlePeerEOF(0xc00031da00, {0xc000a4a630?, 0x16e0060?}, {0x4403a50, 0xc019a628c0}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/comm.go:139 +0x144 created by github.com/libp2p/go-libp2p-pubsub.(*PubSub).handleNewPeer /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/comm.go:118 +0x3db goroutine 550 [select, 6 minutes]: github.com/ipfs/go-graphsync/messagequeue.(*MessageQueue).runQueue(0xc000aeab40) /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/messagequeue/messagequeue.go:157 +0x12c created by github.com/ipfs/go-graphsync/messagequeue.(*MessageQueue).Startup /home/stuart/go/pkg/mod/github.com/ipfs/go-graphsync@v0.13.1/messagequeue/messagequeue.go:142 +0x56 goroutine 519 [select, 6 minutes]: github.com/libp2p/go-libp2p/p2p/protocol/identify.(*peerHandler).loop(0xc0002db0e0, {0x43f01a8, 0xc0135dfcc0}, 0xc000a0e400?) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/protocol/identify/peer_loop.go:89 +0x105 created by github.com/libp2p/go-libp2p/p2p/protocol/identify.(*peerHandler).start /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/protocol/identify/peer_loop.go:71 +0xf0 goroutine 498 [IO wait]: internal/poll.runtime_pollWait(0x7f9a24702b88, 0x72) /usr/local/go/src/runtime/netpoll.go:302 +0x89 internal/poll.(*pollDesc).wait(0xc000a76280?, 0xc011eed000?, 0x0) /usr/local/go/src/internal/poll/fd_poll_runtime.go:83 +0x32 internal/poll.(*pollDesc).waitRead(...) /usr/local/go/src/internal/poll/fd_poll_runtime.go:88 internal/poll.(*FD).Read(0xc000a76280, {0xc011eed000, 0x1000, 0x1000}) /usr/local/go/src/internal/poll/fd_unix.go:167 +0x25a net.(*netFD).Read(0xc000a76280, {0xc011eed000?, 0xc02ef14460?, 0x1c?}) /usr/local/go/src/net/fd_posix.go:55 +0x29 net.(*conn).Read(0xc000e1a9c8, {0xc011eed000?, 0x74?, 0xc014015d48?}) /usr/local/go/src/net/net.go:183 +0x45 bufio.(*Reader).Read(0xc000a46240, {0xc000aea090, 0x2, 0x6fa62e?}) /usr/local/go/src/bufio/bufio.go:236 +0x1b4 io.ReadAtLeast({0x43d7260, 0xc000a46240}, {0xc000aea090, 0x2, 0x2}, 0x2) /usr/local/go/src/io/io.go:331 +0x9a io.ReadFull(...) /usr/local/go/src/io/io.go:350 github.com/libp2p/go-libp2p/p2p/security/noise.(*secureSession).readNextInsecureMsgLen(0xc000aea000) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/security/noise/rw.go:133 +0x3b github.com/libp2p/go-libp2p/p2p/security/noise.(*secureSession).Read(0xc000aea000, {0xc0141ae000, 0xc, 0xc}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/security/noise/rw.go:52 +0x22f io.ReadAtLeast({0x7f9a049c0680, 0xc000aea000}, {0xc0141ae000, 0xc, 0xc}, 0xc) /usr/local/go/src/io/io.go:331 +0x9a io.ReadFull(...) /usr/local/go/src/io/io.go:350 github.com/libp2p/go-yamux/v3.(*Session).recvLoop(0xc0003e5e60) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:644 +0xf7 github.com/libp2p/go-yamux/v3.(*Session).recv(0x8?) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:616 +0x1e created by github.com/libp2p/go-yamux/v3.newSession /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:158 +0x66a goroutine 499 [select]: github.com/libp2p/go-yamux/v3.(*Session).sendLoop(0xc0003e5e60) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:559 +0x5c5 github.com/libp2p/go-yamux/v3.(*Session).send(0xc006aba7b8?) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:488 +0x1e created by github.com/libp2p/go-yamux/v3.newSession /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:159 +0x6aa goroutine 465 [select, 6 minutes]: github.com/filecoin-project/go-statemachine.(*StateMachine).run(0xc000d22460) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/machine.go:53 +0x130 created by github.com/filecoin-project/go-statemachine.(*StateGroup).loadOrCreate /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/group.go:132 +0x570 goroutine 547 [select]: github.com/libp2p/go-yamux/v3.(*Session).sendLoop(0xc0003cf440) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:559 +0x5c5 github.com/libp2p/go-yamux/v3.(*Session).send(0xc006ab77b8?) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:488 +0x1e created by github.com/libp2p/go-yamux/v3.newSession /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:159 +0x6aa goroutine 504 [select, 6 minutes]: github.com/libp2p/go-yamux/v3.(*Session).AcceptStream(0xc0003e5e60) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:262 +0xb3 github.com/libp2p/go-libp2p/p2p/muxer/yamux.(*conn).AcceptStream(0x13882a5?) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/muxer/yamux/conn.go:43 +0x19 github.com/libp2p/go-libp2p/p2p/net/swarm.(*Conn).start.func1() /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/net/swarm/swarm_conn.go:110 +0xae created by github.com/libp2p/go-libp2p/p2p/net/swarm.(*Conn).start /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/net/swarm/swarm_conn.go:105 +0x56 goroutine 217 [select, 6 minutes]: github.com/libp2p/go-libp2p/p2p/protocol/identify.(*peerHandler).loop(0xc0071c33b0, {0x43f01a8, 0xc014dc4900}, 0xc006a4a700?) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/protocol/identify/peer_loop.go:89 +0x105 created by github.com/libp2p/go-libp2p/p2p/protocol/identify.(*peerHandler).start /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/protocol/identify/peer_loop.go:71 +0xf0 goroutine 834791 [select]: github.com/libp2p/go-libp2p-pubsub.(*PubSub).handleSendingMessages(0xc00031da00?, {0x43f01a8, 0xc000bd9640}, {0x4403a50?, 0xc0a1d0f280}, 0xc02ef49800) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/comm.go:164 +0x305 created by github.com/libp2p/go-libp2p-pubsub.(*PubSub).handleNewPeer /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/comm.go:117 +0x334 goroutine 447 [select, 6 minutes]: github.com/libp2p/go-libp2p-pubsub.(*PubSub).handleSendingMessages(0xc0141b4480?, {0x43f01a8, 0xc000bd9640}, {0x4403a50?, 0xc0141bc380}, 0xc0089f8660) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/comm.go:164 +0x305 created by github.com/libp2p/go-libp2p-pubsub.(*PubSub).handleNewPeer /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/comm.go:117 +0x334 goroutine 448 [select, 6 minutes]: github.com/libp2p/go-yamux/v3.(*Stream).Read(0xc0012ac000, {0xc000239000, 0x1000, 0x1000}) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/stream.go:111 +0x1a6 github.com/libp2p/go-libp2p/p2p/muxer/yamux.(*stream).Read(0xc000a4a770?, {0xc000239000?, 0x6fa3dc?, 0x0?}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/muxer/yamux/stream.go:17 +0x1e github.com/libp2p/go-libp2p/p2p/net/swarm.(*Stream).Read(0xc000a0ed80, {0xc000239000?, 0x30000000135c459?, 0x43dc9e0?}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/net/swarm/swarm_stream.go:55 +0x36 github.com/multiformats/go-multistream.(*lazyClientConn).Read(0xc0141b2070?, {0xc000239000?, 0x1000?, 0x1000?}) /home/stuart/go/pkg/mod/github.com/multiformats/go-multistream@v0.3.3/lazyClient.go:68 +0xa3 github.com/libp2p/go-libp2p/p2p/host/basic.(*streamWrapper).Read(0x681aad?, {0xc000239000?, 0xc014eb8310?, 0x7f9a5c6e85b8?}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/host/basic/basic_host.go:1079 +0x27 bufio.(*Reader).fill(0xc013f42540) /usr/local/go/src/bufio/bufio.go:106 +0x103 bufio.(*Reader).ReadByte(0xc013f42540) /usr/local/go/src/bufio/bufio.go:264 +0x2c github.com/multiformats/go-varint.ReadUvarint({0x43d7240, 0xc013f42540}) /home/stuart/go/pkg/mod/github.com/multiformats/go-varint@v0.0.6/varint.go:80 +0x74 github.com/libp2p/go-msgio/protoio.(*uvarintReader).ReadMsg(0xc01107d1c0, {0x43ebb38, 0xc014eb8310}) /home/stuart/go/pkg/mod/github.com/libp2p/go-msgio@v0.2.0/protoio/uvarint_reader.go:70 +0x85 github.com/libp2p/go-libp2p-pubsub.(*PubSub).handlePeerEOF(0xc00031da00, {0xc0003e5e60?, 0xc006aba7d0?}, {0x4403a50, 0xc0141bc380}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/comm.go:139 +0x144 created by github.com/libp2p/go-libp2p-pubsub.(*PubSub).handleNewPeer /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/comm.go:118 +0x3db goroutine 85924 [select]: github.com/libp2p/go-yamux/v3.(*Session).sendLoop(0xc016ecafc0) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:559 +0x5c5 github.com/libp2p/go-yamux/v3.(*Session).send(0xc011108d40?) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:488 +0x1e created by github.com/libp2p/go-yamux/v3.newSession /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:159 +0x6aa goroutine 128 [select, 6 minutes]: github.com/libp2p/go-yamux/v3.(*Stream).Read(0xc00025c540, {0xc013f27000, 0x1000, 0x1000}) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/stream.go:111 +0x1a6 github.com/libp2p/go-libp2p/p2p/muxer/yamux.(*stream).Read(0x6b2927?, {0xc013f27000?, 0x70?, 0x1?}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/muxer/yamux/stream.go:17 +0x1e github.com/libp2p/go-libp2p/p2p/net/swarm.(*Stream).Read(0xc000a76080, {0xc013f27000?, 0xc014036000?, 0xc0191001c0?}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/net/swarm/swarm_stream.go:55 +0x36 bufio.(*Reader).fill(0xc000a4d380) /usr/local/go/src/bufio/bufio.go:106 +0x103 bufio.(*Reader).ReadByte(0xc000a4d380) /usr/local/go/src/bufio/bufio.go:264 +0x2c github.com/multiformats/go-varint.ReadUvarint({0x43d7240, 0xc000a4d380}) /home/stuart/go/pkg/mod/github.com/multiformats/go-varint@v0.0.6/varint.go:80 +0x74 github.com/libp2p/go-msgio/protoio.(*uvarintReader).ReadMsg(0xc019102600, {0x43ebb38, 0xc014036000}) /home/stuart/go/pkg/mod/github.com/libp2p/go-msgio@v0.2.0/protoio/uvarint_reader.go:70 +0x85 github.com/libp2p/go-libp2p-pubsub.(*PubSub).handleNewStream(0xc00031da00, {0x4403ae0, 0xc000a76080}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/comm.go:66 +0x30d github.com/libp2p/go-libp2p/p2p/host/basic.(*BasicHost).SetStreamHandler.func1({0xc01426e4e0, 0xe}, {0x7f9a04a2e390?, 0xc000a76080}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/host/basic/basic_host.go:569 +0x76 created by github.com/libp2p/go-libp2p/p2p/host/basic.(*BasicHost).newStreamHandler /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/host/basic/basic_host.go:410 +0x715 goroutine 642 [select, 6 minutes]: github.com/filecoin-project/go-statemachine.(*StateMachine).run(0xc0087a4aa0) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/machine.go:53 +0x130 created by github.com/filecoin-project/go-statemachine.(*StateGroup).loadOrCreate /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/group.go:132 +0x570 goroutine 864174 [select]: github.com/libp2p/go-yamux/v3.(*Stream).Read(0xc01c745420, {0xc018c85000, 0x1000, 0x1000}) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/stream.go:111 +0x1a6 github.com/libp2p/go-libp2p/p2p/muxer/yamux.(*stream).Read(0x6b2927?, {0xc018c85000?, 0x6b2960?, 0xc0004c2680?}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/muxer/yamux/stream.go:17 +0x1e github.com/libp2p/go-libp2p/p2p/net/swarm.(*Stream).Read(0xc030a26f80, {0xc018c85000?, 0xc015d12ee0?, 0xc015d12e70?}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/net/swarm/swarm_stream.go:55 +0x36 bufio.(*Reader).fill(0xc02335bc20) /usr/local/go/src/bufio/bufio.go:106 +0x103 bufio.(*Reader).ReadByte(0xc02335bc20) /usr/local/go/src/bufio/bufio.go:264 +0x2c github.com/multiformats/go-varint.ReadUvarint({0x43d7240, 0xc02335bc20}) /home/stuart/go/pkg/mod/github.com/multiformats/go-varint@v0.0.6/varint.go:80 +0x74 github.com/libp2p/go-msgio/protoio.(*uvarintReader).ReadMsg(0xc0203dd580, {0x43ebb38, 0xc015d12ee0}) /home/stuart/go/pkg/mod/github.com/libp2p/go-msgio@v0.2.0/protoio/uvarint_reader.go:70 +0x85 github.com/libp2p/go-libp2p-pubsub.(*PubSub).handleNewStream(0xc00031da00, {0x4403ae0, 0xc030a26f80}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/comm.go:66 +0x30d github.com/libp2p/go-libp2p/p2p/host/basic.(*BasicHost).SetStreamHandler.func1({0xc011f2d5f0, 0xe}, {0x7f9a04a2e390?, 0xc030a26f80}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/host/basic/basic_host.go:569 +0x76 created by github.com/libp2p/go-libp2p/p2p/host/basic.(*BasicHost).newStreamHandler /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/host/basic/basic_host.go:410 +0x715 goroutine 414 [select, 6 minutes]: github.com/libp2p/go-yamux/v3.(*Stream).Read(0xc001194540, {0xc01741d000, 0x1000, 0x1000}) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/stream.go:111 +0x1a6 github.com/libp2p/go-libp2p/p2p/muxer/yamux.(*stream).Read(0xc01311b820?, {0xc01741d000?, 0x6fa3dc?, 0x0?}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/muxer/yamux/stream.go:17 +0x1e github.com/libp2p/go-libp2p/p2p/net/swarm.(*Stream).Read(0xc011eac980, {0xc01741d000?, 0x300000000000059?, 0x43f14e8?}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/net/swarm/swarm_stream.go:55 +0x36 github.com/multiformats/go-multistream.(*lazyClientConn).Read(0xc0187b4000?, {0xc01741d000?, 0x1000?, 0x1000?}) /home/stuart/go/pkg/mod/github.com/multiformats/go-multistream@v0.3.3/lazyClient.go:68 +0xa3 github.com/libp2p/go-libp2p/p2p/host/basic.(*streamWrapper).Read(0x681aad?, {0xc01741d000?, 0xc02041bb20?, 0x7f9a5c6e8f18?}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/host/basic/basic_host.go:1079 +0x27 bufio.(*Reader).fill(0xc0012f2060) /usr/local/go/src/bufio/bufio.go:106 +0x103 bufio.(*Reader).ReadByte(0xc0012f2060) /usr/local/go/src/bufio/bufio.go:264 +0x2c github.com/multiformats/go-varint.ReadUvarint({0x43d7240, 0xc0012f2060}) /home/stuart/go/pkg/mod/github.com/multiformats/go-varint@v0.0.6/varint.go:80 +0x74 github.com/libp2p/go-msgio/protoio.(*uvarintReader).ReadMsg(0xc011f5d140, {0x43ebb38, 0xc02041bb20}) /home/stuart/go/pkg/mod/github.com/libp2p/go-msgio@v0.2.0/protoio/uvarint_reader.go:70 +0x85 github.com/libp2p/go-libp2p-pubsub.(*PubSub).handlePeerEOF(0xc00031da00, {0xc0003cf560?, 0xc000d367d0?}, {0x4403a50, 0xc019a70560}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/comm.go:139 +0x144 created by github.com/libp2p/go-libp2p-pubsub.(*PubSub).handleNewPeer /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/comm.go:118 +0x3db goroutine 582 [select, 6 minutes]: github.com/libp2p/go-libp2p/p2p/protocol/identify.(*peerHandler).loop(0xc00f1912c0, {0x43f01a8, 0xc00e294d40}, 0xc013f2e7b0?) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/protocol/identify/peer_loop.go:89 +0x105 created by github.com/libp2p/go-libp2p/p2p/protocol/identify.(*peerHandler).start /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/protocol/identify/peer_loop.go:71 +0xf0 goroutine 409 [select, 6 minutes]: github.com/libp2p/go-libp2p-pubsub.(*PubSub).handleSendingMessages(0xc000901770?, {0x43f01a8, 0xc000bd9640}, {0x4403a50?, 0xc019a628c0}, 0xc0012f2660) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/comm.go:164 +0x305 created by github.com/libp2p/go-libp2p-pubsub.(*PubSub).handleNewPeer /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/comm.go:117 +0x334 goroutine 553 [select, 6 minutes]: github.com/libp2p/go-yamux/v3.(*Session).AcceptStream(0xc0003cf440) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:262 +0xb3 github.com/libp2p/go-libp2p/p2p/muxer/yamux.(*conn).AcceptStream(0x13882a5?) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/muxer/yamux/conn.go:43 +0x19 github.com/libp2p/go-libp2p/p2p/net/swarm.(*Conn).start.func1() /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/net/swarm/swarm_conn.go:110 +0xae created by github.com/libp2p/go-libp2p/p2p/net/swarm.(*Conn).start /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/net/swarm/swarm_conn.go:105 +0x56 goroutine 494 [select, 6 minutes]: github.com/filecoin-project/go-statemachine.(*StateMachine).run(0xc000a485f0) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/machine.go:53 +0x130 created by github.com/filecoin-project/go-statemachine.(*StateGroup).loadOrCreate /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/group.go:132 +0x570 goroutine 488 [select, 6 minutes]: github.com/filecoin-project/go-statemachine.(*StateMachine).run(0xc000a48410) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/machine.go:53 +0x130 created by github.com/filecoin-project/go-statemachine.(*StateGroup).loadOrCreate /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/group.go:132 +0x570 goroutine 629 [select]: github.com/libp2p/go-yamux/v3.(*Session).sendLoop(0xc0004a6120) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:559 +0x5c5 github.com/libp2p/go-yamux/v3.(*Session).send(0xc001323fb8?) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:488 +0x1e created by github.com/libp2p/go-yamux/v3.newSession /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:159 +0x6aa goroutine 640 [select, 6 minutes]: github.com/filecoin-project/go-statemachine.(*StateMachine).run(0xc014382190) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/machine.go:53 +0x130 created by github.com/filecoin-project/go-statemachine.(*StateGroup).loadOrCreate /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/group.go:132 +0x570 goroutine 638 [select, 6 minutes]: github.com/filecoin-project/go-statemachine.(*StateMachine).run(0xc0143820a0) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/machine.go:53 +0x130 created by github.com/filecoin-project/go-statemachine.(*StateGroup).loadOrCreate /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/group.go:132 +0x570 goroutine 740 [select, 6 minutes]: github.com/filecoin-project/go-statemachine.(*StateMachine).run(0xc013f04500) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/machine.go:53 +0x130 created by github.com/filecoin-project/go-statemachine.(*StateGroup).loadOrCreate /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/group.go:132 +0x570 goroutine 523 [select, 6 minutes]: github.com/filecoin-project/go-statemachine.(*StateMachine).run(0xc0015885a0) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/machine.go:53 +0x130 created by github.com/filecoin-project/go-statemachine.(*StateGroup).loadOrCreate /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/group.go:132 +0x570 goroutine 569 [select, 6 minutes]: github.com/libp2p/go-yamux/v3.(*Stream).Read(0xc0012ac620, {0xc0122ae000, 0x1000, 0x1000}) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/stream.go:111 +0x1a6 github.com/libp2p/go-libp2p/p2p/muxer/yamux.(*stream).Read(0x6b2927?, {0xc0122ae000?, 0x6b2960?, 0xc0004c2680?}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/muxer/yamux/stream.go:17 +0x1e github.com/libp2p/go-libp2p/p2p/net/swarm.(*Stream).Read(0xc011eac000, {0xc0122ae000?, 0xc019a20b60?, 0xc019a20af0?}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/net/swarm/swarm_stream.go:55 +0x36 bufio.(*Reader).fill(0xc0002cfec0) /usr/local/go/src/bufio/bufio.go:106 +0x103 bufio.(*Reader).ReadByte(0xc0002cfec0) /usr/local/go/src/bufio/bufio.go:264 +0x2c github.com/multiformats/go-varint.ReadUvarint({0x43d7240, 0xc0002cfec0}) /home/stuart/go/pkg/mod/github.com/multiformats/go-varint@v0.0.6/varint.go:80 +0x74 github.com/libp2p/go-msgio/protoio.(*uvarintReader).ReadMsg(0xc0141d8040, {0x43ebb38, 0xc019a20b60}) /home/stuart/go/pkg/mod/github.com/libp2p/go-msgio@v0.2.0/protoio/uvarint_reader.go:70 +0x85 github.com/libp2p/go-libp2p-pubsub.(*PubSub).handleNewStream(0xc00031da00, {0x4403ae0, 0xc011eac000}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/comm.go:66 +0x30d github.com/libp2p/go-libp2p/p2p/host/basic.(*BasicHost).SetStreamHandler.func1({0xc0111c4100, 0xe}, {0x7f9a04a2e390?, 0xc011eac000}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/host/basic/basic_host.go:569 +0x76 created by github.com/libp2p/go-libp2p/p2p/host/basic.(*BasicHost).newStreamHandler /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/host/basic/basic_host.go:410 +0x715 goroutine 784577 [select, 1 minutes]: github.com/libp2p/go-yamux/v3.(*Stream).Read(0xc0e08ff340, {0xc0edba6000, 0x1000, 0x1000}) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/stream.go:111 +0x1a6 github.com/libp2p/go-libp2p/p2p/muxer/yamux.(*stream).Read(0x6b2927?, {0xc0edba6000?, 0x6b2960?, 0xc0004c2680?}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/muxer/yamux/stream.go:17 +0x1e github.com/libp2p/go-libp2p/p2p/net/swarm.(*Stream).Read(0xc021c66300, {0xc0edba6000?, 0xc0ed201f80?, 0xc0ed201f10?}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/net/swarm/swarm_stream.go:55 +0x36 bufio.(*Reader).fill(0xc026f2c4e0) /usr/local/go/src/bufio/bufio.go:106 +0x103 bufio.(*Reader).ReadByte(0xc026f2c4e0) /usr/local/go/src/bufio/bufio.go:264 +0x2c github.com/multiformats/go-varint.ReadUvarint({0x43d7240, 0xc026f2c4e0}) /home/stuart/go/pkg/mod/github.com/multiformats/go-varint@v0.0.6/varint.go:80 +0x74 github.com/libp2p/go-msgio/protoio.(*uvarintReader).ReadMsg(0xc0ed8156c0, {0x43ebb38, 0xc0ed201f80}) /home/stuart/go/pkg/mod/github.com/libp2p/go-msgio@v0.2.0/protoio/uvarint_reader.go:70 +0x85 github.com/libp2p/go-libp2p-pubsub.(*PubSub).handleNewStream(0xc00031da00, {0x4403ae0, 0xc021c66300}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.0/comm.go:66 +0x30d github.com/libp2p/go-libp2p/p2p/host/basic.(*BasicHost).SetStreamHandler.func1({0xc0ed9479a0, 0xe}, {0x7f9a04a2e390?, 0xc021c66300}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/host/basic/basic_host.go:569 +0x76 created by github.com/libp2p/go-libp2p/p2p/host/basic.(*BasicHost).newStreamHandler /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/host/basic/basic_host.go:410 +0x715 goroutine 685 [select, 6 minutes]: github.com/filecoin-project/go-statemachine.(*StateMachine).run(0xc00f190960) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/machine.go:53 +0x130 created by github.com/filecoin-project/go-statemachine.(*StateGroup).loadOrCreate /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/group.go:132 +0x570 goroutine 628 [IO wait]: internal/poll.runtime_pollWait(0x7f9a351292c8, 0x72) /usr/local/go/src/runtime/netpoll.go:302 +0x89 internal/poll.(*pollDesc).wait(0xc0000df080?, 0xc019114000?, 0x0) /usr/local/go/src/internal/poll/fd_poll_runtime.go:83 +0x32 internal/poll.(*pollDesc).waitRead(...) /usr/local/go/src/internal/poll/fd_poll_runtime.go:88 internal/poll.(*FD).Read(0xc0000df080, {0xc019114000, 0x1000, 0x1000}) /usr/local/go/src/internal/poll/fd_unix.go:167 +0x25a net.(*netFD).Read(0xc0000df080, {0xc019114000?, 0xc0237e4700?, 0x1c?}) /usr/local/go/src/net/fd_posix.go:55 +0x29 net.(*conn).Read(0xc0009855f0, {0xc019114000?, 0xc0110ea998?, 0xc019123d48?}) /usr/local/go/src/net/net.go:183 +0x45 bufio.(*Reader).Read(0xc000a4d440, {0xc00025ec90, 0x2, 0x6fa62e?}) /usr/local/go/src/bufio/bufio.go:236 +0x1b4 io.ReadAtLeast({0x43d7260, 0xc000a4d440}, {0xc00025ec90, 0x2, 0x2}, 0x2) /usr/local/go/src/io/io.go:331 +0x9a io.ReadFull(...) /usr/local/go/src/io/io.go:350 github.com/libp2p/go-libp2p/p2p/security/noise.(*secureSession).readNextInsecureMsgLen(0xc00025ec00) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/security/noise/rw.go:133 +0x3b github.com/libp2p/go-libp2p/p2p/security/noise.(*secureSession).Read(0xc00025ec00, {0xc00f5509e4, 0xc, 0xc}) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/security/noise/rw.go:52 +0x22f io.ReadAtLeast({0x7f9a049c0680, 0xc00025ec00}, {0xc00f5509e4, 0xc, 0xc}, 0xc) /usr/local/go/src/io/io.go:331 +0x9a io.ReadFull(...) /usr/local/go/src/io/io.go:350 github.com/libp2p/go-yamux/v3.(*Session).recvLoop(0xc0004a6120) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:644 +0xf7 github.com/libp2p/go-yamux/v3.(*Session).recv(0x0?) /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:616 +0x1e created by github.com/libp2p/go-yamux/v3.newSession /home/stuart/go/pkg/mod/github.com/libp2p/go-yamux/v3@v3.1.2/session.go:158 +0x66a /filecoin-project/go-statemachine@v1.0.2/group.go:132 +0x570 goroutine 2977 [select, 6 minutes]: github.com/filecoin-project/go-statemachine.(*StateMachine).run(0xc01cf1fa90) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/machine.go:53 +0x130 created by github.com/filecoin-project/go-statemachine.(*StateGroup).loadOrCreate /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/group.go:132 +0x570 goroutine 8986 [select, 6 minutes]: github.com/filecoin-project/go-statemachine.(*StateMachine).run(0xc022600640) /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/machine.go:53 +0x130 created by github.com/filecoin-project/go-statemachine.(*StateGroup).loadOrCreate /home/stuart/go/pkg/mod/github.com/filecoin-project/go-statemachine@v1.0.2/group.go:132 +0x570 goroutine 467580 [select, 3 minutes]: github.com/libp2p/go-libp2p/p2p/protocol/identify.(*peerHandler).loop(0xc02aa7ee10, {0x43f01a8, 0xc0161f3700}, 0xc026a63e00?) /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/protocol/identify/peer_loop.go:89 +0x105 created by github.com/libp2p/go-libp2p/p2p/protocol/identify.(*peerHandler).start /home/stuart/go/pkg/mod/github.com/libp2p/go-libp2p@v0.22.0/p2p/protocol/identify/peer_loop.go:71 +0xf0

[stuberman]

Pprof output

pprof.log

[dirkmc]

@stuberman thanks for the bug report. It looks like an issue in the underlying json-rpc library that boost uses to call out to lotus

[dirkmc]

Closing in favour of https://github.com/filecoin-project/go-jsonrpc/issues/87

[dirkmc]

Reopening - I misdiagnosed the source of the panic. Looks like it's actually coming from this line in go-data-transfer: https://github.com/filecoin-project/go-data-transfer/blob/v1.15.2/channels/channel_state.go#L105

[dirkmc]

Fixed in https://github.com/filecoin-project/go-data-transfer/pull/354 and merged into boost in https://github.com/filecoin-project/boost/pull/1056