filecoin-project / lotus

Reference implementation of the Filecoin protocol, written in Go
https://lotus.filecoin.io/
Other
2.86k stars 1.27k forks source link

Connecting worker with shared storage reports N^N useless information #10454

Open RobQuistNL opened 1 year ago

RobQuistNL commented 1 year ago

Checklist

Lotus component

Lotus Version

1.20.3

Repro Steps

  1. Start miner with multiple paths to long term storage
  2. Run multiple workers that also connect to those same long term storages (e.g. over NFS) (I read this is somehow a requirement for finalize tasks to work... which makes no sense at all, but okay)
  3. Note that every time a worker connects, it submits the same storages and its containing sectors.

This also happens for the "storage health" reports. Completely useless to send information about a folder that the miner/worker itself already has locally...

Describe the Bug

This is useless because the sectorstore.json tells that the store is already "indexed" and known. The workers don't have to each submit the contents. Enormous waste of I/O.

Logging Information

-
RobQuistNL commented 1 year ago

Probably a result of the millions of requests being done:

{"level":"info","ts":"2023-03-11T20:51:56.453Z","logger":"main","caller":"lotus-worker/main.go:522","msg":"Opening local storage; connecting to master"}
{"level":"warn","ts":"2023-03-11T20:51:56.455Z","logger":"main","caller":"lotus-worker/main.go:115","msg":"opening path /mnt/s2/storage0:\n    github.com/filecoin-project/lotus/storage/paths.(*Local).open\n        /home/filecoinnew/lotus/storage/paths/local.go:236\n  - declaring storage in index:\n    github.com/filecoin-project/lotus/storage/paths.(*Local).OpenPath\n        /home/filecoinnew/lotus/storage/paths/local.go:196\n  - RPC client error: sendRequest failed: Post \"http://172.16.0.1:2345/rpc/v0\": context canceled"}