Closed Fatman13 closed 1 year ago
Hi @Fatman13, thank you for your proposal! We are currently reviewing this grant and expect to have more information available next week.
Hi @Fatman13, thank you for your ongoing patience! This grant is still under our review. We will be in touch as soon as we have completed connecting with our ecosystem experts.
HI @Fatman13, can you confirm if the metrics are venus-specific or network-wide? Many thanks!
HI @Fatman13, can you confirm if the metrics are venus-specific or network-wide? Many thanks!
Metrics are specific to Venus and being built into Venus with embedded exporter for a front end to consume.
Thanks, @Fatman13! This grant has been approved. Would you like us to use the contact information on file for this grant?
Great to hear that! The email will be venus@ipfsforce.com. Thank you so much!
Thanks, @Fatman13!
Open Grant Proposal:
Data Onboarding Metrics
Name of Project:
Data Onboarding Metrics - Venus
Proposal Category: Choose one ofcore-dev
,devtools-libraries
Proposer:ipfs-force-community
(Optional) Technical Sponsor: Do you agree to open source all work you do on behalf of this RFP and dual-license under MIT, APACHE2, or GPL licenses?: YesProject Description
One of the issues that new SPs or even many veteran SPs facing everyday when they on-board loads of sectors is getting a clear picture of the heartbeat for their storage system to diagnose whatever has gone wrong in their pipeline. A thousand things could go wrong when moving sectors through SP’s storage systems such as chain head out of sync, messages stuck in mpool, missing block producing round, high API latency and etc. SPs have to navigate through these anomalies all the time and be quick to response to these conditions. This is where
Data Onboarding Metrics
for Venus Filecoin comes into play. We propose to build a series of critical metrics for each component of Venus Filecoin to reflect the live health of a storage system so that operators could have better knowledge of what’s going with their systems and then could better react to different situations instead of relying on guessing, digging through tons of logs or overly extensive dev-ops experience.Value
There are many benefits we see that
Data Onboarding Metrics
could bring to SPs to take control of their storage systems back instead of spending a lot time troubleshooting a black box. We believe metrics provides the toolbox for SP to minimize the impact of their operation errors, to get to see if winidowPost messages get properly sent out in time, to monitor time/latency for PoST computation and much more so that SPs do not get punished by the protocol unintentionally.Deliverables
Development Roadmap
The development could be loosely broken down into three parts: 1) Design 2) Implementation and lastly 3) maintenance of the metrics system.
Design
This phase includes milestone A and B in the above deliverable table. The team will be collecting ideas from community, concieve the 1st design of metrics system, and lastly build a POC/MVP for miner component. A embedded exporter that allows custom configuration will be included for easier integration with third party tools. A
metrics
module will be added to the miner project which may contain below parameters for SPs to monitor their storage pipeline.Implementation
This phase includes milestone C to E in the above deliverable table. The team will be continuing to collect ideas from community while implementing the metrics system for the rest of the Venus components. A list of parameters that
metrics
module will be adopting are listed below…messager
gateway
market
daemon
TBD
cluster
Note that all metrics are not final and subject to have more parameters when community see fit.
Maintenance
This phase includes milestone F to H in the above deliverable table. The team will be continuing to collect ideas and feedbacks from community while iterating on the metrics system for all Venus components. Documentations and easy-to-follow tutorials will be produced to help push metrics system to be adopted by broader community members. We hope after we are done with this phase SPs will have the tools they need to remove any obstacles when on-boarding large amount of sectors.
Total Budget Requested
The total budget requests is $48,000. The breakdown of the budget is associated with the deliverables of each milestone, defined above.
Maintenance and Upgrade Plans
The goal of the team is to support metrics system long term, which including continuously adding more critical parameters that community deemed worthy of monitoring. Therefore, easing the process of on-boarding large amount of data to the network.
Team
Team Members
Force community engineering team
Team Member LinkedIn Profiles
Team Website
https://forcecommunity.io/
Relevant Experience
Force community has been an active contributor to Web3 ecosystem and Filecoin ecosystem in general. The engineering team from Force community has a track record of contributing code to Lotus as far back as Testnet and Space Race.
Team code repositories
https://github.com/ipfs-force-community
Additional information
Force community is committed to become a major contributor to Web3 infrastructure and we see Filecoin at the core of the big Web3 migration. We hope that we could fast track the realization of Web3 adoption by contributing our software development capacity to the course and join hand in hand with all other ecosystem developers around the globe through this historical journey!