filecoin-project / notary-governance

114 stars 58 forks source link

v5 Notary Allocator Application: CIDgravity data preparation and onboarding #1095

Closed s0nik42 closed 3 months ago

s0nik42 commented 8 months ago

v5 Notary Allocator Application

To apply to be an allocator, organizations will submit one application for each proposed pathway to DataCap. If you will be designing multiple specific pathways, you will need to submit multiple applications.

Please complete the following steps:

1. Fill out the information below and create a new GitHub Issue

  1. Notary Allocator Pathway Name (This can be your name, or the name of your pathway/program. For example "E-Fil+"): CIDgravity data preparation and onboarding

  2. Organization Name: Twinquasar - CIDgravity

  3. On-chain address for Allocator (Provide a NEW unique address. During ratification, you will need to initialize this address on-chain): f1pwityjf7zqokgwpjqple46ma5j5v7lsdfxwsasy

  4. Country of Operation (Where your organization is legally based): France

  5. Region of Operation (What region will you serve?): World Wide

  6. Type of Allocator, diligence process: (Automated/programmatic, Market-based, or Manual (human-in-the-loop at some phase): Automated/programmatic

  7. DataCap requested for allocator for 12 months of activity (This should be an estimate of overall expected activity. Estimate the total amount of DataCap you will be distributing to clients in 12 months, in TiB or PiB): 100PiB

2. Access allocator application (download to save answers)

Click link below to access a Google doc version of the allocator application that can be used to save your answers if you are not prepared to fully submit the application in Step 3. https://docs.google.com/document/d/1-Ze8bo7ZlIJe8qX0YSFNPTka4CMprqoNB1D6V7WJJjo/copy

3. Submit allocation application

Clink link below to access full allocator questionnaire and officially submit your answers: https://airtable.com/appvyE0VHcgpAkt4Z/shrQxaAIsD693e1ns

Note: Sections of your responses WILL BE posted back into the GitHub issue tracking your application. The final section (Additional Disclosures) will NOT be posted to GitHub, and will be maintained by the Filecoin Foundation. Application information for notaries not accepted and ratified in this round will be deleted.

ghost commented 8 months ago

Basic Information

1. Notary Allocator Pathway Name: CIDgravity data preparation and onboarding

2. Organization: Twinquasar - CIDgravity

3. On Chain Address for Allocator: f1pwityjf7zqokgwpjqple46ma5j5v7lsdfxwsasy

4. Country of Operation: France

5. Region(s) of operation: Other, South America, North America, Japan, Oceania, Europe, Greater China, Asia minus GCR, Africa

6. Type of Allocator: Automatic

7. DataCap requested for allocator for 12 months of activity: 100PiB

8. Is your allocator providing a unique, new, or diverse pathway to DataCap? How does this allocator differentiate itself from other applicants, new or existing?: This pathway operates as a fully S3 bucket, prioritizing key benefits:

Allocation Process Overview

  1. Registration:
    • The client connects to CIDgravity with its GitHub account and claims a Filecoin address in CIDgravity (signs a challenge).
    • The client register all information KYC information
    • KYC is processed
    • Upon success, the contact is check by an external international KYC service
    • Upon success the client get an s3/webdav environment created and share the key with the CIDgravity team.
  2. Onboarding:
    • The client drop files to the S3/WEbdav enpoints.
    • The data is automatically packed and pushed to Filecoin. The data is distributed over at least 10 different entities
    • CIDgravity keep a local copy of the data (not on Filecoin).
    • When the datacap is as low as 1 TiB, an audit report is created, verifying and logging the allocation for external audit (fil+ etc…)and the next tranche is automatically allocated.

Reporting / Monitoring:

9. As a member in the Filecoin Community, I acknowledge that I must adhere to the Community Code of Conduct, as well other End User License Agreements for accessing various tools and services, such as GitHub and Slack.: Acknowledge

Client Diligence

10. Who are your target clients?: Individuals learning about Filecoin, Small-scale developers or data owners, Enterprise Data Clients, Other (specified above)

11. Describe in as much detail as possible how you will perform due diligence on clients. If you are proposing an automated pathway, what diligence mechanism will you use to determine client eligibility?: Corporate Details:

Documentation:

12. Please specify how many questions you’ll ask, and provide a brief overview of the questions.:

13. Will you use a 3rd-party Know your client (KYC) service?: yes

14. Can any client apply to your pathway, or will you be closed to only your own internal clients? (eg: bizdev or self-referral): Any clients

15. How do you plan to track the rate at which DataCap is being distributed to your clients?: CIDgravity has direct access to the chain and the preceding datacap allocation epoch before assigning the new one. All allocation will be logs to our tracking DB and transparently shared publicly

Data Diligence

16. As an operating entity in the Filecoin Community, you are required to follow all local & regional regulations relating to any data, digital and otherwise. This may include PII and data deletion requirements, as well as the storing, transmit: Acknowledge

17. What type(s) of data would be applicable for your pathway?: Public Open Dataset (Research/Non-Profit), Public Open Commercial/Enterprise, Private Commercial/Enterprise, Private Non-Profit/Social Impact, Other (specified elsewhere)

18. How will you verify a client’s data ownership? Will you use 3rd-party KYB (know your business) service to verify enterprise clients?: No KYB nor data ownership will be performed as this pathway is specifically crafted to prevent collaboration between clients and Service Providers (SPs) in attempting to cheat the system and the block reward mechanism. More below 19.

19. How will you ensure the data meets local & regional legal requirements?: We will respond to notifications through our Notice and Takedown process. Upon notification, the data will be promptly removed from our platform. No further data processing will occur for this account until clarification is obtained. Additionally, we will inform the responsible Service Providers (SPs) storing the content without delay.

20. What types of data preparation will you support or require?: We provide a complete pipeline with a S3 and Webdav connector to Filecoin. The pipeline rely on a mix of OSS tools + CIDgravity

21. What tools or methodology will you use to sample and verify the data aligns with your pathway?: N/A

Data Distribution

22. How many replicas will you require to meet programmatic requirements for distribution?: 2+

23. What geographic or regional distribution will you require?: Taking into account that the client do not decide which miner will store the data. We have the guarantee it will be widly distributed through SPs The client has no role in selecting the data location. The data will be distributed across all regions, allowing for up to 10 copies as per the client's preference. Our track record includes successfully onboarding datasets to 25 different miners, as documented here: GitHub Link. For private data, clients may have specific constraints regarding the data location; we can further tailor this to accommodate their needs.

24. How many Storage Provider owner/operators will you require to meet programmatic requirements for distribution?: 2+

25. Do you require equal percentage distribution for your clients to their chosen SPs? Will you require preliminary SP distribution plans from the client before allocating any DataCap?: CIDgravity's algorithm is designed to leverage the best reputation for the selected client. The algorithm ensures the distribution of data across as many Service Providers (SPs) as possible. Our estimation indicates that easily reaching 20 SPs is achievable through this approach, optimizing the distribution for enhanced reliability and availability.

26. What tooling will you use to verify client deal-making distribution?: CIDgravity will keep track of deals, compare them with the Filecoin Chain, and make the audit results public.

27. How will clients meet SP distribution requirements?: We initiate the deal on behalf of the client, ensuring that the distribution will be respected.

28. As an allocator, do you support clients that engage in deal-making with SPs utilizing a VPN?: Yes, we do

DataCap Allocation Strategy

29. Will you use standardized DataCap allocations to clients?: Yes, standardized

30. Allocation Tranche Schedule to clients:: • First: 10TiB • Second:10TiB • Third:10TiB • Fourth:10TiB • Max per client overall: No limit

31. Will you use programmatic or software based allocations?: Yes, standardized and software based

32. What tooling will you use to construct messages and send allocations to clients?: CIDgravity is already plugged to the filecoin chain. It will support to send the message natively

33. Describe the process for granting additional DataCap to previously verified clients.: Datacap Exhaustion Check: - When the Datacap reaches 1TiB, the API ceases to provide information on the next Service Provider (SP) and instead returns an error message: ""No more datacap available. Waiting for next tranche validation."" Concurrent Datacap Renewal Process: - Simultaneously, the datacap renewal process is initiated. - If the datacap has been consumed as per CIDgravity's request, the next tranche is automatically allocated by sending a message on-chain.

34. Describe in as much detail as possible the tools used for: • client discoverability & applications • due diligence & investigation • bookkeeping • on-chain message construction • client deal-making behavior • tracking overall allocator health • disput: • client discoverability & applications ð Current CIDGravity onboarding process , with a new section for selecting datacap pathway • due diligence & investigation N/A • bookkeeping All actions will be recorded in the CIDgravity database, and a public WebUI will provide access to these logs. • on-chain message construction Message send direction by CIDgravity • client deal-making behavior We already track any deals since proposed by CIDgravity to onchain status. • tracking overall allocator health Stats will be available. • dispute discussion & resolution Dispute will be done via slack for private and github for public • community updates & comms * We will utilize our Slack channel: #fil-cid-gravity for announcements. These announcements will also be crossposted to all Service Provider (SP) workgroups , #fil-plus and #fil-hobby.

Tools and Bookkeeping

35. Will you use open-source tooling from the Fil+ team?: To maintain transparency, we will keep our stack’s compliance with the filplus-checker-app.

36. Where will you keep your records for bookkeeping? How will you maintain transparency in your allocation decisions?: We are committed to full transparency by publicly disclosing the following information through our portal:

Risk Mitigation, Auditing, Compliance

37. Describe your proposed compliance check mechanisms for your own clients.: As previously outlined, adherence to our datacap consumption guidances is mandatory. Any deviation from these guidelines will result in the automatic suspension of the account. Clients seeking a deeper analysis and account reactivation will be subject to a fee.

38. Describe your process for handling disputes. Highlight response times, transparency, and accountability mechanisms.: Disputes between clients and CIDgravity are handled through the outlined process via Slack. Disputes involving Allocators and CIDgravity will be managed through the community-defined process on Slack and GitHub.

39. Detail how you will announce updates to tooling, pathway guidelines, parameters, and process alterations.: Part of the CIDgravity annoucement already in place and as descibed above.

40. How long will you allow the community to provide feedback before implementing changes?:

41. Regarding security, how will you structure and secure the on-chain notary address? If you will utilize a multisig, how will it be structured? Who will have administrative & signatory rights?: The entire process will be fully automated. Instead of utilizing a multisig, the signatory key will be encrypted with a passphrase known exclusively to Julien NOEL.

42. Will you deploy smart contracts for program or policy procedures? If so, how will you track and fund them?: Not planned yet.

Monetization

43. Outline your monetization models for the services you provide as a notary allocator pathway.:

44. Describe your organization's structure, such as the legal entity and other business & market ventures.: CIDgravity is a solution developed by Twin Quasar. A French company leader in the Filecoin ecosystem / PLN. Since 2021

45. Where will accounting for fees be maintained?: Bank

Past Experience, Affiliations, Reputation

46. If you've received DataCap allocation privileges before, please link to prior notary applications.: We’ve been notary since 2021 round 1. Notary Round 1 : https://github.com/filecoin-project/notary-governance/issues/5 Notary Round 2 : https://github.com/filecoin-project/notary-governance/issues/136 Notary Round 3 : https://github.com/filecoin-project/notary-governance/issues/468 Notary Round 4 : https://github.com/filecoin-project/notary-governance/issues/739

47. How are you connected to the Filecoin ecosystem? Describe your (or your organization's) Filecoin relationships, investments, or ownership.: Twin Quasar provides top-tier solutions, delivering infrastructure and open-source tooling designed for storage providers and Filecoin ecosystem partners as well as seamless data onboarding and Filecoin+ services. Strategically located in Paris, our facilities are directly connected to our 12,000km fiber backbone network. Explore our Tier 3 colocation services, managed services, IaaS, and Filecoin expertise, all marked by a first-class 100% client satisfaction rate, and run on 100% renewable energy.

48. How are you estimating your client demand and pathway usage? Do you have existing clients and an onboarding funnel?: We already have 2 prospects identified that could be interested in this pathway

filplus-bot commented 6 months ago

The request has been signed by a new Root Key Holder

Message sent to Filecoin Network

bafy2bzaceb7fqqtj2kepi5tpnwtpwft55apgf623lls7hfhwtxdp7y2l2egr2

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceb7fqqtj2kepi5tpnwtpwft55apgf623lls7hfhwtxdp7y2l2egr2

galen-mcandrew commented 6 months ago

Datacap Request for Allocator

Address

f1pwityjf7zqokgwpjqple46ma5j5v7lsdfxwsasy

Datacap Allocated

5PiB

filplus-bot commented 6 months ago

The request has been signed by a new Root Key Holder

Message sent to Filecoin Network

bafy2bzaceck2olnwbq2uten5qqbjsxrbwz5xryq6gt5crw6cuxpsz5dvqvrqc

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceck2olnwbq2uten5qqbjsxrbwz5xryq6gt5crw6cuxpsz5dvqvrqc

galen-mcandrew commented 3 months ago

@s0nik42 following up here, it looks like this may have missed second RKH signature.

I've checked two dashboards, and this address (f1pwityjf7zqokgwpjqple46ma5j5v7lsdfxwsasy // f02824311) is not showing up.

Can you verify if this address still needs the initial 5PiB allocation?

s0nik42 commented 3 months ago

Yes, it does, I was wondering why only 2 allocation shows up during our call yesterday, that's why :

f1pwityjf7zqokgwpjqple46ma5j5v7lsdfxwsasy = "Allocator CIDgravity data preparation and onboarding"
s0nik42 commented 3 months ago

let me know if you need anything from me

galen-mcandrew commented 3 months ago

Looks like there was an error in the approval message! That explains it. https://filscan.io/en/message/bafy2bzacebqvame6wyg6kw34ea3ouk7gduphcx4jpzsxuawyex7adrpcpttsm/

"Code": "ErrIllegalArgument(16)",

s0nik42 commented 3 months ago

Can you resend it ?

galen-mcandrew commented 3 months ago

Yes, working with RKH to resend, and watching for any exit code issues. Will update you this week. Thanks!

filplus-bot commented 3 months ago

The request has been signed by a new Root Key Holder

Message sent to Filecoin Network

bafy2bzacebq3c6gvf75ea6kukubqce7jxfwk4ibltdvk4k5svfg7cxe6xqf3o

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebq3c6gvf75ea6kukubqce7jxfwk4ibltdvk4k5svfg7cxe6xqf3o

galen-mcandrew commented 3 months ago

https://filfox.info/en/message/bafy2bzacecz2i2sedwugmiuz64kb6oklrom23dmhytryiagmedlua5wkktrgq

Just checked Lotus too:

lotus filplus check-notary-datacap f02824311 5629499534213120

@s0nik42 let me know if anything comes up.