search

filecoin-project / filecoin-plus-large-datasets

Hub for client applications for DataCap at a large scale
110 stars 62 forks source link

[DataCap Application] - CognoCloud #388

Open jessie8o8 opened 2 years ago

jessie8o8 commented 2 years ago

Large Dataset Notary Application

To apply for DataCap to onboard your dataset to Filecoin, please fill out the following.

Core Information

Please respond to the questions below by replacing the text saying "Please answer here". Include as much detail as you can in your answer.

Project details

Share a brief history of your project and organization. Cloud services company offering iaas and dramas for our customers. We wish to have an archived backup copy of the data

What is the primary source of funding for this project? N/A

What other projects/ecosystem stakeholders is this project associated with? Moose Mining and other reputable North American SPs

Use-case details

Describe the data being stored onto Filecoin VMDKs, VHDs, files, etc of our customers

Where was the data in this dataset sourced from? Our servers and storage in Toronto

Can you share a sample of the data? A link to a file, an image, a table, etc., are good ways to do this. cannot be shared

Confirm that this is a public dataset that can be retrieved by anyone on the Network (i.e., no specific permissions or access rights are required to view the data). Data will be encrypted

What is the expected retrieval frequency for this data? Very rare

For how long do you plan to keep this dataset stored on Filecoin? As long as possible for longer term archiving

DataCap allocation plan

In which geographies (countries, regions) do you plan on making storage deals? United States and Canada

How will you be distributing your data to storage providers? Is there an offline data transfer process? Online over the internet

How do you plan on choosing the storage providers with whom you will be making deals? This should include a plan to ensure the data is retrievable in the future both by you and others. Moose Mining and other reputable North American SPs

How will you be distributing deals across storage providers? There is going to be an offline data transfer process where files are hosted online and other storage providers can download them.

Do you have the resources/funding to start making deals as soon as you receive DataCap? Yes

What support from the community would help you onboard onto Filecoin? ESPA Community

large-datacap-requests[bot] commented 2 years ago

Thanks for your request! :exclamation: We have found some problems in the information provided. We could not find your Name in the information provided We could not find your Filecoin address in the information provided We could not find the Datacap requested in the information provided We could not find any Web site or social media info in the information provided We could not find any Expected weekly DataCap usage rate in the information provided We could not find any Region in the information provided

Please, take a look at the request and edit the body of the issue providing all the required information.
large-datacap-requests[bot] commented 2 years ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

large-datacap-requests[bot] commented 2 years ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

large-datacap-requests[bot] commented 2 years ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

large-datacap-requests[bot] commented 2 years ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

dkkapur commented 2 years ago

@jessie8o8 - going to need a few things from you for this one:

  1. need some form of KYC verification that you are indeed representing CongoCloud. Easiest way to do this is to send an email thread with them and you on it to filplus@fil.org that we can screencap and share with the community.
  2. need some proof that the data is actually useful and valuable to bring onto Filecoin for notaries to come onboard with. is there a way for you to share some more details on the nature of the data and why it is being onboarded to Filecoin? Notaries that typically support enterprise use cases include @MegTei @swatchliu @Fenbushi-Filecoin @dannyob @neogeweb3. Others that might be interested: @cryptowhizzard @flyworker.

The default path for encrypted datasets today is what Seal did here: https://github.com/filecoin-project/notary-governance/issues/489, however that was with a higher burden of proof that required NDAs to discuss the client and the dataset. If you can garner enough support from at least 5 notaries, we can either proceed the same way or find a better path here.

cryptowhizzard commented 2 years ago

I will support this one, and yes it’s PikNik

jessie8o8 commented 2 years ago

Hi @dkkapur,

Correct, I work for PiKNiK! We understand the reasonable doubt you may have and are please to present the following proposal with the support of 5 notaries. The following is the proposal for Cognocloud's dataset. Please let me know if you have any further questions!

Cognocloud LDN Proposal

Project Description We propose the onboarding of 2 PiBs of data to the Filecoin Network to help one of the first ESPA participants onboard their very first customer who provides MSP services to major Canadian and international firms.

Client we are representing : CognoCloud Site: https://cognocloud.ca/ Founded: 2014 LinkedIn: https://www.linkedin.com/company/cognocloud/about/

ESPA Participant that sourced the client: Moose Mining LLC Cohort: #1 ESPA Cohort #1 Testimonial Video that features Moose Mining: https://youtu.be/dRD0OIxjPeU?t=136

Transparency in KYC CognoCloud is an Ontario-based company and leading provider of disaster recovery as a service, data center design, virtualization & hyperconverged infrastructure. Founded in 2014, the company specializes in modern day enterprise IT infrastructure to allow other companies to grow and scale while protecting the environment from any disasters that may occur. They work to solve problems of modern day data centers at the enterprise scale. The organization serves major companies such as EY, Red River Mutual and AccessSMT.

We are completing a KYC process with CognoCloud with the help of trusted community notaries. Additionally, an email thread has been forward to filplus@fil.org to validate our relationship with senior leadership at CognoCloud.

Data Set CognoCloud protects its significant volume of data in the form of VMDKs and VHDs. Because it is pertinent to the privacy of the customers of CognoCloud, these data should remain private and are considered confidential.

Why is it useful to the Filecoin network? CognoCloud further proves the value of Web2 use cases on our Web3 storage platform. They provide disaster recovery and protect their customers from ransomware, data loss, and hardware failure. Its data is not only useful to CognoCloud, but also their end customers and the data marketplace itself.

Another reason why CognoCloud is useful to the Filecoin network is because they are one of the first private clients willing to work with the ESPA community’s newly trained storage provider participants (Moose Mining, Lucky Strike, etc.), which increases the diversity of SPs willing to support real enterprise and FIL+ data beyond the usual few entities who have recently been dominating LDN requests.

Notaries that Support the Project [person / org / region / Github app]

Kobby Chen / Fenbushi / Greater China / https://github.com/filecoin-project/notary-governance/issues/418 Meg Dennis / Holon / Oceania & Global / https://github.com/filecoin-project/notary-governance/issues/460 Wijnand / DCENT / Speedium, DCENT BV / Europe / https://github.com/filecoin-project/notary-governance/issues/414 Eric / Bytebase / Greater China / https://github.com/filecoin-project/notary-governance/issues/419 Julien / Twinquasar/ Europe / https://github.com/filecoin-project/notary-governance/issues/468

s0nik42 commented 2 years ago

I will support it as well

Kevin-PiKNiK commented 2 years ago

Thanks to @cryptowhizzard and @s0nik42 for the support. Proud to see the progress on this request made with Moose Mining, one of the startups from the very first ESPA Cohort.

Fenbushi-Filecoin commented 2 years ago

We can support the application.

jessie8o8 commented 2 years ago

@dkkapur Is this sufficient for approval? Let me know what else is needed.

Destore2023 commented 2 years ago

This proposal involves two points we need check: one is whether the actual owner of dataset knows and authorizes Congo Cloud to transfer their data to the public filecoin network (not the same concept with data encryption); The other is that Congo Cloud authorizes PikNik to submit this applications.

We are willing to support this one after these two points are clear.

raghavrmadya commented 2 years ago

Hi @jessie8o8 , can you share more details on how you plan to deploy the 2 PiBs?

dkkapur commented 2 years ago

Confirming that email thread with senior leadership of CognoCloud (Managing Director) was forwarded to the filplus inbox, and specifically suggested they were looking forward to using Filecoin to break into Web3. I think that addresses @swatchliu's comments above.

Two next steps:

jessie8o8 commented 2 years ago

Thanks for the confirmation of the email @dkkapur

@raghavrmadya - We plan to collaborate with other ESPA participants such as HPI3 (Cohort 1 as seen in video), Ideal Storage (Cohort 2).

Destore2023 commented 2 years ago

@dkkapur I would like to point out in particular that the first and most important point I mentioned needs to be clarified. The actual owner of private dataset is NOT Congo Cloud. Congo Cloud is only a storage provider.

20af4c90f9099a832b7c8301ddfb7ac

raghavrmadya commented 2 years ago

We need clarification on @swatchliu's point and support from a 5th notary to proceed @jessie8o8

kernelogic commented 2 years ago

I can support this LDN as well.

jessie8o8 commented 2 years ago

Thanks for the support @kernelogic! We are currently reaching out to Cognocloud to answer @swatchliu's question :)

jessie8o8 commented 2 years ago

@swatchliu point is addressed by the following comment from Cognocloud:

The customers hired Cogno to protect their data in the event a disaster happens or they lose it. CognoCloud already uses (and have used) 3rd party clouds (such as AWS, Google, Azure etc) for storing customer data for longer term retention as extra failsafe copies. So with Filecoin we have the opportunity to provide our customers a 4th copy of their encrypted data at NO COST, it is simply a win for the customer and it is now part of our service offering moving forward. Assuming storing the data with Filecoin is the same as storing it in any other cloud/data center like AWS, Azure, etc.....this should not be a concern. Should we be concerned about what is happening to the data on Filecoin? AWS/Google/Azure would never be asking us these types of questions as it is not unusual for managed service providers (like Cogno) to use AWS, Azure, etc to store customer backup data. If there's some weird regulation/requirement where each of our customers needs to be onboarded individually, that would be inefficient and complicates things greatly. I hope this helps.

jessie8o8 commented 2 years ago

@raghavrmadya Let me know if I'm missing anything to be approved! :)

glutenfreepolice commented 2 years ago

Wait so who exactly is the client here? Why are they hiding? Too many SPs involved. Seems weird.

large-datacap-requests[bot] commented 2 years ago

Deleting comment

@raghavrmadya hasn't the permissions to post this comment.

Please, contact the assignee of this issue.

raghavrmadya commented 2 years ago

While the client has been satisfied getting support from 5 notaries, all notaries should ensure a high level of confidence before signing this application

raghavrmadya commented 2 years ago

The gov team will create a custom LDN with the notaries who have supported the application.

galen-mcandrew commented 2 years ago

@jessie8o8 Since this is an encrypted dataset, I think it falls under the 'exception' flow for large dataset applications, similar to Antarctic and Beacon.

We are working on a template to make this easier, but in the meantime can you please open an issue in the Notary Goverance Repo similar to these?

Additionally, please add some more explicit information about the distribution of storage providers. It sounds like you are a notary and storage provider, representing (sourcing as bizdev) a web2 org which is willing to perform KYC (not private and undisclosed data owner), the data is encrypted, and you will be distributing deals across storage providers, but not across regions. Is that correct?

jessie8o8 commented 2 years ago

Hi @galen-mcandrew . I have opened this application as an issue.

To answer your question (also mentioned in the issue), For this application we are acting as representatives of the Cognocloud (confirmed with an email thread forwarded to filplus@fil.org) . CognoCloud was sourced by MooseMining, an ESPA cohort one participant, and have asked PiKNiK for help with the datacap process. The client has requested the data remain in North America, thus the SP's chosen are all in North America.

Yes, PiKNiK is a notary but we are not acting a notary on this application.

Hope this helps!

jessie8o8 commented 2 years ago

@galen-mcandrew @dkkapur @raghavrmadya Please let me know what is needed to move forward with this application :)

large-datacap-requests[bot] commented 1 year ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

dkkapur commented 1 year ago

Datacap Request Trigger

Total DataCap requested

2 PiB

Expected weekly DataCap usage rate

100 TiB

Client address

f1e577krmeli4iuclarvl37pbxtthv66ceuuk4hii

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Multisig Notary address

f01924203

Client address

f1e577krmeli4iuclarvl37pbxtthv66ceuuk4hii

DataCap allocation requested

50TiB

s0nik42 commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzaceadogo633ivpw4tvux66lg43j6o72cpavrcyuieedenk2qns3vevk

Address

f1e577krmeli4iuclarvl37pbxtthv66ceuuk4hii

Datacap Allocated

50.00TiB

Signer Address

f1wxhnytjmklj2czezaqcfl7eb4nkgmaxysnegwii

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceadogo633ivpw4tvux66lg43j6o72cpavrcyuieedenk2qns3vevk

cryptowhizzard commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaceaceecngt5zbnc7qi3ncxo2smzjvx4vfxwxwh43h5sdmthyynarja

Address

f1e577krmeli4iuclarvl37pbxtthv66ceuuk4hii

Datacap Allocated

50.00TiB

Signer Address

f1krmypm4uoxxf3g7okrwtrahlmpcph3y7rbqqgfa

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceaceecngt5zbnc7qi3ncxo2smzjvx4vfxwxwh43h5sdmthyynarja

Sunnyiscoming commented 1 year ago

Are there any problems with using datacap?

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 2

Multisig Notary address

f01924203

Client address

f1e577krmeli4iuclarvl37pbxtthv66ceuuk4hii

DataCap allocation requested

100TiB

Id

597e9151-8f0b-4878-89f5-7bf465848c6c

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f01924203

Client address

f1e577krmeli4iuclarvl37pbxtthv66ceuuk4hii

Rule to calculate the allocation request amount

100% of weekly dc amount requested

DataCap allocation requested

100TiB

Total DataCap granted for client so far

50TiB

Datacap to be granted to reach the total amount requested by the client (2 PiB)

1.95PiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
1195 2 50TiB 56.16 11.74TiB
Fenbushi-Filecoin commented 1 year ago

Are there any reasons that the application is not activated for almost half a year?

jamerduhgamer commented 1 year ago

Hi @Sunnyiscoming, no issues with using the datacap. @Fenbushi-Filecoin, one of the SPs has had issues sourcing the FIL collateral required to seal their copy so we have had to pause this project. SP has recently acquired said FIL and we have started sealing this dataset.

Fenbushi-Filecoin commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacea26i3jxerqti4cjgavzulz5xqnwufg24agdknzxqttbwprirzeki

Address

f1e577krmeli4iuclarvl37pbxtthv66ceuuk4hii

Datacap Allocated

100.00TiB

Signer Address

f1yqydpmqb5en262jpottko2kd65msajax7fi4rmq

Id

597e9151-8f0b-4878-89f5-7bf465848c6c

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacea26i3jxerqti4cjgavzulz5xqnwufg24agdknzxqttbwprirzeki

mjroddy commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacebz6izuoztb2r6ov22v5o2m2vptr5eae3fsao4xdbv4xuufi6yftc

Address

f1e577krmeli4iuclarvl37pbxtthv66ceuuk4hii

Datacap Allocated

100.00TiB

Signer Address

f1ystxl2ootvpirpa7ebgwl7vlhwkbx2r4zjxwe5i

Id

597e9151-8f0b-4878-89f5-7bf465848c6c

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebz6izuoztb2r6ov22v5o2m2vptr5eae3fsao4xdbv4xuufi6yftc

herrehesse commented 1 year ago

Awesome application! Had a talk with PikNik and the client, very supportive of these types of LDNs bringing real customers to Filecoin. Keep up the good work, you got my support.

cryptowhizzard commented 1 year ago

Also very supportive. Great work PikNik.

s0nik42 commented 1 year ago

me too

kenzz45 commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

Storage Provider Distribution

⚠️ 1 storage providers sealed more than 70% of total datacap - f01872494: 86.23%

⚠️ All storage providers are located in the same region.

Deal Data Replication

⚠️ 100.00% of deals are for data replicated across less than 3 storage providers.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

kenzz45 commented 1 year ago

Awesome application! Had a talk with PikNik and the client, very supportive of these types of LDNs bringing real customers to Filecoin. Keep up the good work, you got my support.

No actual data owner info, No SP introduction, Serious deviation from the weekly schedule, Zero HTTP retrieval success rate, Same region, bad report...

@herrehesse How can you come to such a ridiculous conclusion?

@mjroddy Signing with no due diligence! Warning!!

Requesting a pauze on this application.

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 3

Multisig Notary address

f01924203

Client address

f1e577krmeli4iuclarvl37pbxtthv66ceuuk4hii

DataCap allocation requested

200TiB

Id

1c692f70-fa44-4241-85bf-5e7cded046dd

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f01924203

Client address

f1e577krmeli4iuclarvl37pbxtthv66ceuuk4hii

Rule to calculate the allocation request amount

200% of weekly dc amount requested

DataCap allocation requested

200TiB

Total DataCap granted for client so far

9094.9YiB

Datacap to be granted to reach the total amount requested by the client (2 PiB)

9094.9YiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
4316 2 100TiB 84.63 14.05TiB
jamerduhgamer commented 1 year ago

Hi @kenzz45, thank you for the concerns.

The original data owner is mentioned in the proposal here where Moose Mining LLC was the ESPA Participant that sourced the client CognoCloud. This was explained in the comments above.

This LDN follows the old E-FIL+ application process where 5 notaries were needed to support the application which is also mentioned in the proposal.

@mjroddy is part of the Holon team which is one of the 5 notaries mentioned and due diligence was done by the original 5 notaries when the LDN was created a year ago.