search

filecoin-project / filecoin-plus-large-datasets

Hub for client applications for DataCap at a large scale
110 stars 62 forks source link

[DataCap Application] BitFuFu - Cloud-mining Dataset #1094

Closed smallbull closed 11 months ago

smallbull commented 1 year ago

name: Large Dataset Notary application about: Clients should use this application form to request a DataCap allocation via a LDN for a dataset title: "BitFuFu - Cloud-mining Dataset" labels: 'application, Phase: Diligence' assignees: ''

Large Dataset Notary Application

To apply for DataCap to onboard your dataset to Filecoin, please fill out the following.

Core Information

Please respond to the questions below by replacing the text saying "Please answer here". Include as much detail as you can in your answer.

Project details

Share a brief history of your project and organization.

BitFuFu is a fast-growing digital asset mining service and world-leading cloud-mining service provider. BitFuFu has been invested by, and is the only cloud-mining strategic partner of Bitmain to date, a world-leading cryptocurrency mining hardware manufacturer. 
The Company had a hosting capacity of 140 MW at the end of 2021 across its global mining facilities network and strategic partnership with Bitmain.

What is the primary source of funding for this project?

Own funds and revenue of the company.

What other projects/ecosystem stakeholders is this project associated with?

BITMAIN related companies.

Use-case details

Describe the data being stored onto Filecoin

Promotional and training videos and documents produced during the company's business operations. A large number of logs generated by the company's servers and self-operated mining machines.These data does not involve the company's sensitive information. Total size: At least 5 PiB now

Where was the data in this dataset sourced from?

AWS Open dataset / youtube / Website / Not currently stored in a public database / public advertisement data backup in company internal storage

Can you share a sample of the data? A link to a file, an image, a table, etc., are good ways to do this. 

https://www.youtube.com/channel/UCEO7IgJoFsHPQIWRNoGmwuw

Confirm that this is a public dataset that can be retrieved by anyone on the Network (i.e., no specific permissions or access rights are required to view the data).

We confirm this is a public dataset that can be retrieved by anyone on the network with no specific permissions or access rights required.

What is the expected retrieval frequency for this data?

Multiple times per year.

For how long do you plan to keep this dataset stored on Filecoin?

At least 18 months.

DataCap allocation plan

In which geographies (countries, regions) do you plan on making storage deals?

All regions.

How will you be distributing your data to storage providers? Is there an offline data transfer process?

We will negotiate with SP, both online and offline are acceptable

How do you plan on choosing the storage providers with whom you will be making deals? This should include a plan to ensure the data is retrievable in the future both by you and others.

We will consider a combination of cost, service and geography, and give preference to miners of good scale and reputation.

How will you be distributing deals across storage providers?

We will store approximately 3-5 replica of data on different SPs, and we will follow the filecoin guideline to make our data more reliable and secure.
A detailed allocation plan for the SPs is in the works, we will share the details once it is ready.

Do you have the resources/funding to start making deals as soon as you receive DataCap? What support from the community would help you onboard onto Filecoin?

We have enough resources and funds to start trading. Perhaps we need more weekly allocation of DC.
large-datacap-requests[bot] commented 1 year ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

raghavrmadya commented 1 year ago

The sample is not enough to justify 5 PiBs. provide as much information as possible.

smallbull commented 1 year ago

The sample is not enough to justify 5 PiBs. provide as much information as possible.

link: https://pan.baidu.com/s/1SFgZzYYdrUG6oACudjOzuQ code: 34bd

simonkim0515 commented 1 year ago

Datacap Request Trigger

Total DataCap requested

5PiB

Expected weekly DataCap usage rate

100TiB

Client address

f3rm7hrijz2l3qth6nihxlf2no3so55vsb2dwcplwxwm7e6szwlx3on7fjerzomdinuitldmyngwlzxtz33vnq

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Multisig Notary address

f02049625

Client address

f3rm7hrijz2l3qth6nihxlf2no3so55vsb2dwcplwxwm7e6szwlx3on7fjerzomdinuitldmyngwlzxtz33vnq

DataCap allocation requested

50TiB

Id

8a0414b2-514c-42e0-b788-dba94b298a27

newwebgroup commented 1 year ago

About KYC&KYB 1：Could you send an email to filplus-app-review@fil.org ？

The content should include the number of the LDN application. If possible, please attach copies of the business license and other valid certificates

2：Can you provide more detailed information about other storage providers participated in this program, such as you can list SPs you have contacted with at present? 3.How large is your existing dataset? How much is the data growth per month?

smallbull commented 1 year ago

About KYC&KYB 1：Could you send an email to filplus-app-review@fil.org ？

The content should include the number of the LDN application. If possible, please attach copies of the business license and other valid certificates

2：Can you provide more detailed information about other storage providers participated in this program, such as you can list SPs you have contacted with at present? 3.How large is your existing dataset? How much is the data growth per month?

I have sent KYC&KYB email from the company mailbox. Please check and reply, Many thanks!

newwebgroup commented 1 year ago

About 1，Please provide a screenshot of the email sent

About 2 ，3 ；Please reply in detail

smallbull commented 1 year ago

About 1，Please provide a screenshot of the email sent

email-screenshot

About 2 ；Please reply in detail According to the SP resource form provided by the foundation, try to contact: https://docs.google.com/spreadsheets/d/1rwT9ie6qlOM9hVUkYZLGej523X1pMenaVKhFuvQOiNE/edit#gid=442291216 We also have contacted SPs: f0436065 f0701089 f01016248 f0694908

About 3 ；Please reply in detail Our existing dataset is 10PiB+. The data growth is 60~80TiB/month. These data include our daily business data and publicity data. Here is sample data: 【超级会员V6】通过百度网盘分享的文件：bitfufu 链接:https://pan.baidu.com/s/17Rz0tP0KVZausqizdYDqjA 提取码:9jb6 复制这段内容打开「百度网盘APP 即可获取」 https://www.bitfufu.com/ https://twitter.com/bitfufu1 https://www.youtube.com/watch?v=yiicKlsObRQ

newwebgroup commented 1 year ago

Because it was the first round, the Client answered my relevant questions, and the conditions for triggering signature had been met, so I chose to pass. If anyone has more questions about the application, they can keep watching and checking.

newwebgroup commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacecx2xvzxbkxo3tusjg3qjigpoe227dd5m7h5cehf5qxkjjnozwgus

Address

f3rm7hrijz2l3qth6nihxlf2no3so55vsb2dwcplwxwm7e6szwlx3on7fjerzomdinuitldmyngwlzxtz33vnq

Datacap Allocated

50.00TiB

Signer Address

f1e77zuityhvvw6u2t6tb5qlnsegy2s67qs4lbbbq

Id

8a0414b2-514c-42e0-b788-dba94b298a27

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacecx2xvzxbkxo3tusjg3qjigpoe227dd5m7h5cehf5qxkjjnozwgus

Joss-Hua commented 1 year ago

agree with newwebgroup.

Joss-Hua commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacebtivzrbhgqb5xltxczec2oftrzknwlzvgfzzfa4vj4bxd3swf334

Address

f3rm7hrijz2l3qth6nihxlf2no3so55vsb2dwcplwxwm7e6szwlx3on7fjerzomdinuitldmyngwlzxtz33vnq

Datacap Allocated

50.00TiB

Signer Address

f1tfg54zzscugttejv336vivknmsnzzmyudp3t7wi

Id

8a0414b2-514c-42e0-b788-dba94b298a27

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebtivzrbhgqb5xltxczec2oftrzknwlzvgfzzfa4vj4bxd3swf334

herrehesse commented 1 year ago

Dear Applicant,

Due to the increased amount of erroneous/wrong Filecoin+ data recently, on behalf of the entire community, we feel compelled to go deeper into datacap requests. Hereby to ensure that the overall value of the Filecoin network and Filecoin+ program increases and is not abused.

Please answer the questions below as comprehensively as possible.

Customer data

We expect that for the onboarding of customers with the scale of an LDN there would have been at least multiple email and perhaps several chat conversations preceding it. A single email with an agreement does not qualify here.

Should this only be soley for acquiring datacap this is of course out of the question. The customer must have a legitimate reason for wanting to use the Filecoin+ program which is intended as a program to store useful and public datasets on the network.

(As an intermediate solution Filecoin offers the FIL-E program or the glif.io website for business datasets that do not meet the requirements for a Filecoin+ dataset)

Files and Processing

Hopefully you understand the caution the overall community has for onboarding the wrong data. We understand the increased need for Filecoin+, however, we must not allow the program to be misused. Everything depends on a valuable and useful network, let's do our best to make this happen. Together.

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 2

Multisig Notary address

f02049625

Client address

f3rm7hrijz2l3qth6nihxlf2no3so55vsb2dwcplwxwm7e6szwlx3on7fjerzomdinuitldmyngwlzxtz33vnq

DataCap allocation requested

100TiB

Id

647d4b41-7ff6-4cf2-af74-030ec2fe8732

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f01858410

Client address

f3rm7hrijz2l3qth6nihxlf2no3so55vsb2dwcplwxwm7e6szwlx3on7fjerzomdinuitldmyngwlzxtz33vnq

Last two approvers

Joss-Hua & newwebgroup

Rule to calculate the allocation request amount

100% of weekly dc amount requested

DataCap allocation requested

100TiB

Total DataCap granted for client so far

50TiB

Datacap to be granted to reach the total amount requested by the client (5PiB)

4.95PiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
0 0 50TiB NaN 64GiB
filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report[^1]

No active deals found for this client.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

cryptowhizzard commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report[^1]

Storage Provider Distribution

The below table shows the distribution of storage providers that have stored data for this client.

If this is the first time a provider takes verified deal, it will be marked as new.

For most of the datacap application, below restrictions should apply.

Since this is the 3rd allocation, the following restrictions have been relaxed:

✔️ Storage provider distribution looks healthy.

Provider Location Total Deals Sealed Percentage Unique Data Duplicate Deals
f0867300 Tokyo, Tokyo, JP
Alibaba (US) Technology Co., Ltd.		 12.48 TiB 25.22% 12.48 TiB 0.00%
f01228000 Seoul, Seoul, KR
Alibaba (US) Technology Co., Ltd.		 12.45 TiB 25.16% 12.45 TiB 0.00%
f01228008 Sydney, New South Wales, AU
Alibaba (US) Technology Co., Ltd.		 12.30 TiB 24.84% 12.30 TiB 0.00%
f0522948 Singapore, Singapore, SG
Alibaba (US) Technology Co., Ltd.		 12.27 TiB 24.78% 12.27 TiB 0.00%

Provider Distribution

Deal Data Replication

The below table shows how each many unique data are replicated across storage providers.

Since this is the 3rd allocation, the following restrictions have been relaxed:

✔️ Data replication looks healthy.

Unique Data Size Total Deals Made Number of Providers Deal Percentage
448.00 GiB 1.31 TiB 3 2.65%
12.05 TiB 48.19 TiB 4 97.35%

Replication Distribution

Deal Data Shared with other Clients

The below table shows how many unique data are shared with other clients. Usually different applications owns different data and should not resolve to the same CID.

However, this could be possible if all below clients use same software to prepare for the exact same dataset or they belong to a series of LDN applications for the same dataset.

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

C00kies77 commented 1 year ago

Dear @smallbull checking retrievals, we will come back to you shortly

Normalnoise commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the full report.

Joss-Hua commented 1 year ago

image I am ready to support it, but there was an error retrieving the data. Please take a look at what happened.

smallbull commented 1 year ago

image I am ready to support it, but there was an error retrieving the data. Please take a look at what happened.

It support retrieve data. Please try again.

image image
Joss-Hua commented 1 year ago

Good job. I will support it in the next round because I have already signed in the previous round.

Fatman13 commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the full report.

Fatman13 commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacebjwekknnyjzajkl5q2coh23hzizjowumqvbavtn2wtxk6qzxmu6y

Address

f3rm7hrijz2l3qth6nihxlf2no3so55vsb2dwcplwxwm7e6szwlx3on7fjerzomdinuitldmyngwlzxtz33vnq

Datacap Allocated

100.00TiB

Signer Address

f1j3u7crhjzwb2cj5mq7vodlt4o66yoyci7lhcauy

Id

647d4b41-7ff6-4cf2-af74-030ec2fe8732

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebjwekknnyjzajkl5q2coh23hzizjowumqvbavtn2wtxk6qzxmu6y

Fatman13 commented 1 year ago

Reached out by the client on Slack. CIDchecker looks good.

igoovo commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the full report.

igoovo commented 1 year ago

Everything looks fine. WechatIMG14

igoovo commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacebgompza77avdfwqsarv7chllkmqxur4e7okwqeiem7sgr3xs4x3q

Address

f3rm7hrijz2l3qth6nihxlf2no3so55vsb2dwcplwxwm7e6szwlx3on7fjerzomdinuitldmyngwlzxtz33vnq

Datacap Allocated

100.00TiB

Signer Address

f1shnsfayxqll77svffaxnjenms7bbbysbqcatrpy

Id

647d4b41-7ff6-4cf2-af74-030ec2fe8732

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebgompza77avdfwqsarv7chllkmqxur4e7okwqeiem7sgr3xs4x3q

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 3

Multisig Notary address

f02049625

Client address

f3rm7hrijz2l3qth6nihxlf2no3so55vsb2dwcplwxwm7e6szwlx3on7fjerzomdinuitldmyngwlzxtz33vnq

DataCap allocation requested

200TiB

Id

05b3a1a0-d8d7-43f3-b558-7d3c64c9425d

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f02049625

Client address

f3rm7hrijz2l3qth6nihxlf2no3so55vsb2dwcplwxwm7e6szwlx3on7fjerzomdinuitldmyngwlzxtz33vnq

Rule to calculate the allocation request amount

200% of weekly dc amount requested

DataCap allocation requested

200TiB

Total DataCap granted for client so far

9094.9YiB

Datacap to be granted to reach the total amount requested by the client (5PiB)

9094.9YiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
1858 3 100TiB 37.68 15.12TiB
github-actions[bot] commented 12 months ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

github-actions[bot] commented 11 months ago

This application has not seen any responses in the last 14 days, so for now it is being closed. Please feel free to contact the Fil+ Gov team to re-open the application if it is still being processed. Thank you!

large-datacap-requests[bot] commented 5 months ago

Thanks for your request! :exclamation: We have found some problems in the information provided. We could not find Website \/ Social Media field in the information provided We could not find Total amount of DataCap being requested (between 500 TiB and 5 PiB) field in the information provided We could not find Weekly allocation of DataCap requested (usually between 1-100TiB) field in the information provided We could not find On-chain address for first allocation field in the information provided We could not find Data Type of Application field in the information provided

Please, take a look at the request and edit the body of the issue providing all the required information.
large-datacap-requests[bot] commented 4 months ago

RootKeyHolders have approved multisig account. You can now request first datacap release