search

filecoin-project / filecoin-plus-large-datasets

Hub for client applications for DataCap at a large scale
110 stars 62 forks source link

[DataCap Application] MinerPlus - Mining Dataset #1840

Open KizenYang opened 1 year ago

KizenYang commented 1 year ago

Data Owner Name

MinerPlus

Data Owner Country/Region

Singapore

Data Owner Industry

Web3 / Crypto

Website

https://www.bitdeer.com/minerplus

Social Media

https://twitter.com/BitdeerOfficial
https://twitter.com/VIP_MinerPlus
https://twitter.com/VIP_MinerPlus/status/1479407700437995521?s=20

Total amount of DataCap being requested

5PiB

Weekly allocation of DataCap requested

100TiB

On-chain address for first allocation

f1lfep3skcfkahiabbjq7atktkwfzqrcwpxrnbyni

Data Type of Application

Public, Open Commercial/Enterprise

Custom multisig

Identifier

No response

Share a brief history of your project and organization

The Smart Solution Provider For Digital Assets Mining.
MinerPlus provides a one-stop management system for ASIC and GPU miners.
By the end of 2022, the team has over 600,000 miners under our management, and had a hosting capacity of 140 MW in global network of mining facilities.

Is this project associated with other projects/ecosystem stakeholders?

Yes

If answered yes, what are the other projects/ecosystem stakeholders

Own funds and revenue of the company.
Bitdeer related teams.

Describe the data being stored onto Filecoin

A large number of logs generated by the management system and self-operated mining machines.
These data does not involve the company's sensitive information. Total size: At least 5 PiB now.

Where was the data currently stored in this dataset sourced from

AWS Cloud

If you answered "Other" in the previous question, enter the details here

No response

How do you plan to prepare the dataset

IPFS, lotus

If you answered "other/custom tool" in the previous question, enter the details here

No response

Please share a sample of the data

Link: https://pan.baidu.com/s/1c2mALWySiigsqkSnoW0wRw?pwd=pb4v
Extraction code: pb4v

Confirm that this is a public dataset that can be retrieved by anyone on the Network

If you chose not to confirm, what was the reason

No response

What is the expected retrieval frequency for this data

Yearly

For how long do you plan to keep this dataset stored on Filecoin

1.5 to 2 years

In which geographies do you plan on making storage deals

Asia other than Greater China

How will you be distributing your data to storage providers

IPFS

How do you plan to choose storage providers

Partners

If you answered "Others" in the previous question, what is the tool or platform you plan to use

No response

If you already have a list of storage providers to work with, fill out their names and provider IDs below

No response

How do you plan to make deals to your storage providers

Lotus client

If you answered "Others/custom tool" in the previous question, enter the details here

No response

Can you confirm that you will follow the Fil+ guideline

Yes

large-datacap-requests[bot] commented 1 year ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

large-datacap-requests[bot] commented 1 year ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

Sunnyiscoming commented 1 year ago

Can you provide more data samples or informations to prove that you have 5 PB data already?

Sunnyiscoming commented 1 year ago

Any update here?

KizenYang commented 1 year ago

Can you provide more data samples or informations to prove that you have 5 PB data already?

Most of our data has been stored on AWS S3. Screenshots only capture a part of it.

image

Sunnyiscoming commented 1 year ago

Can you provide more data samples not just screenshot here?

Sunnyiscoming commented 1 year ago

Any update here?

KizenYang commented 1 year ago

I have made a part of the data sample stored on S3 publicly available for download to your local machine. Large-scale downloads may incur some traffic costs on S3 storage. The public availability is temporarily set for 7 days to facilitate community review. Public address: https://public-sharing-mining-data.s3.us-west-2.amazonaws.com/

You can copy the file name between the keys and then paste it directly after the address to download. For example,

miner_data_2023_3_21_0/part-00000-3f17a10e-3034-408a-813d-2a10af2852e1-c000.csv

The corresponding download address is: https://public-sharing-mining-data.s3.us-west-2.amazonaws.com/miner_data_2023_3_21_0/part-00000-3f17a10e-3034-408a-813d-2a10af2852e1-c000.csv

image
Sunnyiscoming commented 1 year ago

Datacap Request Trigger

Total DataCap requested

5PiB

Expected weekly DataCap usage rate

100TiB

Client address

f1lfep3skcfkahiabbjq7atktkwfzqrcwpxrnbyni

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Multisig Notary address

f02049625

Client address

f1lfep3skcfkahiabbjq7atktkwfzqrcwpxrnbyni

DataCap allocation requested

50TiB

Id

dddf6de0-44bf-494c-a375-750a5ff846e1

kernelogic commented 1 year ago

The sampling public bucket is no longer accessible. Also I think you need to do a KYC to prove you are associated with bitdeer

aws s3 ls --no-sign-request s3://public-sharing-mining-data
An error occurred (AccessDenied) when calling the ListObjectsV2 operation: Access Denied

KizenYang commented 1 year ago

The sampling public bucket is no longer accessible. Also I think you need to do a KYC to prove you are associated with bitdeer

aws s3 ls --no-sign-request s3://public-sharing-mining-data An error occurred (AccessDenied) when calling the ListObjectsV2 operation: Access Denied

We are very sorry that we temporarily closed the access to s3 before. We now have s3 open and ready to access. In addition, for kyc, how should we provide it? Should we send it to the email? Or how should we prove it?

kernelogic commented 1 year ago
image
kernelogic commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzaced74rbjxphj3drafy5g2suffazxsenjqn3xywakm2m7l7cwn57646

Address

f1lfep3skcfkahiabbjq7atktkwfzqrcwpxrnbyni

Datacap Allocated

50.00TiB

Signer Address

f1yjhnsoga2ccnepb7t3p3ov5fzom3syhsuinxexa

Id

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaced74rbjxphj3drafy5g2suffazxsenjqn3xywakm2m7l7cwn57646

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report[^1]

No application info found for this issue on https://filplus.d.interplanetary.one/clients.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

KizenYang commented 1 year ago

Dear @Sunnyiscoming Do we need to do a KYC ？If so, how?

Joss-Hua commented 1 year ago

Hi, @KizenYang you could send a confirmation email to filplus-app-review@fil.org using the email address of your company domain. The email name should include the issue id #1840. And @Sunnyiscoming  please help verify and confirm.

KizenYang commented 1 year ago

Hi, @KizenYang you could send a confirmation email to filplus-app-review@fil.org using the email address of your company domain. The email name should include the issue id #1840. And @Sunnyiscoming  please help verify and confirm.

Hi @Joss-Hua
I have sent an email to filplus-app-review@fil.org, please check.

image
KizenYang commented 1 year ago

Hi,team I have resent an email to filplus-app-review@fil.org, please help verify and confirm. @galen-mcandrew @raghavrmadya @Kevin-FF-USA @Sunnyiscoming

jomozz commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacedry6jbob4pnicqasmwhd3yuwesfcvaqo5s55ookm5q2cg5gaydgi

Address

f1lfep3skcfkahiabbjq7atktkwfzqrcwpxrnbyni

Datacap Allocated

50.00TiB

Signer Address

f1qpdwt2f2fse74nl5wi27uk5jcyqqqyil47yqt7i

Id

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedry6jbob4pnicqasmwhd3yuwesfcvaqo5s55ookm5q2cg5gaydgi

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 14 days, so for now it is being closed. Please feel free to re-open if this is relevant, or start a new application for DataCap anytime. Thank you!

KizenYang commented 1 year ago

Hello! We have encountered an issue with our DC application. We tried to submit an issue on GitHub, but it was closed by a robot, which prevented us from allocating more DC space. The DC space we previously applied for has already been used up.

https://datacapstats.io/clients?filter=f1lfep3skcfkahiabbjq7atktkwfzqrcwpxrnbyni

Would you please help us resolve this issue? Thank you! @galen-mcandrew @raghavrmadya @Kevin-FF-USA @Sunnyiscoming @simonkim0515

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 2

Multisig Notary address

f02049625

Client address

f1lfep3skcfkahiabbjq7atktkwfzqrcwpxrnbyni

DataCap allocation requested

100TiB

Id

bb9be259-8b6f-4929-ad8d-44d9a30f773a

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f02049625

Client address

f1lfep3skcfkahiabbjq7atktkwfzqrcwpxrnbyni

Rule to calculate the allocation request amount

100% of weekly dc amount requested

DataCap allocation requested

100TiB

Total DataCap granted for client so far

50TiB

Datacap to be granted to reach the total amount requested by the client (5PiB)

4.95PiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
1600 6 50TiB 26.5 0B
Joss-Hua commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval report.

NiwanDao commented 1 year ago

The report result looks good to me, willing to support it.

NiwanDao commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacecn4wragpeke5bolyhujzvq4do7r7jeltgwdooq52ac77yhvc5dl2

Address

f1lfep3skcfkahiabbjq7atktkwfzqrcwpxrnbyni

Datacap Allocated

100.00TiB

Signer Address

f1a2lia2cwwekeubwo4nppt4v4vebxs2frozarz3q

Id

bb9be259-8b6f-4929-ad8d-44d9a30f773a

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacecn4wragpeke5bolyhujzvq4do7r7jeltgwdooq52ac77yhvc5dl2

spaceT9 commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval report.

Joss-Hua commented 1 year ago

Currently very healthy

Joss-Hua commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacebswq22jb6typzhjuifx6gicvppy467uj3pcubdzc7sbm5ieidaqq

Address

f1lfep3skcfkahiabbjq7atktkwfzqrcwpxrnbyni

Datacap Allocated

100.00TiB

Signer Address

f1tfg54zzscugttejv336vivknmsnzzmyudp3t7wi

Id

bb9be259-8b6f-4929-ad8d-44d9a30f773a

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebswq22jb6typzhjuifx6gicvppy467uj3pcubdzc7sbm5ieidaqq

BobbyChoii commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacebc3golriv2f62ubpy6ga2p5wudbj55jazriyytdrercnx3wud7ga

Address

f1lfep3skcfkahiabbjq7atktkwfzqrcwpxrnbyni

Datacap Allocated

100.00TiB

Signer Address

f1irqs2gmctiv3jcdfwuch7oxvf4ixh3k4b2wc24i

Id

bb9be259-8b6f-4929-ad8d-44d9a30f773a

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebc3golriv2f62ubpy6ga2p5wudbj55jazriyytdrercnx3wud7ga

BobbyChoii commented 1 year ago

Report looks healthy，willing to support.

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

KizenYang commented 1 year ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

Please keep this application open 😁

Joss-Hua commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

MatrixStorage commented 1 year ago

Report looks healthy，willing to support.

MatrixStorage commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacebuq2hg52zhdafedmxkged2tkdd4psqz4fufe56kpldtss7grgcmi

Address

f1lfep3skcfkahiabbjq7atktkwfzqrcwpxrnbyni

Datacap Allocated

100.00TiB

Signer Address

f1tbxqwjxfyv7swsdin4einirlsfquv3vnmlapley

Id

bb9be259-8b6f-4929-ad8d-44d9a30f773a

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebuq2hg52zhdafedmxkged2tkdd4psqz4fufe56kpldtss7grgcmi

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

KizenYang commented 1 year ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

Please keep this application open 😁

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

KizenYang commented 1 year ago

Please keep this application open

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 4

Multisig Notary address

f02049625

Client address

f1lfep3skcfkahiabbjq7atktkwfzqrcwpxrnbyni

DataCap allocation requested

400TiB

Id

e1f185ad-6bd2-416c-90b2-728c7d3640f4

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f02049625

Client address

f1lfep3skcfkahiabbjq7atktkwfzqrcwpxrnbyni

Rule to calculate the allocation request amount

400% of weekly dc amount requested

DataCap allocation requested

400TiB

Total DataCap granted for client so far

9094947017729291264.0YiB

Datacap to be granted to reach the total amount requested by the client (5PiB)

9094947017729291264.0YiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
4143 7 100TiB 39.56 120.53TiB
Aaron01230 commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

⚠️ 63.80% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

Aaron01230 commented 1 year ago

The report shows that 63.80% of deals are for data replicated across less than 4 storage providers, and there are 7 nodes deal with this LDN, are you planning to improve the data replication next round ?

KizenYang commented 1 year ago

The report shows that 63.80% of deals are for data replicated across less than 4 storage providers, and there are 7 nodes deal with this LDN, are you planning to improve the data replication next round ?

Based on the results of the report, we have noticed the situation of low data redundancy, where data is stored with fewer than 4 providers. We take this issue very seriously and plan to take measures to address it in the next round of improvements to enhance data redundancy and network reliability. We will strive to ensure that we meet this standard.