filecoin-project / filecoin-plus-large-datasets

Hub for client applications for DataCap at a large scale
109 stars 62 forks source link

Dcent - Help Ukrainian miners - Part 2 #540

Closed cryptowhizzard closed 1 year ago

cryptowhizzard commented 2 years ago

Large Dataset Notary Application

To apply for DataCap to onboard your dataset to Filecoin, please fill out the following.

Core Information

Please respond to the questions below by replacing the text saying "Please answer here". Include as much detail as you can in your answer.

Project details

Share a brief history of your project and organization.

Speedium / Dcent has engaged in Slingshot starting 2.6. We have successfully stored more than 15 different datasets with 20+ different miners.

What is the primary source of funding for this project?

Company account

What other projects/ecosystem stakeholders is this project associated with?

None.

Use-case details

Describe the data being stored onto Filecoin

We have been contacted by miners from Ukraine to help them make a backup of their data in case anything happens given the situation in their country. The data will consists of sectors of the miners involved. The batch will be 500 TB we intend and we intend to spread it over 4 to 6 SP's.

This issue is a followup to issue #348. 

Set 1 contais 400 TB of sectors and cache data. We have packed it and uploaded it. In the mean time we received another 400 TiB of data for this LDN.

Where was the data in this dataset sourced from?

The miners uploaded it towards us.

Can you share a sample of the data? A link to a file, an image, a table, etc., are good ways to do this.

Yes, but we prefer to keep that private until uploaded for security reasons.

Confirm that this is a public dataset that can be retrieved by anyone on the Network (i.e., no specific permissions or access rights are required to view the data).

Yes.

What is the expected retrieval frequency for this data?

Just a few times for checking but we hope that it won't be necessary  offcourse.

For how long do you plan to keep this dataset stored on Filecoin?

18 months or longer

DataCap allocation plan

In which geographies (countries, regions) do you plan on making storage deals?

EU / China / US / Australia.

How will you be distributing your data to storage providers? Is there an offline data transfer process?

The data will be transferred both offline and online.

How do you plan on choosing the storage providers with whom you will be making deals? This should include a plan to ensure the data is retrievable in the future both by you and others.

We have a few providers who have been working with us during Slingshot Restore program and we'd like to continue working with them for ongoing Slingshot competition.

How will you be distributing deals across storage providers?

Max 2 copy's per storage provider if stored on different miners / locations.

Do you have the resources/funding to start making deals as soon as you receive DataCap? What support from the community would help you onboard onto Filecoin?

Yes, we have the resources.
large-datacap-requests[bot] commented 2 years ago

Thanks for your request!

Heads up, you’re requesting more than the typical weekly onboarding rate of DataCap!
large-datacap-requests[bot] commented 2 years ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

large-datacap-requests[bot] commented 2 years ago

Thanks for your request!

Heads up, you’re requesting more than the typical weekly onboarding rate of DataCap!
large-datacap-requests[bot] commented 2 years ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

large-datacap-requests[bot] commented 2 years ago

Thanks for your request!

Heads up, you’re requesting more than the typical weekly onboarding rate of DataCap!
large-datacap-requests[bot] commented 2 years ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

kevzak commented 2 years ago

@raghavrmadya

Sunnyiscoming commented 2 years ago

Can you share a sample of the data? A link to a file, an image, a table, etc., are good ways to do this. Yes, but we prefer to keep that private until uploaded for security reasons.

Hi, @cryptowhizzard As you said, this dataset is more suitable for the private enterprise data request pattern.

As Deep mentioned filecoin-project/notary-governance#514, you will need to find notaries that can support your application since it includes private enterprise data.

You will need to follow a process similar to filecoin-project/notary-governance#489. This means:

Find notaries that can support your application. You can do this best in Slack, #fil-plus or #fil-plus-application-review Once you have them, make an issue in the Notary Governance repo, similar to filecoin-project/notary-governance#489, stating your plan and the notaries that can support your request

Sunnyiscoming commented 2 years ago

https://www.linkedin.com/dcent This link can not be found.

cryptowhizzard commented 2 years ago

https://www.linkedin.com/dcent This link can not be found.

I will page Hidde for this

cryptowhizzard commented 2 years ago

https://www.linkedin.com/company/dcent-b-v/

raghavrmadya commented 2 years ago

Datacap Request Trigger

Total DataCap requested

5 PiB

Expected weekly DataCap usage rate

300 TiB

Client address

f15cps7yo2x4fosvp45opfihk3a4wg2qukmwyet7a

large-datacap-requests[bot] commented 2 years ago

DataCap Allocation requested

Multisig Notary address

f01858410

Client address

f15cps7yo2x4fosvp45opfihk3a4wg2qukmwyet7a

DataCap allocation requested

150TiB

psh0691 commented 2 years ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacedsti53tip3aj6hyehffxdlw5vti3z4jqngxyfpvdxnfykdypd2cg

Address

f15cps7yo2x4fosvp45opfihk3a4wg2qukmwyet7a

Datacap Allocated

150.00TiB

Signer Address

f1qdko4jg25vo35qmyvcrw4ak4fmuu3f5rif2kc7i

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedsti53tip3aj6hyehffxdlw5vti3z4jqngxyfpvdxnfykdypd2cg

dannyob commented 2 years ago

Hey @cryptowhizzard , I'm willing to consider this for approval (at least until we get the Enterprise program up and running). Would you be prepared to share details with me of the clients and the anticipated data?

cryptowhizzard commented 2 years ago

Yes , please contact me on slack for this.

s0nik42 commented 2 years ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaceasqlaf3dbiywtisywi3nezth6sw6i3fr44y23mgaedrtlrgi6t4u

Address

f15cps7yo2x4fosvp45opfihk3a4wg2qukmwyet7a

Datacap Allocated

150.00TiB

Signer Address

f1wxhnytjmklj2czezaqcfl7eb4nkgmaxysnegwii

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceasqlaf3dbiywtisywi3nezth6sw6i3fr44y23mgaedrtlrgi6t4u

large-datacap-requests[bot] commented 2 years ago

DataCap Allocation requested

Request number 2

Multisig Notary address

f01858410

Client address

f15cps7yo2x4fosvp45opfihk3a4wg2qukmwyet7a

DataCap allocation requested

300TiB

large-datacap-requests[bot] commented 2 years ago

Stats & Info for DataCap Allocation

Multisig Notary address

f01858410

Client address

f15cps7yo2x4fosvp45opfihk3a4wg2qukmwyet7a

Last two approvers

s0nik42 & psh0691

Rule to calculate the allocation request amount

100% of weekly dc amount requested

DataCap allocation requested

300TiB

Total DataCap granted for client so far

3.99PiB

Datacap to be granted to reach the total amount requested by the client (5 PiB)

1.00PiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
227633 28 150TiB 18.12 727.91GiB
Reiers commented 2 years ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacedlyszuasi4o3jokvry4tiytypt2huqjxo4ositbi3oh37ojtmvj4

Address

f15cps7yo2x4fosvp45opfihk3a4wg2qukmwyet7a

Datacap Allocated

300.00TiB

Signer Address

f1oz43ckvmtxmmsfzqm6bpnemqlavz4ifyl524chq

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedlyszuasi4o3jokvry4tiytypt2huqjxo4ositbi3oh37ojtmvj4

kernelogic commented 2 years ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacecz3ujfpyfm5oqw55xpppbjnuis7zjqqgav47ructu2d2zlkr45ac

Address

f15cps7yo2x4fosvp45opfihk3a4wg2qukmwyet7a

Datacap Allocated

300.00TiB

Signer Address

f1yjhnsoga2ccnepb7t3p3ov5fzom3syhsuinxexa

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacecz3ujfpyfm5oqw55xpppbjnuis7zjqqgav47ructu2d2zlkr45ac

large-datacap-requests[bot] commented 2 years ago

DataCap Allocation requested

Request number 3

Multisig Notary address

f01858410

Client address

f15cps7yo2x4fosvp45opfihk3a4wg2qukmwyet7a

DataCap allocation requested

600TiB

Id

08798a40-b9d1-40f0-9102-07bf2e20e67c

large-datacap-requests[bot] commented 2 years ago

Stats & Info for DataCap Allocation

Multisig Notary address

f01858410

Client address

f15cps7yo2x4fosvp45opfihk3a4wg2qukmwyet7a

Last two approvers

kernelogic & Reiers

Rule to calculate the allocation request amount

200% of weekly dc amount requested

DataCap allocation requested

600TiB

Total DataCap granted for client so far

4.29PiB

Datacap to be granted to reach the total amount requested by the client (5 PiB)

726.96TiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
240448 29 300TiB 17.24 74.62TiB
kernelogic commented 2 years ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacec43ci5n5u6bbsauancevwxorbdrvq3a5ef7uc4hjnvjazqwhzmiw

Address

f15cps7yo2x4fosvp45opfihk3a4wg2qukmwyet7a

Datacap Allocated

600.00TiB

Signer Address

f1yjhnsoga2ccnepb7t3p3ov5fzom3syhsuinxexa

Id

08798a40-b9d1-40f0-9102-07bf2e20e67c

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacec43ci5n5u6bbsauancevwxorbdrvq3a5ef7uc4hjnvjazqwhzmiw

liyunzhi-666 commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacedidzycmyxlgqj3dyi2g4pfhxb3iowmd5bh6zmxybkx6e2ynbqqiw

Address

f15cps7yo2x4fosvp45opfihk3a4wg2qukmwyet7a

Datacap Allocated

600.00TiB

Signer Address

f1pszcrsciyixyuxxukkvtazcokexbn54amf7gvoq

Id

08798a40-b9d1-40f0-9102-07bf2e20e67c

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedidzycmyxlgqj3dyi2g4pfhxb3iowmd5bh6zmxybkx6e2ynbqqiw

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f01858410

Client address

f15cps7yo2x4fosvp45opfihk3a4wg2qukmwyet7a

Last two approvers

liyunzhi-666 & kernelogic

Rule to calculate the allocation request amount

400% of weekly dc amount requested

DataCap allocation requested

1.17PiB

Total DataCap granted for client so far

4.87PiB

Datacap to be granted to reach the total amount requested by the client (5 PiB)

126.96TiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
255432 31 600TiB 16.13 146.62TiB
mjroddy commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzaceafy7hpgrumcylonjndoni5x7bcujszpnvxey4cngyufarbxyhbow

Address

f15cps7yo2x4fosvp45opfihk3a4wg2qukmwyet7a

Datacap Allocated

1.17PiB

Signer Address

f1ystxl2ootvpirpa7ebgwl7vlhwkbx2r4zjxwe5i

Id

23cd768e-7ca0-4ba0-b50b-c26a0940a7c8

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceafy7hpgrumcylonjndoni5x7bcujszpnvxey4cngyufarbxyhbow

xinaxu commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacebzk5b4bu5eiwqy7ijrtb4yroy6w6nmes446fxhbpkuhi3ue2seb6

Address

f15cps7yo2x4fosvp45opfihk3a4wg2qukmwyet7a

Datacap Allocated

1.17PiB

Signer Address

f1k3ysofkrrmqcot6fkx4wnezpczlltpirmrpsgui

Id

23cd768e-7ca0-4ba0-b50b-c26a0940a7c8

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebzk5b4bu5eiwqy7ijrtb4yroy6w6nmes446fxhbpkuhi3ue2seb6

cryptowhizzard commented 1 year ago

Hello,

This was approved while the faucet was empty. Can someone reset it so we can get the tranche of 1.17 PB approved?

raghavrmadya commented 1 year ago

DataCap Allocation requested

Request number 4

Multisig Notary address

f01858410

Client address

f15cps7yo2x4fosvp45opfihk3a4wg2qukmwyet7a

DataCap allocation requested

1.17PiB

raghavrmadya commented 1 year ago

Checked and confirmed the error recognized by the client did happen. Retriggering request #4 manually. @mjroddy @xinaxu , please sign again if you still support

xinaxu commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzaceblnu3zo2xiipfuctr5t7d7kfxxb4ybqlevzt6mqws6szikoy4hei

Address

f15cps7yo2x4fosvp45opfihk3a4wg2qukmwyet7a

Datacap Allocated

1.17PiB

Signer Address

f1k3ysofkrrmqcot6fkx4wnezpczlltpirmrpsgui

Id

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceblnu3zo2xiipfuctr5t7d7kfxxb4ybqlevzt6mqws6szikoy4hei

mjroddy commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacec6kmbpyb5ifbcdly4qppwrrjow5mwgdkrwrjgqysovolf5luj75s

Address

f15cps7yo2x4fosvp45opfihk3a4wg2qukmwyet7a

Datacap Allocated

1.17PiB

Signer Address

f1ystxl2ootvpirpa7ebgwl7vlhwkbx2r4zjxwe5i

Id

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacec6kmbpyb5ifbcdly4qppwrrjow5mwgdkrwrjgqysovolf5luj75s

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 5

Multisig Notary address

f01858410

Client address

f15cps7yo2x4fosvp45opfihk3a4wg2qukmwyet7a

DataCap allocation requested

2.34PiB

Id

989ad6c3-e1be-40bb-b021-66ee31df599e

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f01858410

Client address

f15cps7yo2x4fosvp45opfihk3a4wg2qukmwyet7a

Last two approvers

megtei & xinaxu

Rule to calculate the allocation request amount

800% of weekly dc amount requested

DataCap allocation requested

2.34PiB

Total DataCap granted for client so far

4.87PiB

Datacap to be granted to reach the total amount requested by the client (5 PiB)

126.96TiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
261344 31 1.17PiB 15.71 1.68GiB
mjroddy commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacedb2u43vcdfmysfh5fql56g7cwtz6o6sxnqbqbgzod4ptoyfnycfu

Address

f15cps7yo2x4fosvp45opfihk3a4wg2qukmwyet7a

Datacap Allocated

2.34PiB

Signer Address

f1ystxl2ootvpirpa7ebgwl7vlhwkbx2r4zjxwe5i

Id

989ad6c3-e1be-40bb-b021-66ee31df599e

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedb2u43vcdfmysfh5fql56g7cwtz6o6sxnqbqbgzod4ptoyfnycfu

cryptowhizzard commented 1 year ago

@raghavrmadya

Can you please recheck this?

Seems the datacap still was not applied.

kernelogic commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacedro4yfggqynlazznr6c2s6zxq2eeqrikz65aqqwqzasstbscv7vg

Address

f15cps7yo2x4fosvp45opfihk3a4wg2qukmwyet7a

Datacap Allocated

2.34PiB

Signer Address

f1yjhnsoga2ccnepb7t3p3ov5fzom3syhsuinxexa

Id

989ad6c3-e1be-40bb-b021-66ee31df599e

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedro4yfggqynlazznr6c2s6zxq2eeqrikz65aqqwqzasstbscv7vg

filplus-checker commented 1 year ago

DataCap and CID Checker Report[^1]

If this is the first time a provider takes verified deal, it will be marked as new.

For most of the datacap application, below restrictions should apply.

⚠️ f01886710 has unknown IP location.

⚠️ f01923553 has unknown IP location.

⚠️ f01848694 has unknown IP location.

Provider Location Total Deals Sealed Percentage Unique Data Duplicate Deals
f01928520 Maywood Park, Oregon, US 715.13 TiB 12.76% 715.13 TiB 0.00%
f01882184 Singapore, Singapore, SG 543.26 TiB 9.69% 543.26 TiB 0.00%
f01886710 Unknown 493.90 TiB 8.81% 468.30 TiB 5.18%
f01345523 Antwerpen, Flanders, BE 381.28 TiB 6.80% 381.25 TiB 0.01%
f01392893 Amsterdam, North Holland, NL 375.23 TiB 6.70% 374.25 TiB 0.26%
f01385207 Lincoln, Nebraska, US 357.92 TiB 6.39% 357.23 TiB 0.19%
f01877571 Singapore, Singapore, SG 343.21 TiB 6.12% 343.21 TiB 0.00%
f01953985new Shenzhen, Guangdong, CN 304.68 TiB 5.44% 304.40 TiB 0.09%
f01652333 Sunnyvale, California, US 215.93 TiB 3.85% 201.04 TiB 6.90%
f01880047 Singapore, Singapore, SG 192.55 TiB 3.44% 192.55 TiB 0.00%
f01915033 Chengdu, Sichuan, CN 178.22 TiB 3.18% 177.94 TiB 0.16%
f022352 Oslo, Oslo, NO 165.48 TiB 2.95% 165.04 TiB 0.26%
f01923553new Unknown 150.03 TiB 2.68% 150.00 TiB 0.02%
f0117434new Chengdu, Sichuan, CN 134.07 TiB 2.39% 133.94 TiB 0.10%
f01208803 Sydney, New South Wales, AU 133.80 TiB 2.39% 122.01 TiB 8.82%
f01929568new Stockholm, Stockholm, SE 129.50 TiB 2.31% 129.47 TiB 0.02%
f01208189 Sydney, New South Wales, AU 102.87 TiB 1.84% 102.44 TiB 0.41%
f01829137 New Albany, Ohio, US 96.65 TiB 1.72% 96.65 TiB 0.00%
f01208154 Sydney, New South Wales, AU 87.68 TiB 1.56% 72.47 TiB 17.35%
f01886704 Las Vegas, Nevada, US 79.07 TiB 1.41% 79.07 TiB 0.00%
f01886690 Las Vegas, Nevada, US 74.67 TiB 1.33% 74.67 TiB 0.00%
f01208632 Sydney, New South Wales, AU 53.04 TiB 0.95% 52.44 TiB 1.15%
f01882569 Atlanta, Georgia, US 52.83 TiB 0.94% 52.82 TiB 0.01%
f01157271 Sydney, New South Wales, AU 32.76 TiB 0.58% 32.29 TiB 1.43%
f01732189 San Diego, California, US 25.40 TiB 0.45% 25.15 TiB 0.98%
f01206408 Sydney, New South Wales, AU 23.32 TiB 0.42% 23.29 TiB 0.13%
f01157249 Sydney, New South Wales, AU 21.22 TiB 0.38% 21.22 TiB 0.00%
f01157027 Sydney, New South Wales, AU 21.06 TiB 0.38% 21.03 TiB 0.15%
f01156975 Sydney, New South Wales, AU 20.11 TiB 0.36% 18.05 TiB 10.25%
f01157018 Sydney, New South Wales, AU 19.94 TiB 0.36% 19.79 TiB 0.74%
f01848694 Unknown 17.93 TiB 0.32% 17.93 TiB 0.00%
f01156901 Sydney, New South Wales, AU 16.39 TiB 0.29% 16.33 TiB 0.33%
f01207874 Sydney, New South Wales, AU 11.38 TiB 0.20% 11.35 TiB 0.27%
f01207954 Sydney, New South Wales, AU 10.45 TiB 0.19% 10.45 TiB 0.04%
f01208042 Sydney, New South Wales, AU 8.73 TiB 0.16% 8.66 TiB 0.76%
f01127678 Łódź, Łódź Voivodeship, PL 8.54 TiB 0.15% 8.54 TiB 0.00%
f01889910 Phoenix, Arizona, US 4.12 TiB 0.07% 4.12 TiB 0.00%
f01154023 Sydney, New South Wales, AU 960.51 GiB 0.02% 960.51 GiB 0.00%
f01156538 Sydney, New South Wales, AU 256.00 GiB 0.00% 256.00 GiB 0.00%
f01199430 Heerhugowaard, North Holland, NL 96.00 GiB 0.00% 96.00 GiB 0.00%

Provider Distribution

Deal Data Replication

The below table shows how each many unique data are replicated across storage providers.

⚠️ 26.21% of deals are for data replicated across less than 4 storage providers.

Unique Data Size Total Deals Made Number of Providers Deal Percentage
9.88 TiB 9.88 TiB 1 0.18%
164.79 TiB 330.15 TiB 2 5.89%
371.30 TiB 1.10 PiB 3 20.14%
193.00 TiB 789.50 TiB 4 14.09%
65.09 TiB 326.89 TiB 5 5.83%
56.25 TiB 340.02 TiB 6 6.07%
65.27 TiB 463.82 TiB 7 8.28%
75.79 TiB 622.53 TiB 8 11.11%
63.03 TiB 575.54 TiB 9 10.27%
45.09 TiB 456.63 TiB 10 8.15%
32.56 TiB 358.69 TiB 11 6.40%
10.84 TiB 130.18 TiB 12 2.32%
3.97 TiB 51.79 TiB 13 0.92%
1.39 TiB 19.57 TiB 14 0.35%

Replication Distribution

Deal Data Shared with other Clients

The below table shows how many unique data are shared with other clients. Usually different applications owns different data and should not resolve to the same CID.

⚠️ CID sharing has been observed.

Other Client Application Total Deals Affected Unique CIDs Verifier
f1z7jogzx4x42wtilzb4lu6iotlad5rptt2acbzpi Speedium network 752.00 TiB 18,148 LDN v3 multisig

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 6

Multisig Notary address

f01858410

Client address

f15cps7yo2x4fosvp45opfihk3a4wg2qukmwyet7a

DataCap allocation requested

475.76TiB

Id

bf95490f-b431-4678-83ab-670933da0ca4

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f01858410

Client address

f15cps7yo2x4fosvp45opfihk3a4wg2qukmwyet7a

Last two approvers

kernelogic & megtei

Rule to calculate the allocation request amount

800% of weekly dc amount requested

DataCap allocation requested

475.76TiB

Total DataCap granted for client so far

7.21PiB

Datacap to be granted to reach the total amount requested by the client (5 PiB)

-2495011785749298B

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
361577 44 2.34PiB 11.09 597.86TiB
filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report[^1]

Storage Provider Distribution

The below table shows the distribution of storage providers that have stored data for this client.

If this is the first time a provider takes verified deal, it will be marked as new.

For most of the datacap application, below restrictions should apply.

⚠️ f01886710 has unknown IP location.

⚠️ 34.78% of total deal sealed by f01156975 are duplicate data.

⚠️ f01923553 has unknown IP location.

⚠️ f01848694 has unknown IP location.

Provider Location Total Deals Sealed Percentage Unique Data Duplicate Deals
f01208803 Sydney, New South Wales, AU
Anycast Global Backbone
133.80 TiB 1.92% 122.01 TiB 8.82%
f01208632 Sydney, New South Wales, AU
Anycast Global Backbone
57.06 TiB 0.82% 56.33 TiB 1.29%
f01157271 Sydney, New South Wales, AU
Anycast Global Backbone
38.88 TiB 0.56% 38.32 TiB 1.45%
f01157249 Sydney, New South Wales, AU
Anycast Global Backbone
36.27 TiB 0.52% 35.33 TiB 2.61%
f01156835 Sydney, New South Wales, AU
Anycast Global Backbone
2.20 TiB 0.03% 2.17 TiB 1.60%
f01127678 Poznań, Greater Poland, PL
Beyond.pl sp. z o.o.
8.54 TiB 0.12% 8.54 TiB 0.00%
f01929568new Stockholm, Stockholm, SE
Binero AB
129.50 TiB 1.86% 129.47 TiB 0.02%
f022352 Oslo, Oslo, NO
Blix Solutions AS
165.48 TiB 2.38% 165.04 TiB 0.26%
f01915033 Chengdu, Sichuan, CN
China Mobile Communications Group Co., Ltd.
230.82 TiB 3.32% 230.32 TiB 0.22%
f0117434new Chengdu, Sichuan, CN
China Mobile Communications Group Co., Ltd.
134.86 TiB 1.94% 134.73 TiB 0.10%
f01953985new Shenzhen, Guangdong, CN
China Telecom (Group)
304.68 TiB 4.38% 304.40 TiB 0.09%
f01928520 Maywood Park, Oregon, US
Flexential Colorado Corp.
715.13 TiB 10.28% 715.13 TiB 0.00%
f033462 University Park, Texas, US
Flexential Colorado Corp.
583.59 TiB 8.39% 583.59 TiB 0.00%
f01882184 Herndon, Virginia, US
PCCW Global, Inc.
543.26 TiB 7.81% 543.26 TiB 0.00%
f01886710 Unknown
Unknown
493.90 TiB 7.10% 468.30 TiB 5.18%
f01208189 Sydney, New South Wales, AU
Anycast Global Backbone
104.93 TiB 1.51% 104.19 TiB 0.71%
f01208154 Sydney, New South Wales, AU
Anycast Global Backbone
89.79 TiB 1.29% 74.58 TiB 16.94%
f01156975 Sydney, New South Wales, AU
Anycast Global Backbone
31.99 TiB 0.46% 20.86 TiB 34.78%
f01157027 Sydney, New South Wales, AU
Anycast Global Backbone
23.39 TiB 0.34% 23.33 TiB 0.27%
f01157018 Sydney, New South Wales, AU
Anycast Global Backbone
23.15 TiB 0.33% 20.64 TiB 10.83%
f01156901 Sydney, New South Wales, AU
Anycast Global Backbone
21.18 TiB 0.30% 19.81 TiB 6.47%
f01154023 Sydney, New South Wales, AU
Anycast Global Backbone
12.33 TiB 0.18% 10.15 TiB 17.71%
f01156538 Sydney, New South Wales, AU
Anycast Global Backbone
416.00 GiB 0.01% 416.00 GiB 0.00%
f01345523 Antwerpen, Flanders, BE
Cogent Communications
381.28 TiB 5.48% 381.25 TiB 0.01%
f01882569 Atlanta, Georgia, US
Cyxtera Technologies Inc
52.83 TiB 0.76% 52.82 TiB 0.01%
f01889910 Phoenix, Arizona, US
Cyxtera Technologies Inc
4.12 TiB 0.06% 4.12 TiB 0.00%
f01206408 Sydney, New South Wales, AU
DATACOM SYSTEMS (AU) PTY LTD
23.32 TiB 0.34% 23.29 TiB 0.13%
f01207874 Sydney, New South Wales, AU
DATACOM SYSTEMS (AU) PTY LTD
11.38 TiB 0.16% 11.35 TiB 0.27%
f01207954 Sydney, New South Wales, AU
DATACOM SYSTEMS (AU) PTY LTD
10.45 TiB 0.15% 10.45 TiB 0.04%
f01208042 Sydney, New South Wales, AU
DATACOM SYSTEMS (AU) PTY LTD
8.73 TiB 0.13% 8.66 TiB 0.76%
f01971600 University Park, Texas, US
Flexential Colorado Corp.
376.62 TiB 5.42% 375.95 TiB 0.18%
f01992630 University Park, Texas, US
Flexential Colorado Corp.
197.05 TiB 2.83% 197.05 TiB 0.00%
f01392893 Amsterdam, North Holland, NL
Fusix Networks B.V.
375.23 TiB 5.40% 374.25 TiB 0.26%
f01886704 Las Vegas, Nevada, US
GTT Communications Inc.
79.07 TiB 1.14% 79.07 TiB 0.00%
f01886690 Las Vegas, Nevada, US
GTT Communications Inc.
74.67 TiB 1.07% 74.67 TiB 0.00%
f01911083 Tuusula, Uusimaa, FI
Hetzner Online GmbH
74.05 TiB 1.06% 74.02 TiB 0.04%
f01385207 Lincoln, Nebraska, US
LightEdge Solutions
357.92 TiB 5.15% 357.23 TiB 0.19%
f01652333 Sunnyvale, California, US
Packet Host, Inc.
215.93 TiB 3.11% 201.04 TiB 6.90%
f01877571 Herndon, Virginia, US
PCCW Global, Inc.
343.21 TiB 4.94% 343.21 TiB 0.00%
f01880047 Herndon, Virginia, US
PCCW Global, Inc.
192.55 TiB 2.77% 192.55 TiB 0.00%
f01923553new Unknown
Unknown
150.03 TiB 2.16% 150.00 TiB 0.02%
f01829137 Columbus, Ohio, US
PCM Sales, Inc
96.65 TiB 1.39% 96.65 TiB 0.00%
f01732189 San Diego, California, US
ScaleMatrix
25.40 TiB 0.37% 25.15 TiB 0.98%
f01848694 Unknown
Unknown
17.93 TiB 0.26% 17.93 TiB 0.00%
f01199430 Heerhugowaard, North Holland, NL
Wijnand Schouten trading as Speedium
96.00 GiB 0.00% 96.00 GiB 0.00%

Provider Distribution

Deal Data Replication

The below table shows how each many unique data are replicated across storage providers.

✔️ Data replication looks healthy.

Unique Data Size Total Deals Made Number of Providers Deal Percentage
63.03 GiB 63.03 GiB 1 0.00%
8.01 TiB 16.02 TiB 2 0.23%
142.19 TiB 427.05 TiB 3 6.14%
373.65 TiB 1.48 PiB 4 21.75%
209.47 TiB 1.05 PiB 5 15.51%
61.19 TiB 368.71 TiB 6 5.30%
49.03 TiB 345.54 TiB 7 4.97%
67.81 TiB 549.60 TiB 8 7.90%
82.14 TiB 756.05 TiB 9 10.87%
60.53 TiB 613.93 TiB 10 8.83%
37.20 TiB 414.80 TiB 11 5.97%
23.92 TiB 287.63 TiB 12 4.14%
26.95 TiB 350.30 TiB 13 5.04%
10.83 TiB 151.70 TiB 14 2.18%
3.96 TiB 59.67 TiB 15 0.86%
1.38 TiB 22.16 TiB 16 0.32%

Replication Distribution

Deal Data Shared with other Clients

The below table shows how many unique data are shared with other clients. Usually different applications owns different data and should not resolve to the same CID.

However, this could be possible if all below clients use same software to prepare for the exact same dataset or they belong to a series of LDN applications for the same dataset.

⚠️ CID sharing has been observed.

Other Client Application Total Deals Affected Unique CIDs Approvers
f1z7jogzx4x42wtilzb4lu6iotlad5rptt2acbzpi Speedium network 752.00 TiB 18,148 1flyworker
1kernelogic
4MegTei
2psh0691
3Reiers
3s0nik42

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

herrehesse commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report[^1]

Storage Provider Distribution

The below table shows the distribution of storage providers that have stored data for this client.

If this is the first time a provider takes verified deal, it will be marked as new.

For most of the datacap application, below restrictions should apply.

⚠️ 34.78% of total deal sealed by f01156975 are duplicate data.

⚠️ f01923553 has unknown IP location.

⚠️ f01848694 has unknown IP location.

Provider Location Total Deals Sealed Percentage Unique Data Duplicate Deals
f01208803 Melbourne, Victoria, AU
Anycast Global Backbone
135.24 TiB 1.80% 122.71 TiB 9.26%
f01208189 Melbourne, Victoria, AU
Anycast Global Backbone
105.14 TiB 1.40% 104.27 TiB 0.82%
f01208154 Melbourne, Victoria, AU
Anycast Global Backbone
91.69 TiB 1.22% 76.43 TiB 16.64%
f01208632 Melbourne, Victoria, AU
Anycast Global Backbone
63.70 TiB 0.85% 62.81 TiB 1.40%
f01157249 Melbourne, Victoria, AU
Anycast Global Backbone
39.70 TiB 0.53% 36.78 TiB 7.35%
f01157271 Melbourne, Victoria, AU
Anycast Global Backbone
39.04 TiB 0.52% 38.47 TiB 1.44%
f01156975 Melbourne, Victoria, AU
Anycast Global Backbone
31.99 TiB 0.42% 20.86 TiB 34.78%
f01157027 Melbourne, Victoria, AU
Anycast Global Backbone
23.36 TiB 0.31% 23.30 TiB 0.27%
f01157018 Melbourne, Victoria, AU
Anycast Global Backbone
23.15 TiB 0.31% 20.64 TiB 10.83%
f01156901 Melbourne, Victoria, AU
Anycast Global Backbone
21.18 TiB 0.28% 19.81 TiB 6.47%
f01154023 Melbourne, Victoria, AU
Anycast Global Backbone
12.33 TiB 0.16% 10.15 TiB 17.71%
f01156835 Melbourne, Victoria, AU
Anycast Global Backbone
2.21 TiB 0.03% 2.18 TiB 1.59%
f01156538 Melbourne, Victoria, AU
Anycast Global Backbone
1.97 TiB 0.03% 1.97 TiB 0.00%
f01127678 Poznań, Greater Poland, PL
Beyond.pl sp. z o.o.
8.54 TiB 0.11% 8.54 TiB 0.00%
f01929568new Stockholm, Stockholm, SE
Binero AB
129.50 TiB 1.72% 129.47 TiB 0.02%
f022352 Oslo, Oslo, NO
Blix Solutions AS
165.48 TiB 2.20% 165.04 TiB 0.26%
f01915033 Chengdu, Sichuan, CN
China Mobile Communications Group Co., Ltd.
230.82 TiB 3.07% 230.32 TiB 0.22%
f0117434new Chengdu, Sichuan, CN
China Mobile Communications Group Co., Ltd.
134.86 TiB 1.79% 134.73 TiB 0.10%
f01953985new Shenzhen, Guangdong, CN
China Telecom (Group)
304.68 TiB 4.05% 304.40 TiB 0.09%
f01886710 Las Vegas, Nevada, US
Cogent Communications
493.90 TiB 6.56% 468.30 TiB 5.18%
f01345523 Antwerpen, Flanders, BE
Cogent Communications
381.28 TiB 5.06% 381.25 TiB 0.01%
f01886690 Las Vegas, Nevada, US
Cogent Communications
74.67 TiB 0.99% 74.67 TiB 0.00%
f01882569 Atlanta, Georgia, US
Cyxtera Technologies Inc
52.83 TiB 0.70% 52.82 TiB 0.01%
f01206408 Sydney, New South Wales, AU
DATACOM SYSTEMS (AU) PTY LTD
23.32 TiB 0.31% 23.29 TiB 0.13%
f01207874 Sydney, New South Wales, AU
DATACOM SYSTEMS (AU) PTY LTD
11.38 TiB 0.15% 11.35 TiB 0.27%
f01207954 Sydney, New South Wales, AU
DATACOM SYSTEMS (AU) PTY LTD
10.45 TiB 0.14% 10.45 TiB 0.04%
f01208042 Sydney, New South Wales, AU
DATACOM SYSTEMS (AU) PTY LTD
8.73 TiB 0.12% 8.66 TiB 0.76%
f01928520 Maywood Park, Oregon, US
Flexential Colorado Corp.
715.13 TiB 9.50% 715.13 TiB 0.00%
f033462 Dallas, Texas, US
Flexential Colorado Corp.
583.59 TiB 7.75% 583.59 TiB 0.00%
f01971600 Dallas, Texas, US
Flexential Colorado Corp.
376.62 TiB 5.00% 375.95 TiB 0.18%
f01972356 Maywood Park, Oregon, US
Flexential Colorado Corp.
228.03 TiB 3.03% 200.58 TiB 12.03%
f01992630 Dallas, Texas, US
Flexential Colorado Corp.
197.05 TiB 2.62% 197.05 TiB 0.00%
f01392893 Amsterdam, North Holland, NL
Fusix Networks B.V.
375.23 TiB 4.98% 374.25 TiB 0.26%
f01882184 Singapore, Singapore, SG
Gateway Communications
543.26 TiB 7.21% 543.26 TiB 0.00%
f01877571 Singapore, Singapore, SG
Gateway Communications
343.21 TiB 4.56% 343.21 TiB 0.00%
f01880047 Singapore, Singapore, SG
Gateway Communications
192.55 TiB 2.56% 192.55 TiB 0.00%
f01829137 Las Vegas, Nevada, US
GTT Communications Inc.
96.65 TiB 1.28% 96.65 TiB 0.00%
f01886704 Las Vegas, Nevada, US
GTT Communications Inc.
79.07 TiB 1.05% 79.07 TiB 0.00%
f01911083 Tuusula, Uusimaa, FI
Hetzner Online GmbH
143.03 TiB 1.90% 143.00 TiB 0.02%
f01889910 Phoenix, Arizona, US
Level 3 Parent, LLC
4.12 TiB 0.05% 4.12 TiB 0.00%
f01385207 Lincoln, Nebraska, US
LightEdge Solutions
357.92 TiB 4.75% 357.23 TiB 0.19%
f01845552new Kansas City, Missouri, US
LightEdge Solutions
139.84 TiB 1.86% 112.54 TiB 19.52%
f01838599new Kansas City, Missouri, US
LightEdge Solutions
124.57 TiB 1.65% 103.94 TiB 16.56%
f01652333 Sunnyvale, California, US
Packet Host, Inc.
215.93 TiB 2.87% 201.04 TiB 6.90%
f01923553new Unknown
Unknown
150.03 TiB 1.99% 150.00 TiB 0.02%
f01732189 San Diego, California, US
ScaleMatrix
25.40 TiB 0.34% 25.15 TiB 0.98%
f01848694 Unknown
Unknown
17.93 TiB 0.24% 17.93 TiB 0.00%
f01199430 Heerhugowaard, North Holland, NL
Wijnand Schouten trading as Speedium
96.00 GiB 0.00% 96.00 GiB 0.00%

Provider Distribution

Deal Data Replication

The below table shows how each many unique data are replicated across storage providers.

✔️ Data replication looks healthy.

Unique Data Size Total Deals Made Number of Providers Deal Percentage
1.02 GiB 1.02 GiB 1 0.00%
6.06 TiB 12.13 TiB 2 0.16%
118.67 TiB 357.30 TiB 3 4.74%
343.29 TiB 1.37 PiB 4 18.59%
193.21 TiB 1013.22 TiB 5 13.46%
72.39 TiB 450.14 TiB 6 5.98%
72.36 TiB 515.72 TiB 7 6.85%
60.09 TiB 484.94 TiB 8 6.44%
63.60 TiB 579.93 TiB 9 7.70%
74.15 TiB 758.56 TiB 10 10.07%
58.16 TiB 648.70 TiB 11 8.61%
35.27 TiB 428.97 TiB 12 5.70%
23.55 TiB 312.09 TiB 13 4.14%
21.44 TiB 309.82 TiB 14 4.11%
10.74 TiB 167.68 TiB 15 2.23%
3.96 TiB 66.57 TiB 16 0.88%
1.38 TiB 24.82 TiB 17 0.33%

Replication Distribution

Deal Data Shared with other Clients

The below table shows how many unique data are shared with other clients. Usually different applications owns different data and should not resolve to the same CID.

However, this could be possible if all below clients use same software to prepare for the exact same dataset or they belong to a series of LDN applications for the same dataset.

⚠️ CID sharing has been observed.

Other Client Application Total Deals Affected Unique CIDs Approvers
f1z7jogzx4x42wtilzb4lu6iotlad5rptt2acbzpi Speedium network 752.00 TiB 18,148 1flyworker
1kernelogic
4MegTei
2psh0691
3Reiers
3s0nik42

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

kernelogic commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacecozahbh2omh4sfkvvoiyics5ojpnaghgdca3vjlk5kdkm4jde2wi

Address

f15cps7yo2x4fosvp45opfihk3a4wg2qukmwyet7a

Datacap Allocated

475.76TiB

Signer Address

f1yjhnsoga2ccnepb7t3p3ov5fzom3syhsuinxexa

Id

bf95490f-b431-4678-83ab-670933da0ca4

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacecozahbh2omh4sfkvvoiyics5ojpnaghgdca3vjlk5kdkm4jde2wi

flyworker commented 1 year ago

CID Checker pass and can you provide a command for retrieval?

lvschouwen commented 1 year ago

@flyworker Let me write a short tutorial on how to retrieve verified data. We start with going to https://filplus.d.interplanetary.one/ and search the client, in this case we are looking for f01837713 - https://filplus.d.interplanetary.one/clients/f01837713

Once the page is done loading we can click on Export to CSV. You can then search for the Miner ID or any random deal you would like to retrieve, we need to "Verified Deal ID".

The command to use is; lotus state get-deal 1234 (where 1234 is the Verified Deal ID)

This will result in a output with the PieceCID, Provider (this is also in the CSV) and the Label. Its the label we need for the retrieval.

The command to use is; lotus client retrieval --provider f01234 $Label $PieceCID

This will try and retrieve the file from the miner.

I hope this helps.

psh0691 commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacecwszs3vex6v7plw46vw63vhdsx67afn7azeelolo2ufi6qqbexb2

Address

f15cps7yo2x4fosvp45opfihk3a4wg2qukmwyet7a

Datacap Allocated

475.76TiB

Signer Address

f1qdko4jg25vo35qmyvcrw4ak4fmuu3f5rif2kc7i

Id

bf95490f-b431-4678-83ab-670933da0ca4

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacecwszs3vex6v7plw46vw63vhdsx67afn7azeelolo2ufi6qqbexb2

large-datacap-requests[bot] commented 1 year ago

The issue reached the total datacap requested. This should be closed

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f01858410

Client address

f15cps7yo2x4fosvp45opfihk3a4wg2qukmwyet7a

Last two approvers

psh0691 & kernelogic

Rule to calculate the allocation request amount

total dc reached

DataCap allocation requested

0

Total DataCap granted for client so far

7.21PiB

Datacap to be granted to reach the total amount requested by the client (5 PiB)

-2495011785749298B

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
399308 47 475.76TiB 10.2 409.19MiB