filecoin-project / filecoin-plus-large-datasets

Hub for client applications for DataCap at a large scale
109 stars 62 forks source link

[DataCap Application] Bearded LDN-03 Seal Storage Technology #955

Closed scharfstein closed 1 year ago

scharfstein commented 2 years ago

Large Dataset Notary Application

To apply for DataCap to onboard your dataset to Filecoin, please fill out the following.

Core Information

Please respond to the questions below by replacing the text saying "Please answer here". Include as much detail as you can in your answer.

Project details

Share a brief history of your project and organization.

NOTE: this is LDN app 3 of 3. Total data cap requested for this project is 10 PiB.
LDN app 1 of 3 can be found here: https://github.com/filecoin-project/filecoin-plus-large-datasets/issues/326
LDN app 2 of 3 can be found here: https://github.com/filecoin-project/filecoin-plus-large-datasets/issues/327

This new LDN application is in response to this comment: https://github.com/filecoin-project/filecoin-plus-large-datasets/issues/327#issuecomment-1191761175

and @dkkapur's recommendation to open a new LDN application.

*** The rest of this information is from the original LDN applications ***

Our Customer is world-class research institution that develops and inspires knowledge-based solutions and educates future leaders for just and prosperous societies on a healthy planet. The data set involved in this project is a 1.15 PiB public climate data set.

The project is a pilot to prove out the value propositions of decentralized data storage.  This DataCap application is being submitted by Seal Storage on behalf of our Customer, who wishes to remain confidential for the duration of this pilot project. Project details can be made available under an NDA.

We kicked off the project on March 8, 2022 with ingestion estimated to begin in mid March 2022 via download from the cloud. The data set is public climate data. 

Due to the perceived risk associated with cryptocurrency projects, our Customer feels it is best to delay public announcement of our collaboration until we have successfully completed the project.

Seal is a carbon-neutral, decentralized cloud storage provider. Seal's technical leadership brings decades of experience from traditional enterprise storage companies including Seagate and Oracle, as well as world-class experience on the Filecoin Network. Today, Seal operates data centers across the US and Canada with enterprise-grade infrastructure and data policies.

What is the primary source of funding for this project?

Seal is funding the project.

What other projects/ecosystem stakeholders is this project associated with?

None at this time.

Use-case details

Describe the data being stored onto Filecoin

The data sets is public climate data.

Where was the data in this dataset sourced from?

Our Customer has a copy in Google cloud.

Can you share a sample of the data? A link to a file, an image, a table, etc., are good ways to do this.

Yes. The dataset is the CMIP6 Zarr store described here
https://pangeo-data.github.io/pangeo-cmip6-cloud/overview.html
The data are fully public in Google Cloud Storage at: gs://cmip6

Confirm that this is a public dataset that can be retrieved by anyone on the Network (i.e., no specific permissions or access rights are required to view the data).

Confirmed. This is a public data set.

What is the expected retrieval frequency for this data?

Archival is primary. The data will be accessed by external collaborators and Researchers.

For how long do you plan to keep this dataset stored on Filecoin?

Three years, at least.

DataCap allocation plan

In which geographies (countries, regions) do you plan on making storage deals?

Six (6) full replicas shall be stored across six (6) cities, three (3) countries, two (2) continents
Each copy is 1.15 PiB

How will you be distributing your data to storage providers? Is there an offline data transfer process?

Seal Storage has dual 100 Gbps internet connections. SPs will download data from Seal. Offline data transfer may be possible.

How do you plan on choosing the storage providers with whom you will be making deals? This should include a plan to ensure the data is retrievable in the future both by you and others.

We are currently discussing capabilities and performing due diligence with several SPs and have chosen three SPs for this project. We chose these based on their current storage capacity, compute capabilities, enterprise-grade DCs and bandwidth.

How will you be distributing deals across storage providers?

W3b Cloud [USA] / 1.15 PiB
PikNik [San Diego, USA] / 1.15 PiB
Holon [Sydney, AUS] / 1.15 PiB
ElioVP [Antwerp, Belgium and Utrecht, Netherlands] / 1.15 PiB
Seal Montreal, Canada / 1.15 PiB
Seal Las Vegas, USA / 1.15 PiB

Seal must also keep a full hot copy for our Customer.

Do you have the resources/funding to start making deals as soon as you receive DataCap? What support from the community would help you onboard onto Filecoin?

Yes, we have the resources/funding to begin making deals once we receive DataCap. 

We currently have the support we need.
large-datacap-requests[bot] commented 2 years ago

Thanks for your request!

Heads up, you’re requesting more than the typical weekly onboarding rate of DataCap!
large-datacap-requests[bot] commented 2 years ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

large-datacap-requests[bot] commented 2 years ago

Thanks for your request!

Heads up, you’re requesting more than the typical weekly onboarding rate of DataCap!
large-datacap-requests[bot] commented 2 years ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

raghavrmadya commented 2 years ago

Datacap Request Trigger

Total DataCap requested

3.1PiB

Expected weekly DataCap usage rate

400TiB

Client address

f1jeautnpgr6js6uqtm4aa6mf6dvhn3unbdcvpbpy

dannyob commented 2 years ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacecviwsgedwyt5vqpraugf5xwitsapj6nobcjvc4jg3wjlln4o2mbg

Address

f1jeautnpgr6js6uqtm4aa6mf6dvhn3unbdcvpbpy

Datacap Allocated

158.71TiB

Signer Address

f1k6wwevxvp466ybil7y2scqlhtnrz5atjkkyvm4a

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacecviwsgedwyt5vqpraugf5xwitsapj6nobcjvc4jg3wjlln4o2mbg

Fenbushi-Filecoin commented 2 years ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaceb2ywpipkptpxop72i5uhvf45j3rwjpexef4loexgiocuavkfexpk

Address

f1jeautnpgr6js6uqtm4aa6mf6dvhn3unbdcvpbpy

Datacap Allocated

158.71TiB

Signer Address

f1yqydpmqb5en262jpottko2kd65msajax7fi4rmq

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceb2ywpipkptpxop72i5uhvf45j3rwjpexef4loexgiocuavkfexpk

raghavrmadya commented 2 years ago

DataCap Allocation requested

Multisig Notary address

f01924196

Client address

f1jeautnpgr6js6uqtm4aa6mf6dvhn3unbdcvpbpy

DataCap allocation requested

158.71TiB

raghavrmadya commented 2 years ago

@dannyob @Fenbushi-Filecoin , please sign again as the old multisig didn't have enough DC

Fenbushi-Filecoin commented 2 years ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacecxcrl5ygmb7jpttk5nesl7ewk2lebly7opnklzlz6wxi4grfervy

Address

f1jeautnpgr6js6uqtm4aa6mf6dvhn3unbdcvpbpy

Datacap Allocated

158.71TiB

Signer Address

f1yqydpmqb5en262jpottko2kd65msajax7fi4rmq

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacecxcrl5ygmb7jpttk5nesl7ewk2lebly7opnklzlz6wxi4grfervy

dannyob commented 2 years ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacebhqbfnnw25lavmwvb36u25k7m6wbvpcfiw3cqplw4ulowx4l6mi4

Address

f1jeautnpgr6js6uqtm4aa6mf6dvhn3unbdcvpbpy

Datacap Allocated

158.71TiB

Signer Address

f1k6wwevxvp466ybil7y2scqlhtnrz5atjkkyvm4a

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebhqbfnnw25lavmwvb36u25k7m6wbvpcfiw3cqplw4ulowx4l6mi4

large-datacap-requests[bot] commented 2 years ago

DataCap Allocation requested

Request number 2

Multisig Notary address

f01924196

Client address

f1jeautnpgr6js6uqtm4aa6mf6dvhn3unbdcvpbpy

DataCap allocation requested

317.43TiB

large-datacap-requests[bot] commented 2 years ago

Stats & Info for DataCap Allocation

Multisig Notary address

f01924196

Client address

f1jeautnpgr6js6uqtm4aa6mf6dvhn3unbdcvpbpy

Last two approvers

dannyob & Fenbushi-Filecoin

Rule to calculate the allocation request amount

10% of total dc amount requested

DataCap allocation requested

317.43TiB

Total DataCap granted for client so far

158.70TiB

Datacap to be granted to reach the total amount requested by the client (3.1 PiB)

2.94PiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
1612 14 158.71TiB 45.11 5.88GiB
neogeweb3 commented 2 years ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzaceds4cw5rn3vdndxlravilr2biw3epi7hfzww2iuwbdsibzuu7kylu

Address

f1jeautnpgr6js6uqtm4aa6mf6dvhn3unbdcvpbpy

Datacap Allocated

317.43TiB

Signer Address

f13k5zr6ovc2gjmg3lvd43ladbydhovpylcvbflpa

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceds4cw5rn3vdndxlravilr2biw3epi7hfzww2iuwbdsibzuu7kylu

mjroddy commented 2 years ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacecjphqfc6s26jbgy67qu3i2feye4zeggivxj4vmqrzxqkejwtsfiq

Address

f1jeautnpgr6js6uqtm4aa6mf6dvhn3unbdcvpbpy

Datacap Allocated

317.43TiB

Signer Address

f1ystxl2ootvpirpa7ebgwl7vlhwkbx2r4zjxwe5i

Id

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacecjphqfc6s26jbgy67qu3i2feye4zeggivxj4vmqrzxqkejwtsfiq

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 3

Multisig Notary address

f01924196

Client address

f1jeautnpgr6js6uqtm4aa6mf6dvhn3unbdcvpbpy

DataCap allocation requested

634.88TiB

Id

fa5d6eae-f83e-4575-b9b6-8f9d16e78593

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f01924196

Client address

f1jeautnpgr6js6uqtm4aa6mf6dvhn3unbdcvpbpy

Last two approvers

megtei & neogeweb3

Rule to calculate the allocation request amount

20% of total dc amount requested

DataCap allocation requested

634.88TiB

Total DataCap granted for client so far

476.13TiB

Datacap to be granted to reach the total amount requested by the client (3.1 PiB)

2.63PiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
10410 16 317.43TiB 54.03 77.74TiB
dannyob commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzaceav23pmbudwo4cuc2ctwsnmeojngznzatbfcv5mkl3sirf7mxiji4

Address

f1jeautnpgr6js6uqtm4aa6mf6dvhn3unbdcvpbpy

Datacap Allocated

634.88TiB

Signer Address

f1k6wwevxvp466ybil7y2scqlhtnrz5atjkkyvm4a

Id

fa5d6eae-f83e-4575-b9b6-8f9d16e78593

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceav23pmbudwo4cuc2ctwsnmeojngznzatbfcv5mkl3sirf7mxiji4

Fenbushi-Filecoin commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacebogo3kwhkktmmx2nclp6zcs3ag4jxggzpesqihwbkcmc246w4s64

Address

f1jeautnpgr6js6uqtm4aa6mf6dvhn3unbdcvpbpy

Datacap Allocated

634.88TiB

Signer Address

f1yqydpmqb5en262jpottko2kd65msajax7fi4rmq

Id

fa5d6eae-f83e-4575-b9b6-8f9d16e78593

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebogo3kwhkktmmx2nclp6zcs3ag4jxggzpesqihwbkcmc246w4s64

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 4

Multisig Notary address

f01924196

Client address

f1jeautnpgr6js6uqtm4aa6mf6dvhn3unbdcvpbpy

DataCap allocation requested

1.24PiB

Id

8a46a448-cbdb-4fe9-a0f5-7623d3080ca4

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f01924196

Client address

f1jeautnpgr6js6uqtm4aa6mf6dvhn3unbdcvpbpy

Last two approvers

Fenbushi-Filecoin & dannyob

Rule to calculate the allocation request amount

40% of total dc amount requested

DataCap allocation requested

1.24PiB

Total DataCap granted for client so far

1.08PiB

Datacap to be granted to reach the total amount requested by the client (3.1 PiB)

2.01PiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
26623 22 634.88TiB 27.60 3.29GiB
mjroddy commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacedntymd6wfti4rs4aii3yudxvk72xv5vyfhqdsbh6qaeokyjqmu2m

Address

f1jeautnpgr6js6uqtm4aa6mf6dvhn3unbdcvpbpy

Datacap Allocated

1.24PiB

Signer Address

f1ystxl2ootvpirpa7ebgwl7vlhwkbx2r4zjxwe5i

Id

8a46a448-cbdb-4fe9-a0f5-7623d3080ca4

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedntymd6wfti4rs4aii3yudxvk72xv5vyfhqdsbh6qaeokyjqmu2m

Destore2023 commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaced6w2vagt5gmthvgo77zt3chqeonxybbk2tuq65xj7myxtz3tgmly

Address

f1jeautnpgr6js6uqtm4aa6mf6dvhn3unbdcvpbpy

Datacap Allocated

1.24PiB

Signer Address

f1yh6q3nmsg7i2sys7f7dexcuajgoweudcqj2chfi

Id

8a46a448-cbdb-4fe9-a0f5-7623d3080ca4

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaced6w2vagt5gmthvgo77zt3chqeonxybbk2tuq65xj7myxtz3tgmly

filplus-checker commented 1 year ago

DataCap and CID Checker Report[^1]

If this is the first time a provider takes verified deal, it will be marked as new.

For most of the datacap application, below restrictions should apply.

⚠️ f01506844 has sealed 25.57% of total datacap.

⚠️ f01923555 has unknown IP location.

⚠️ f01923554 has unknown IP location.

⚠️ f01923553 has unknown IP location.

⚠️ f01886710 has unknown IP location.

Provider Location Total Deals Sealed Percentage Unique Data Duplicate Deals
f01506844 Spokane, Washington, US 260.91 TiB 25.57% 260.91 TiB 0.00%
f01923555 Unknown 150.56 TiB 14.75% 150.56 TiB 0.00%
f01851060 Las Vegas, Nevada, US 144.66 TiB 14.18% 144.66 TiB 0.00%
f01923554 Unknown 132.42 TiB 12.98% 132.42 TiB 0.00%
f01923553 Unknown 63.53 TiB 6.23% 63.53 TiB 0.00%
f01873432 Las Vegas, Nevada, US 48.80 TiB 4.78% 48.80 TiB 0.00%
f01154023new Sydney, New South Wales, AU 30.94 TiB 3.03% 30.94 TiB 0.00%
f01156835 Sydney, New South Wales, AU 29.75 TiB 2.92% 29.75 TiB 0.00%
f01157018 Sydney, New South Wales, AU 20.45 TiB 2.00% 20.45 TiB 0.00%
f01157027 Sydney, New South Wales, AU 18.56 TiB 1.82% 18.56 TiB 0.00%
f01156883 Sydney, New South Wales, AU 17.55 TiB 1.72% 17.55 TiB 0.00%
f01156901 Sydney, New South Wales, AU 15.34 TiB 1.50% 15.34 TiB 0.00%
f01208189 Sydney, New South Wales, AU 14.06 TiB 1.38% 14.06 TiB 0.00%
f01208803 Sydney, New South Wales, AU 13.86 TiB 1.36% 13.86 TiB 0.00%
f01156975 Sydney, New South Wales, AU 12.70 TiB 1.24% 12.70 TiB 0.00%
f01208154 Sydney, New South Wales, AU 11.17 TiB 1.09% 11.17 TiB 0.00%
f01156538 Sydney, New South Wales, AU 9.53 TiB 0.93% 9.53 TiB 0.00%
f01157249 Sydney, New South Wales, AU 9.08 TiB 0.89% 9.08 TiB 0.00%
f01157271 Sydney, New South Wales, AU 8.08 TiB 0.79% 8.08 TiB 0.00%
f01208632 Sydney, New South Wales, AU 8.00 TiB 0.78% 8.00 TiB 0.00%
f01886710 Unknown 464.00 GiB 0.04% 464.00 GiB 0.00%

Provider Distribution

Deal Data Replication

The below table shows how each many unique data are replicated across storage providers.

⚠️ 99.97% of deals are for data replicated across less than 4 storage providers.

Unique Data Size Total Deals Made Number of Providers Deal Percentage
778.78 TiB 778.78 TiB 1 76.32%
109.63 TiB 219.25 TiB 2 21.49%
7.36 TiB 22.08 TiB 3 2.16%
80.00 GiB 320.00 GiB 4 0.03%

Replication Distribution

Deal Data Shared with other Clients

The below table shows how many unique data are shared with other clients. Usually different applications owns different data and should not resolve to the same CID.

⚠️ CID sharing has been observed.

Other Client Application Total Deals Affected Unique CIDs Verifier
f13ctc37hbnjhpz5vnksphpjfptfwrnqud2f445oq Seal Storage Technology 2.89 PiB 37,946 LDN #326
f1sxdwnecrck7zb6rbewsyuwi3d44fpdrnq6ngqey Seal Storage Technology 1.10 PiB 21,717 LDN #327

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

dannyob commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacedzsqf6ejyyhpmot5tvz2qo36tkmo7u72fift2guas4pobgnlwcdc

Address

f1jeautnpgr6js6uqtm4aa6mf6dvhn3unbdcvpbpy

Datacap Allocated

1.24PiB

Signer Address

f1k6wwevxvp466ybil7y2scqlhtnrz5atjkkyvm4a

Id

8a46a448-cbdb-4fe9-a0f5-7623d3080ca4

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedzsqf6ejyyhpmot5tvz2qo36tkmo7u72fift2guas4pobgnlwcdc

dannyob commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacec27whywu3xkh64dw2se5ldimil5rejxtqioqp54eesxcb7fjxd7m

Address

f1jeautnpgr6js6uqtm4aa6mf6dvhn3unbdcvpbpy

Datacap Allocated

1.24PiB

Signer Address

f1k6wwevxvp466ybil7y2scqlhtnrz5atjkkyvm4a

Id

8a46a448-cbdb-4fe9-a0f5-7623d3080ca4

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacec27whywu3xkh64dw2se5ldimil5rejxtqioqp54eesxcb7fjxd7m

Fenbushi-Filecoin commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacebkrqg7wfonijlehc4hxfx4dmqzw6okvowcajdgqxbtf7niqvwzyc

Address

f1jeautnpgr6js6uqtm4aa6mf6dvhn3unbdcvpbpy

Datacap Allocated

1.24PiB

Signer Address

f1yqydpmqb5en262jpottko2kd65msajax7fi4rmq

Id

8a46a448-cbdb-4fe9-a0f5-7623d3080ca4

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebkrqg7wfonijlehc4hxfx4dmqzw6okvowcajdgqxbtf7niqvwzyc

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 5

Multisig Notary address

f01924196

Client address

f1jeautnpgr6js6uqtm4aa6mf6dvhn3unbdcvpbpy

DataCap allocation requested

793.61TiB

Id

7fb4f71b-7b3e-41ee-9c50-0b031df31896

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f01924196

Client address

f1jeautnpgr6js6uqtm4aa6mf6dvhn3unbdcvpbpy

Last two approvers

Fenbushi-Filecoin & dannyob

Rule to calculate the allocation request amount

80% of total dc amount requested

DataCap allocation requested

793.61TiB

Total DataCap granted for client so far

2.32PiB

Datacap to be granted to reach the total amount requested by the client (3.1 PiB)

793.62TiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
99481 23 1.24PiB 21.13 35.66TiB
filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report[^1]

Storage Provider Distribution

The below table shows the distribution of storage providers that have stored data for this client.

If this is the first time a provider takes verified deal, it will be marked as new.

For most of the datacap application, below restrictions should apply.

⚠️ f01923555 has unknown IP location.

⚠️ f01923553 has unknown IP location.

Provider Location Total Deals Sealed Percentage Unique Data Duplicate Deals
f01851060 Las Vegas, Nevada, US
PiKNiK & Company Inc.
441.95 TiB 20.64% 441.86 TiB 0.02%
f01923555 Unknown
Unknown
348.80 TiB 16.29% 344.14 TiB 1.33%
f01156835 Melbourne, Victoria, AU
Anycast Global Backbone
47.22 TiB 2.20% 47.22 TiB 0.00%
f01157027 Melbourne, Victoria, AU
Anycast Global Backbone
31.53 TiB 1.47% 31.53 TiB 0.00%
f01157018 Melbourne, Victoria, AU
Anycast Global Backbone
31.05 TiB 1.45% 31.05 TiB 0.00%
f01154023new Melbourne, Victoria, AU
Anycast Global Backbone
30.94 TiB 1.44% 30.94 TiB 0.00%
f01208189 Melbourne, Victoria, AU
Anycast Global Backbone
20.66 TiB 0.96% 20.66 TiB 0.00%
f01208154 Melbourne, Victoria, AU
Anycast Global Backbone
20.09 TiB 0.94% 20.09 TiB 0.00%
f01208803 Melbourne, Victoria, AU
Anycast Global Backbone
19.95 TiB 0.93% 19.95 TiB 0.00%
f01208632 Melbourne, Victoria, AU
Anycast Global Backbone
19.39 TiB 0.91% 19.39 TiB 0.00%
f01156883 Melbourne, Victoria, AU
Anycast Global Backbone
17.73 TiB 0.83% 17.73 TiB 0.00%
f01156901 Melbourne, Victoria, AU
Anycast Global Backbone
17.33 TiB 0.81% 17.33 TiB 0.00%
f01156975 Melbourne, Victoria, AU
Anycast Global Backbone
13.45 TiB 0.63% 13.45 TiB 0.00%
f01156538 Melbourne, Victoria, AU
Anycast Global Backbone
12.17 TiB 0.57% 12.17 TiB 0.00%
f01157249 Melbourne, Victoria, AU
Anycast Global Backbone
9.08 TiB 0.42% 9.08 TiB 0.00%
f01157271 Melbourne, Victoria, AU
Anycast Global Backbone
8.08 TiB 0.38% 8.08 TiB 0.00%
f01345523 Antwerpen, Flanders, BE
Cogent Communications
13.25 TiB 0.62% 13.25 TiB 0.00%
f01886710 Las Vegas, Nevada, US
Cogent Communications
464.00 GiB 0.02% 464.00 GiB 0.00%
f01392893 Amsterdam, North Holland, NL
Fusix Networks B.V.
3.69 TiB 0.17% 3.69 TiB 0.00%
f01923554 Las Vegas, Nevada, US
GTT Communications Inc.
323.51 TiB 15.11% 319.01 TiB 1.39%
f01873432 Las Vegas, Nevada, US
PiKNiK & Company Inc.
233.39 TiB 10.90% 230.27 TiB 1.34%
f01506844 Spokane, Washington, US
Wholesail networks LLC
260.91 TiB 12.18% 260.91 TiB 0.00%
f01923553 Unknown
Unknown
216.91 TiB 10.13% 216.91 TiB 0.00%

Provider Distribution

Deal Data Replication

The below table shows how each many unique data are replicated across storage providers.

⚠️ 98.36% of deals are for data replicated across less than 4 storage providers.

Unique Data Size Total Deals Made Number of Providers Deal Percentage
682.73 TiB 688.90 TiB 1 32.17%
469.28 TiB 942.79 TiB 2 44.02%
157.64 TiB 474.81 TiB 3 22.17%
8.67 TiB 34.80 TiB 4 1.62%
48.00 GiB 240.00 GiB 5 0.01%

Replication Distribution

Deal Data Shared with other Clients

The below table shows how many unique data are shared with other clients. Usually different applications owns different data and should not resolve to the same CID.

However, this could be possible if all below clients use same software to prepare for the exact same dataset or they belong to a series of LDN applications for the same dataset.

⚠️ CID sharing has been observed.

Other Client Application Total Deals Affected Unique CIDs Approvers
f13ctc37hbnjhpz5vnksphpjfptfwrnqud2f445oq Seal Storage Technology 3.98 PiB 53,721 6dannyob
5Fenbushi-Filecoin
5MegTei
1mjroddy
1neogeweb3
2swatchliu
f1sxdwnecrck7zb6rbewsyuwi3d44fpdrnq6ngqey Seal Storage Technology 1.68 PiB 33,296 4dannyob
3Fenbushi-Filecoin
3MegTei
2neogeweb3
1swatchliu

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

dkkapur commented 1 year ago

@dannyob @Fenbushi-Filecoin - the previous allocation proposal exceeded the remaining DataCap on the custom multisig for this application so the DataCap did not go through. Allocation 5 triggered above should go through (https://github.com/filecoin-project/filecoin-plus-large-datasets/issues/955#issuecomment-1421299776), can you folks try signing again?

mjroddy commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacechcab7pc5h7hd2ld33dklatcjv53bnx4fmyekqoa7zdpo6ogimzc

Address

f1jeautnpgr6js6uqtm4aa6mf6dvhn3unbdcvpbpy

Datacap Allocated

1.24PiB

Signer Address

f1ystxl2ootvpirpa7ebgwl7vlhwkbx2r4zjxwe5i

Id

7fb4f71b-7b3e-41ee-9c50-0b031df31896

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacechcab7pc5h7hd2ld33dklatcjv53bnx4fmyekqoa7zdpo6ogimzc

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 6

Multisig Notary address

f01924196

Client address

f1jeautnpgr6js6uqtm4aa6mf6dvhn3unbdcvpbpy

DataCap allocation requested

10.24GiB

Id

dd472725-98f4-4dae-9cfa-0804ba225050

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f01924196

Client address

f1jeautnpgr6js6uqtm4aa6mf6dvhn3unbdcvpbpy

Last two approvers

megtei & Fenbushi-Filecoin

Rule to calculate the allocation request amount

80% of total dc amount requested

DataCap allocation requested

10.24GiB

Total DataCap granted for client so far

2.32PiB

Datacap to be granted to reach the total amount requested by the client (3.1 PiB)

793.62TiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
99965 23 793.61TiB 21.03 21.44TiB
filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report[^1]

Storage Provider Distribution

The below table shows the distribution of storage providers that have stored data for this client.

If this is the first time a provider takes verified deal, it will be marked as new.

For most of the datacap application, below restrictions should apply.

⚠️ f01923555 has unknown IP location.

⚠️ f01923553 has unknown IP location.

Provider Location Total Deals Sealed Percentage Unique Data Duplicate Deals
f01851060 Las Vegas, Nevada, US
PiKNiK & Company Inc.
441.95 TiB 20.55% 441.86 TiB 0.02%
f01923555 Unknown
Unknown
352.14 TiB 16.37% 347.49 TiB 1.32%
f01156835 Melbourne, Victoria, AU
Anycast Global Backbone
47.22 TiB 2.20% 47.22 TiB 0.00%
f01157027 Melbourne, Victoria, AU
Anycast Global Backbone
32.75 TiB 1.52% 32.75 TiB 0.00%
f01157018 Melbourne, Victoria, AU
Anycast Global Backbone
31.64 TiB 1.47% 31.64 TiB 0.00%
f01154023new Melbourne, Victoria, AU
Anycast Global Backbone
30.94 TiB 1.44% 30.94 TiB 0.00%
f01208189 Melbourne, Victoria, AU
Anycast Global Backbone
20.66 TiB 0.96% 20.66 TiB 0.00%
f01208154 Melbourne, Victoria, AU
Anycast Global Backbone
20.09 TiB 0.93% 20.09 TiB 0.00%
f01208803 Melbourne, Victoria, AU
Anycast Global Backbone
19.95 TiB 0.93% 19.95 TiB 0.00%
f01208632 Melbourne, Victoria, AU
Anycast Global Backbone
19.39 TiB 0.90% 19.39 TiB 0.00%
f01156883 Melbourne, Victoria, AU
Anycast Global Backbone
17.73 TiB 0.82% 17.73 TiB 0.00%
f01156901 Melbourne, Victoria, AU
Anycast Global Backbone
17.33 TiB 0.81% 17.33 TiB 0.00%
f01156975 Melbourne, Victoria, AU
Anycast Global Backbone
13.45 TiB 0.63% 13.45 TiB 0.00%
f01156538 Melbourne, Victoria, AU
Anycast Global Backbone
12.17 TiB 0.57% 12.17 TiB 0.00%
f01157249 Melbourne, Victoria, AU
Anycast Global Backbone
9.08 TiB 0.42% 9.08 TiB 0.00%
f01157271 Melbourne, Victoria, AU
Anycast Global Backbone
8.08 TiB 0.38% 8.08 TiB 0.00%
f01345523 Antwerpen, Flanders, BE
Cogent Communications
13.25 TiB 0.62% 13.25 TiB 0.00%
f01886710 Las Vegas, Nevada, US
Cogent Communications
464.00 GiB 0.02% 464.00 GiB 0.00%
f01392893 Amsterdam, North Holland, NL
Fusix Networks B.V.
3.69 TiB 0.17% 3.69 TiB 0.00%
f01923554 Las Vegas, Nevada, US
GTT Communications Inc.
323.51 TiB 15.04% 319.01 TiB 1.39%
f01873432 Las Vegas, Nevada, US
PiKNiK & Company Inc.
233.39 TiB 10.85% 230.27 TiB 1.34%
f01506844 Spokane, Washington, US
Wholesail networks LLC
260.91 TiB 12.13% 260.91 TiB 0.00%
f01923553 Unknown
Unknown
221.10 TiB 10.28% 221.10 TiB 0.00%

Provider Distribution

Deal Data Replication

The below table shows how each many unique data are replicated across storage providers.

⚠️ 98.31% of deals are for data replicated across less than 4 storage providers.

Unique Data Size Total Deals Made Number of Providers Deal Percentage
680.40 TiB 686.58 TiB 1 31.92%
465.14 TiB 934.50 TiB 2 43.45%
163.88 TiB 493.51 TiB 3 22.94%
8.98 TiB 36.05 TiB 4 1.68%
48.00 GiB 240.00 GiB 5 0.01%

Replication Distribution

Deal Data Shared with other Clients

The below table shows how many unique data are shared with other clients. Usually different applications owns different data and should not resolve to the same CID.

However, this could be possible if all below clients use same software to prepare for the exact same dataset or they belong to a series of LDN applications for the same dataset.

⚠️ CID sharing has been observed.

Other Client Application Total Deals Affected Unique CIDs Approvers
f13ctc37hbnjhpz5vnksphpjfptfwrnqud2f445oq Seal Storage Technology 3.98 PiB 53,721 6dannyob
5Fenbushi-Filecoin
5MegTei
1mjroddy
1neogeweb3
2swatchliu
f1sxdwnecrck7zb6rbewsyuwi3d44fpdrnq6ngqey Seal Storage Technology 1.68 PiB 33,296 4dannyob
3Fenbushi-Filecoin
3MegTei
2neogeweb3
1swatchliu

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

Fenbushi-Filecoin commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacebqlmd4s4fxoc7qljwdnemqw4qo4rslohncdf5a2rfsh7gocyvbj2

Address

f1jeautnpgr6js6uqtm4aa6mf6dvhn3unbdcvpbpy

Datacap Allocated

10.24GiB

Signer Address

f1yqydpmqb5en262jpottko2kd65msajax7fi4rmq

Id

dd472725-98f4-4dae-9cfa-0804ba225050

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebqlmd4s4fxoc7qljwdnemqw4qo4rslohncdf5a2rfsh7gocyvbj2

dkkapur commented 1 year ago

folks - this application seems to be on a bit of misdirected journey here. looks like the initial math was incorrect for the final allocation (1.24 PiB > remaining DataCap). I tried to patch this but the proposals were already in from @dannyob, so @mjroddy's attempt just ended up re-approving that rather than resulting in a proposal with the new values I tried to override. the bot then added a final allocation that is very small - 10.24 GiB. this has already been proposed on-chain by @Fenbushi-Filecoin. Can we do the following:

  1. @dannyob @mjroddy whoever sees this first, can you approve the current allocation in the tool and confirm here that you did? this allocation should go through.
  2. i'll force another allocation through with the remaining DataCap balance ~900 TiB
  3. once that message posts here, we need any of the 3 of you to propose and approve it

Thanks!

mjroddy commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacedwgev6wnpshaysly6gsltkz3cvq4pdsvgjaqi3vi2jr45yqtvekw

Address

f1jeautnpgr6js6uqtm4aa6mf6dvhn3unbdcvpbpy

Datacap Allocated

10.24GiB

Signer Address

f1ystxl2ootvpirpa7ebgwl7vlhwkbx2r4zjxwe5i

Id

dd472725-98f4-4dae-9cfa-0804ba225050

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedwgev6wnpshaysly6gsltkz3cvq4pdsvgjaqi3vi2jr45yqtvekw

mjroddy commented 1 year ago

@dkkapur I have approved - back to you

dannyob commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacecdyron6zuy4maqwj55ovrhfmmhul2dpbxacyvr3i2vckhmet3eya

Address

f1jeautnpgr6js6uqtm4aa6mf6dvhn3unbdcvpbpy

Datacap Allocated

10.24GiB

Signer Address

f1k6wwevxvp466ybil7y2scqlhtnrz5atjkkyvm4a

Id

dd472725-98f4-4dae-9cfa-0804ba225050

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacecdyron6zuy4maqwj55ovrhfmmhul2dpbxacyvr3i2vckhmet3eya

dannyob commented 1 year ago

oh whoops, I hope that didn't mess anything up

salstorage commented 1 year ago

@dkkapur signed by notaries as per your request to force next allocation Thanks

dkkapur commented 1 year ago

@dannyob yes you did :( same thing happening as last time. need to either get that approved or get an override.

dkkapur commented 1 year ago

@fabriziogianni7 is there a way we can drop the last request from the front end and we can force an override on the allocation amount?

fabriziogianni7 commented 1 year ago

actually in the dashboard there should be a functionality to cancel the request made by a notary @dannyob should be able to cancel it then it can be posted another request

fabriziogianni7 commented 1 year ago

@dkkapur @dannyob can you try to see here if danny has its transaction listed? Schermata 2023-02-14 alle 19 22 07

neogeweb3 commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaceclyob3lsi6kakv3z4sfwet5fgig2hntiqa6c5nkzesjmd5xlyffy

Address

f1jeautnpgr6js6uqtm4aa6mf6dvhn3unbdcvpbpy

Datacap Allocated

10.24GiB

Signer Address

f13k5zr6ovc2gjmg3lvd43ladbydhovpylcvbflpa

Id

dd472725-98f4-4dae-9cfa-0804ba225050

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceclyob3lsi6kakv3z4sfwet5fgig2hntiqa6c5nkzesjmd5xlyffy

dkkapur commented 1 year ago

DataCap Allocation requested

Request number 7

Multisig Notary address

f01924196

Client address

f1jeautnpgr6js6uqtm4aa6mf6dvhn3unbdcvpbpy

DataCap allocation requested

952TiB

Id

dd472725-98f4-4dae-9cfa-0804ba225051

mjroddy commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacebhngdh2ye7v6c3iezfaoj3b6cbau6q5mogpyes3g522uaml5uiie

Address

f1jeautnpgr6js6uqtm4aa6mf6dvhn3unbdcvpbpy

Datacap Allocated

952.00TiB

Signer Address

f1ystxl2ootvpirpa7ebgwl7vlhwkbx2r4zjxwe5i

Id

dd472725-98f4-4dae-9cfa-0804ba225051

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebhngdh2ye7v6c3iezfaoj3b6cbau6q5mogpyes3g522uaml5uiie

dkkapur commented 1 year ago
Screen Shot 2023-02-16 at 12 07 00 AM

Thanks @mjroddy - proposal for 952 TiB correctly landed on chain to the multisig. Now just need an approver. Labels also correctly auto adjusted with tooling, so this should now go through. @neogeweb3 @dannyob @Fenbushi-Filecoin can one of you give this a shot via the Fil+ registry app?