filecoin-project / filecoin-plus-large-datasets

Hub for client applications for DataCap at a large scale
109 stars 62 forks source link

[DataCap Application] Baikal Seal Storage Technology #1212

Closed salstorage closed 1 year ago

salstorage commented 2 years ago

Large Dataset Notary Application

To apply for DataCap to onboard your dataset to Filecoin, please fill out the following.

Core Information

Please respond to the questions below by replacing the text saying "Please answer here". Include as much detail as you can in your answer.

Project details

Share a brief history of your project and organization.

Due to previous LDN #325 application deprecated due to non activity, FF has requested we resubmit this LDN as new application.

Our customer is a Dark Matter Group within UC Berkeley and Seal is involved in a project with them to store the outputs of their scientific experiments. They would like to upload data to a distributed platform for other globally based researchers to be able to access this data. We will kicked off the project in early March 2022 with ingestion estimated to begin in mid April 2022 via portable disk unit. The customer’s data will not be encrypted, access controls will be implemented. They are looking for storage for at least the next three years.

Seal is a carbon-neutral, decentralized cloud storage provider. Seal's technical leadership brings decades of experience from traditional enterprise storage companies including Seagate and Oracle, as well as world-class experience on the Filecoin Network. Today, Seal operates data centers across the US and Canada with enterprise-grade infrastructure and data policies.

What is the primary source of funding for this project?

Seal is funding the project.

What other projects/ecosystem stakeholders is this project associated with?

None at this time.

Use-case details

Describe the data being stored onto Filecoin

The data sets are the original outputs of scientific experiments.

Where was the data in this dataset sourced from?

The data sets have been created by dark matter-related experiments and instrumentation.

Can you share a sample of the data? A link to a file, an image, a table, etc., are good ways to do this.

Yes. A link will be added shortly.

Confirm that this is a public dataset that can be retrieved by anyone on the Network (i.e., no specific permissions or access rights are required to view the data).

The current data set requires permission based access.

A goal of the pilot project is for Seal to work with our customer to provide a permission based model to access data. Staged data for access will be supported on IPFS, Seaweed FS Open Source tools.

What is the expected retrieval frequency for this data?

Archival is primary. The data will be accessed by external collaborators and Researchers.

For how long do you plan to keep this dataset stored on Filecoin?

Three years, at least.

DataCap allocation plan

In which geographies (countries, regions) do you plan on making storage deals?

We plan to store five copies of the 400 TiB data set [total of 2 PiB] in five different cities, in three different countries and across two continents.

How will you be distributing your data to storage providers? Is there an offline data transfer process?

Seal Storage has dual 100 Gbps internet connections. SPs will download data from Seal. Offline data transfer may be possible.

How do you plan on choosing the storage providers with whom you will be making deals? This should include a plan to ensure the data is retrievable in the future both by you and others.

We are currently discussing capabilities and performing due diligence with several SPs and have chosen three SPs for this project. We chose these based on their current storage capacity, compute capabilities, enterprise-grade DCs and bandwidth.

How will you be distributing deals across storage providers?

Holon, 400 TiB
ElioVP, 400 TiB
PikNik, 400 TiB
Seal, 800 TiB

Seal will also be keeping a hot copy (400 TB) for the Customer available for access.

The data ingestion will follow this approximate schedule:

55 TB right away
by the end of year 1: 5 TB
by end of year 2: 50 TB
by end of year 3: 290 TB

Do you have the resources/funding to start making deals as soon as you receive DataCap? What support from the community would help you onboard onto Filecoin?

Yes, we have the resources/funding to begin making deals once we receive DataCap. 

We currently have the support we need.
large-datacap-requests[bot] commented 2 years ago

Thanks for your request!

Heads up, you’re requesting more than the typical weekly onboarding rate of DataCap!
large-datacap-requests[bot] commented 2 years ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

salstorage commented 2 years ago

As per @dkkapur @galen-mcandrew @simonkim0515 request, original LDN #325 has been deprecated due to inactivity. Delay to #325 was due to internal 'deal API application' Seal Storage Technology is currently developing for this project to share with other SP's assigned to this project. Project is ready, we had already sealed 50TiB DC allocation as per application #325 Seal Storage Technology has been asked to resubmit the LDN in a new application

large-datacap-requests[bot] commented 2 years ago

Thanks for your request!

Heads up, you’re requesting more than the typical weekly onboarding rate of DataCap!
large-datacap-requests[bot] commented 2 years ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

salstorage commented 2 years ago

Paging @raghavrmadya The DC allocation Request has been approved. Checking DC on wallet ID f1rovtu5m3gq7q5vu4kfh4oiiif643gqq7voi4ida has status: unverified How do we get this resolved?

Screen Shot 2022-11-09 at 3 38 35 PM

galen-mcandrew commented 2 years ago

Seeing exit code 16 here: https://filfox.info/en/message/bafy2bzacea7konkezsnvvqezyu5mtbfgwfxbl7khr6uiamcxd4ilamtx6ztcu

I think that means the notary (in this case the v3 msig) does not have the amount of DataCap available to fill the proposal, even though the proposal was correctly approved by a second signature. The RKH are working to approve more DataCap the to v3 notary. Until that happens, proposals and approvals over the remaining notary balance will fail, and after the v3 gets more DataCap it will require two new notary messages (there's no way currently to point at these above messages and say "please accept those now")

raghavrmadya commented 2 years ago

@salstorage , looks like there was some confusion here. As the data is private, we would need you fill out the LDN exceptions template here - https://github.com/filecoin-project/notary-governance/issues/new/choose

raghavrmadya commented 2 years ago

Once it's up, we will create a Custom multisig with the supporting notaries and get this rolling based on past reputation and precedence

raghavrmadya commented 1 year ago

Custom LDN to be created by Gov team

large-datacap-requests[bot] commented 1 year ago

Thanks for your request!

Heads up, you’re requesting more than the typical weekly onboarding rate of DataCap!
large-datacap-requests[bot] commented 1 year ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

kevzak commented 1 year ago

Datacap Request Trigger

Total DataCap requested

2PiB

Expected weekly DataCap usage rate

400TiB

Client address

f1rovtu5m3gq7q5vu4kfh4oiiif643gqq7voi4ida

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Multisig Notary address

f01940930

Client address

f1rovtu5m3gq7q5vu4kfh4oiiif643gqq7voi4ida

DataCap allocation requested

102.39TiB

Id

a8aed019-c936-4c48-aa34-7a396ccf585f

large-datacap-requests[bot] commented 1 year ago

Hello @jamerduhgamer - @Fenbushi-Filecoin , please sign the datacap request

kevzak commented 1 year ago

Datacap Request Trigger

Total DataCap requested

2PiB

Expected weekly DataCap usage rate

400TiB

Client address

f1rovtu5m3gq7q5vu4kfh4oiiif643gqq7voi4ida

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Multisig Notary address

f01940930

Client address

f1rovtu5m3gq7q5vu4kfh4oiiif643gqq7voi4ida

DataCap allocation requested

102.39TiB

Id

b1fd85bb-5327-4989-b2ce-824bb8bcfbed

large-datacap-requests[bot] commented 1 year ago

Hello @MatrixStorage - @swatchliu , please sign the datacap request

kevzak commented 1 year ago

Hello - I can confirm a request from the Trust and Transparency WG https://filecoinproject.slack.com/archives/C0405HANNBT/p1670015428087609 that we are adding this application to the E-Fil+ Pilot. Notaries have been tagged. Please review the application and exceptions template for approval. Thank you.

raghavrmadya commented 1 year ago

Thanks for the trigger KZ. If community members and notaries have concerns, please post them as a comment on the application.

kevzak commented 1 year ago

Datacap Request Trigger

Total DataCap requested

2PiB

Expected weekly DataCap usage rate

400TiB

Client address

f1rovtu5m3gq7q5vu4kfh4oiiif643gqq7voi4ida

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Multisig Notary address

f01940930

Client address

f1rovtu5m3gq7q5vu4kfh4oiiif643gqq7voi4ida

DataCap allocation requested

102.39TiB

Id

af1d4e7c-8e19-43e2-92cc-0fce0b26aa5e

large-datacap-requests[bot] commented 1 year ago

Hello @Fenbushi-Filecoin - @swatchliu , please sign the datacap request

cryptowhizzard commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzaceappxtau2ecxvwhdcy74gbkmwxxq4qeja3dbfn3og55acfcyfq2ji

Address

f1rovtu5m3gq7q5vu4kfh4oiiif643gqq7voi4ida

Datacap Allocated

102.39TiB

Signer Address

f1krmypm4uoxxf3g7okrwtrahlmpcph3y7rbqqgfa

Id

af1d4e7c-8e19-43e2-92cc-0fce0b26aa5e

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceappxtau2ecxvwhdcy74gbkmwxxq4qeja3dbfn3og55acfcyfq2ji

Fenbushi-Filecoin commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaced4d6jqekag4dhkspxw5z4you25z7nub32btjy5vaf6uy4w4pcqsg

Address

f1rovtu5m3gq7q5vu4kfh4oiiif643gqq7voi4ida

Datacap Allocated

102.39TiB

Signer Address

f1yqydpmqb5en262jpottko2kd65msajax7fi4rmq

Id

af1d4e7c-8e19-43e2-92cc-0fce0b26aa5e

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaced4d6jqekag4dhkspxw5z4you25z7nub32btjy5vaf6uy4w4pcqsg

filplus-checker commented 1 year ago

DataCap and CID Checker Report[^1]

If this is the first time a provider takes verified deal, it will be marked as new.

For most of the datacap application, below restrictions should apply.

⚠️ f01392893 has sealed 52.33% of total datacap.

⚠️ f01886710 has sealed 35.67% of total datacap.

⚠️ f01886710 has unknown IP location.

Provider Location Total Deals Sealed Percentage Unique Data Duplicate Deals
f01392893 Amsterdam, North Holland, NL 4.70 TiB 52.33% 4.70 TiB 0.00%
f01886710 Unknown 3.20 TiB 35.67% 3.20 TiB 0.00%
f01154023 Sydney, New South Wales, AU 1.08 TiB 12.01% 1.08 TiB 0.00%

Provider Distribution

Deal Data Replication

The below table shows how each many unique data are replicated across storage providers.

⚠️ 100.00% of deals are for data replicated across less than 4 storage providers.

Unique Data Size Total Deals Made Number of Providers Deal Percentage
6.61 TiB 6.61 TiB 1 73.55%
1.14 TiB 2.28 TiB 2 25.40%
32.00 GiB 96.00 GiB 3 1.04%

Replication Distribution

Deal Data Shared with other Clients

The below table shows how many unique data are shared with other clients. Usually different applications owns different data and should not resolve to the same CID.

⚠️ CID sharing has been observed.

Other Client Application Total Deals Affected Unique CIDs Verifier
f1usscfxtogr5v4jmi32uzkckeql2mgvun72q37ga Seal Storage Technology 13.43 TiB 190 LDN # 325

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 4

Multisig Notary address

f01940930

Client address

f1rovtu5m3gq7q5vu4kfh4oiiif643gqq7voi4ida

DataCap allocation requested

819.19TiB

Id

b7465293-b348-4f58-b269-bb64a7f9cef1

large-datacap-requests[bot] commented 1 year ago

Hello @IPFSUnion - @flyworker , please sign the datacap request

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f01940930

Client address

f1rovtu5m3gq7q5vu4kfh4oiiif643gqq7voi4ida

Last two approvers

Fenbushi-Filecoin & cryptowhizzard

Rule to calculate the allocation request amount

40% of total dc amount requested

DataCap allocation requested

819.19TiB

Total DataCap granted for client so far

102.38TiB

Datacap to be granted to reach the total amount requested by the client (2PiB)

1.90PiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
2496 6 102.39TiB 31.16 22.56TiB
filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report[^1]

Storage Provider Distribution

The below table shows the distribution of storage providers that have stored data for this client.

If this is the first time a provider takes verified deal, it will be marked as new.

For most of the datacap application, below restrictions should apply.

⚠️ f01345523 has sealed 31.31% of total datacap.

⚠️ f01392893 has sealed 30.61% of total datacap.

⚠️ f01886710 has unknown IP location.

⚠️ f01923554 has unknown IP location.

Provider Location Total Deals Sealed Percentage Unique Data Duplicate Deals
f01345523 Antwerpen, Flanders, BE
Cogent Communications
23.21 TiB 31.31% 23.21 TiB 0.00%
f01392893 Amsterdam, North Holland, NL
Fusix Networks B.V.
22.68 TiB 30.61% 22.68 TiB 0.00%
f01886710 Unknown
Unknown
14.58 TiB 19.67% 13.86 TiB 4.93%
f01154023 Sydney, New South Wales, AU
Anycast Global Backbone
3.23 TiB 4.36% 3.23 TiB 0.00%
f01873432 Las Vegas, Nevada, US
PiKNiK & Company Inc.
9.30 TiB 12.54% 9.30 TiB 0.00%
f01923554 Unknown
Unknown
1.11 TiB 1.50% 1.11 TiB 0.00%

Provider Distribution

Deal Data Replication

The below table shows how each many unique data are replicated across storage providers.

⚠️ 99.66% of deals are for data replicated across less than 4 storage providers.

Unique Data Size Total Deals Made Number of Providers Deal Percentage
25.40 TiB 25.43 TiB 1 34.32%
15.60 TiB 31.60 TiB 2 42.64%
5.52 TiB 16.83 TiB 3 22.71%
64.00 GiB 256.00 GiB 4 0.34%

Replication Distribution

Deal Data Shared with other Clients

The below table shows how many unique data are shared with other clients. Usually different applications owns different data and should not resolve to the same CID.

However, this could be possible if all below clients use same software to prepare for the exact same dataset or they belong to a series of LDN applications for the same dataset.

⚠️ CID sharing has been observed.

Other Client Application Total Deals Affected Unique CIDs Approvers
f1usscfxtogr5v4jmi32uzkckeql2mgvun72q37ga Seal Storage Technology 31.32 TiB 495 1dannyob
1TimWilliams00

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

flyworker commented 1 year ago

i have difficulties reading the DataCap and CID Checker Report Will there be some guidelines for that? e.g. standards for Fil+ E vs Fil+

kevzak commented 1 year ago

CID Checker info is standard for any application, public or private

flyworker commented 1 year ago

For most of the datacap application, below restrictions should apply.

Please explain why ⚠️ f01392893 has sealed 52.33% of total datacap.

⚠️ f01886710 has sealed 35.67% of total datacap.

kevzak commented 1 year ago

Notes from slack converation: Charles Cao - FilSwan because i think some FIL+ E is allowed for store less than 3 nodes

Charles Cao - FilSwan For most of the datacap application, below restrictions should apply. Storage provider should not exceed 30% of total datacap. Storage provider should not be storing duplicate data for more than 20%. Storage provider should have published its public IP address. All storage providers should be located in different regions.

Charles Cao - FilSwan

if the standard is the same, this request is violated

Kevin Z No, it's not different. However, I do agree with you that E-Fil will generally have 3-5 SPs storing and each node will store one copy before the next. So you might see only one SP until a full copy is completed.

Kevin Z

You will likely not see >30% distribution until all copies are completed, this will be something to take note of (edited)

Kevin Z

Feel free to ask the client applicant if you have questions about their storage plan. It should be stated clearly

Kevin Z From the application: Holon, 400 TiB ElioVP, 400 TiB PikNik, 400 TiB Seal, 800 TiB Seal will also be keeping a hot copy (400 TB) for the Customer available for access. The data ingestion will follow this approximate schedule: 55 TB right away by the end of year 1: 5 TB by end of year 2: 50 TB by end of year 3: 290 TB

kevzak commented 1 year ago

For most of the datacap application, below restrictions should apply.

  • Storage provider should not exceed 30% of total datacap.
  • Storage provider should not be storing duplicate data for more than 20%.
  • Storage provider should have published its public IP address.
  • All storage providers should be located in different regions.

Please explain why ⚠️ f01392893 has sealed 52.33% of total datacap.

⚠️ f01886710 has sealed 35.67% of total datacap.

@salstorage can you explain current storage status stats and why its not currently split as recommended? Thanks

salstorage commented 1 year ago

@flyworker the numbers according to latest stats the breakdown is as follows:

f01345523 | Antwerpen, Flanders, BECogent Communications | 23.21 TiB | 31.31% f01392893 | Amsterdam, North Holland, NLFusix Networks B.V. | 22.68 TiB | 30.61% f01886710 | UnknownUnknown | 14.58 TiB | 19.67% f01154023 | Sydney, New South Wales, AUAnycast Global Backbone | 3.23 TiB | 4.36% f01873432 | Las Vegas, Nevada, USPiKNiK & Company Inc. | 9.30 TiB | 12.54% f01923554 | UnknownUnknown | 1.11 TiB | 1.50%

As a general explanation, we are using a deal distribution API for SP's to ensure Deals are within 30% of total DC. During December/NewYear 2023 break, some SP noted they we taking miners offline (maintenance etc) and this likely caused the the 30% breach.

kevzak commented 1 year ago

Thanks @salstorage and actually @flyworker I think you were looking at an older CID Checker report from last month. The most recent is here reflecting sal's stats: https://github.com/filecoin-project/filecoin-plus-large-datasets/issues/1212#issuecomment-1374905000

salstorage commented 1 year ago

@kevzak

Paging @IPFSUnion - @flyworker , please sign the datacap request

flyworker commented 1 year ago

i am waiting for the fil plus website available

galen-mcandrew commented 1 year ago

I can load the registry app and see this request, but I'm not sure if the error hits when attempting to sign. Screen Shot 2023-01-19 at 9 50 34 AM

UnionLabs2020 commented 1 year ago

Could you explain why these two nodes cannot be connected? f01923554 and f01886710

flyworker commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacebrpjdiirn7mp2jscmrzl4pt4b2ur3jhngaygln33w6bnas3gsms6

Address

f1rovtu5m3gq7q5vu4kfh4oiiif643gqq7voi4ida

Datacap Allocated

819.19TiB

Signer Address

f1hlubjsdkv4wmsdadihloxgwrz3j3ernf6i3cbpy

Id

b7465293-b348-4f58-b269-bb64a7f9cef1

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebrpjdiirn7mp2jscmrzl4pt4b2ur3jhngaygln33w6bnas3gsms6

kevzak commented 1 year ago

Could you explain why these two nodes cannot be connected? f01923554 and f01886710

@salstorage can you reply to @UnionLabs2020?

salstorage commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report[^1]

Storage Provider Distribution

The below table shows the distribution of storage providers that have stored data for this client.

If this is the first time a provider takes verified deal, it will be marked as new.

For most of the datacap application, below restrictions should apply.

⚠️ f01886710 has sealed 35.80% of total datacap.

⚠️ f01886710 has unknown IP location.

⚠️ f01923554 has unknown IP location.

Provider Location Total Deals Sealed Percentage Unique Data Duplicate Deals
f01886710 Unknown
Unknown
33.25 TiB 35.80% 32.53 TiB 2.16%
f01154023 Melbourne, Victoria, AU
Anycast Global Backbone
3.23 TiB 3.48% 3.23 TiB 0.00%
f01345523 Antwerpen, Flanders, BE
Cogent Communications
23.21 TiB 24.99% 23.21 TiB 0.00%
f01392893 Amsterdam, North Holland, NL
Fusix Networks B.V.
22.78 TiB 24.52% 22.78 TiB 0.00%
f01873432 Las Vegas, Nevada, US
PiKNiK & Company Inc.
9.30 TiB 10.01% 9.30 TiB 0.00%
f01923554 Unknown
Unknown
1.11 TiB 1.19% 1.11 TiB 0.00%

Provider Distribution

Deal Data Replication

The below table shows how each many unique data are replicated across storage providers.

⚠️ 99.73% of deals are for data replicated across less than 4 storage providers.

Unique Data Size Total Deals Made Number of Providers Deal Percentage
8.59 TiB 8.62 TiB 1 9.28%
30.74 TiB 61.88 TiB 2 66.63%
7.28 TiB 22.13 TiB 3 23.82%
64.00 GiB 256.00 GiB 4 0.27%

Replication Distribution

Deal Data Shared with other Clients

The below table shows how many unique data are shared with other clients. Usually different applications owns different data and should not resolve to the same CID.

However, this could be possible if all below clients use same software to prepare for the exact same dataset or they belong to a series of LDN applications for the same dataset.

⚠️ CID sharing has been observed.

Other Client Application Total Deals Affected Unique CIDs Approvers
f1usscfxtogr5v4jmi32uzkckeql2mgvun72q37ga Seal Storage Technology 31.32 TiB 495 1dannyob
1TimWilliams00

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

salstorage commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report[^1]

Storage Provider Distribution

The below table shows the distribution of storage providers that have stored data for this client.

If this is the first time a provider takes verified deal, it will be marked as new.

For most of the datacap application, below restrictions should apply.

⚠️ f01886710 has sealed 35.80% of total datacap.

⚠️ f01923554 has unknown IP location.

Provider Location Total Deals Sealed Percentage Unique Data Duplicate Deals
f01154023 Melbourne, Victoria, AU
Anycast Global Backbone
3.23 TiB 3.48% 3.23 TiB 0.00%
f01345523 Antwerpen, Flanders, BE
Cogent Communications
23.21 TiB 24.99% 23.21 TiB 0.00%
f01392893 Amsterdam, North Holland, NL
Fusix Networks B.V.
22.78 TiB 24.52% 22.78 TiB 0.00%
f01886710 Las Vegas, Nevada, US
GTT Communications Inc.
33.25 TiB 35.80% 32.53 TiB 2.16%
f01873432 Las Vegas, Nevada, US
PiKNiK & Company Inc.
9.30 TiB 10.01% 9.30 TiB 0.00%
f01923554 Unknown
Unknown
1.11 TiB 1.19% 1.11 TiB 0.00%

Provider Distribution

Deal Data Replication

The below table shows how each many unique data are replicated across storage providers.

⚠️ 99.73% of deals are for data replicated across less than 4 storage providers.

Unique Data Size Total Deals Made Number of Providers Deal Percentage
8.59 TiB 8.62 TiB 1 9.28%
30.74 TiB 61.88 TiB 2 66.63%
7.28 TiB 22.13 TiB 3 23.82%
64.00 GiB 256.00 GiB 4 0.27%

Replication Distribution

Deal Data Shared with other Clients

The below table shows how many unique data are shared with other clients. Usually different applications owns different data and should not resolve to the same CID.

However, this could be possible if all below clients use same software to prepare for the exact same dataset or they belong to a series of LDN applications for the same dataset.

⚠️ CID sharing has been observed.

Other Client Application Total Deals Affected Unique CIDs Approvers
f1usscfxtogr5v4jmi32uzkckeql2mgvun72q37ga Seal Storage Technology 31.32 TiB 495 1dannyob
1TimWilliams00

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

salstorage commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report[^1]

Storage Provider Distribution

The below table shows the distribution of storage providers that have stored data for this client.

If this is the first time a provider takes verified deal, it will be marked as new.

For most of the datacap application, below restrictions should apply.

⚠️ f01886710 has sealed 35.80% of total datacap.

Provider Location Total Deals Sealed Percentage Unique Data Duplicate Deals
f01154023 Melbourne, Victoria, AU
Anycast Global Backbone
3.23 TiB 3.48% 3.23 TiB 0.00%
f01345523 Antwerpen, Flanders, BE
Cogent Communications
23.21 TiB 24.99% 23.21 TiB 0.00%
f01392893 Amsterdam, North Holland, NL
Fusix Networks B.V.
22.78 TiB 24.52% 22.78 TiB 0.00%
f01886710 Las Vegas, Nevada, US
GTT Communications Inc.
33.25 TiB 35.80% 32.53 TiB 2.16%
f01923554 Las Vegas, Nevada, US
GTT Communications Inc.
1.11 TiB 1.19% 1.11 TiB 0.00%
f01873432 Las Vegas, Nevada, US
PiKNiK & Company Inc.
9.30 TiB 10.01% 9.30 TiB 0.00%

Provider Distribution

Deal Data Replication

The below table shows how each many unique data are replicated across storage providers.

⚠️ 99.73% of deals are for data replicated across less than 4 storage providers.

Unique Data Size Total Deals Made Number of Providers Deal Percentage
8.59 TiB 8.62 TiB 1 9.28%
30.74 TiB 61.88 TiB 2 66.63%
7.28 TiB 22.13 TiB 3 23.82%
64.00 GiB 256.00 GiB 4 0.27%

Replication Distribution

Deal Data Shared with other Clients

The below table shows how many unique data are shared with other clients. Usually different applications owns different data and should not resolve to the same CID.

However, this could be possible if all below clients use same software to prepare for the exact same dataset or they belong to a series of LDN applications for the same dataset.

⚠️ CID sharing has been observed.

Other Client Application Total Deals Affected Unique CIDs Approvers
f1usscfxtogr5v4jmi32uzkckeql2mgvun72q37ga Seal Storage Technology 31.32 TiB 495 1dannyob
1TimWilliams00

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

salstorage commented 1 year ago

Could you explain why these two nodes cannot be connected? f01923554 and f01886710

@salstorage can you reply to @UnionLabs2020?

@UnionLabs2020 @Kevin-FF-USA

We have resolved the location ID's, we believe it was a bug in Lotus and we had to run actor set-addrs on miner to fix it Can we please get this signed?