filecoin-project / filecoin-plus-large-datasets

Hub for client applications for DataCap at a large scale
110 stars 62 forks source link

Speedium | Dcent - Galaxy Evolution Explorer Satellite dataset - Part 3/3 #1555

Closed cryptowhizzard closed 1 year ago

cryptowhizzard commented 1 year ago

Data Owner Name

NASA

Data Owner Country/Region

United States

Data Owner Industry

Environment

Website

https://www.jpl.nasa.gov/missions/galaxy-evolution-explorer-galex

Social Media

https://www.jpl.nasa.gov/missions/galaxy-evolution-explorer-galex

Total amount of DataCap being requested

5PiB

Weekly allocation of DataCap requested

500TiB

On-chain address for first allocation

f1kyaa43uqypbcsv5isyd4xw7lvgjcumfzqsxag7y

Custom multisig

Identifier

No response

Share a brief history of your project and organization

Since its launch, the Filecoin network has become an important player in the decentralised storage space, offering a secure and transparent alternative to traditional data storage solutions.

We as Speedium / DCENT have been engaged with storing real and valuable datasets on the Filecoin network since Slingshot 2.6 and have been actively developing tools to improve the process. We are always on the lookout for new and useful client data to onboard.

Is this project associated with other projects/ecosystem stakeholders?

No

If answered yes, what are the other projects/ecosystem stakeholders

No response

Describe the data being stored onto Filecoin

The Galaxy Evolution Explorer Satellite (GALEX) was a NASA mission led by the California Institute of Technology, whose primary goal was to investigates how star formation in galaxies evolved from the early Universe up to the present. GALEX used microchannel plate detectors to obtain direct images in the near-UV (NUV) and far-UV (FUV), and a grism to disperse light for low resolution spectroscopy. More information about GALEX is available at MAST

This project is a followup of #487. 487 has been granted 4 PB of datacap but given that we need around 800 TiB of datacap per replica we are at 50%.

Where was the data currently stored in this dataset sourced from

AWS Cloud

If you answered "Other" in the previous question, enter the details here

No response

How do you plan to prepare the dataset

IPFS, lotus, singularity, graphsplit

If you answered "other/custom tool" in the previous question, enter the details here

No response

Please share a sample of the data

https://www.jpl.nasa.gov/missions/galaxy-evolution-explorer-galex

Confirm that this is a public dataset that can be retrieved by anyone on the Network

If you chose not to confirm, what was the reason

No response

What is the expected retrieval frequency for this data

Yearly

For how long do you plan to keep this dataset stored on Filecoin

1.5 to 2 years

In which geographies do you plan on making storage deals

Greater China, Asia other than Greater China, North America, South America, Europe, Australia (continent)

How will you be distributing your data to storage providers

HTTP or FTP server, IPFS, Shipping hard drives, Lotus built-in data transfer

How do you plan to choose storage providers

Slack, Big data exchange, Partners

If you answered "Others" in the previous question, what is the tool or platform you plan to use

No response

If you already have a list of storage providers to work with, fill out their names and provider IDs below

MinerID City Continent Business/Entity
f01944347 Oregon USA Jenny, Dabai
f01952350 Oregon USA Jenny, Dabai
f01972364 Oregon USA Jenny, Dabai
f01972376 Oregon USA Jenny, Dabai
f02000937 Chengdu CN MTY
f01915033 Chengdu CN MTY
f01901765 Hangzhou CN MTY
f01953985 Shenzhen CN MTY
f0120**** Melbourne AU HOLON
f0115**** Melbourne AU HOLON
f01199430 Heerhugowaard EU DCENT
f01786387 Heerhugowaard EU DCENT
f01201327 Heerhugowaard EU DCENT
f01937642 Heerhugowaard EU DCENT
f0198**** Dallas USA GREATERHEAT
f0188**** Singapore AS GREATERHEAT
f01091851 Omaha USA DLTx
f01736668 Omaha USA DLTx
f01820744 Omaha USA DLTx
f0855584 Omaha USA DLTx
f01794610 Omaha USA DLTx
f01838599 Kansas City USA DLTx
f01845552 Kansas City USA DLTx
f01885088 Hong Kong AS Hong Kong Morpho Technology Limited
f01926635 Hong Kong AS Hong Kong Morpho Technology Limited
f01933917 Hong Kong AS Hong Kong Morpho Technology Limited
f01989888 Hong Kong AS Hong Kong Morpho Technology Limited

How do you plan to make deals to your storage providers

Boost client, Lotus client, Singularity

If you answered "Others/custom tool" in the previous question, enter the details here

No response

Can you confirm that you will follow the Fil+ guideline

Yes

large-datacap-requests[bot] commented 1 year ago

Thanks for your request!

Heads up, you’re requesting more than the typical weekly onboarding rate of DataCap!
large-datacap-requests[bot] commented 1 year ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

cryptowhizzard commented 1 year ago

(Proposal https://github.com/filecoin-project/filecoin-plus-large-datasets/issues/1482 is broken this is a re-apply of that request)

simonkim0515 commented 1 year ago

Datacap Request Trigger

Total DataCap requested

5PiB

Expected weekly DataCap usage rate

500TiB

Client address

f1kyaa43uqypbcsv5isyd4xw7lvgjcumfzqsxag7y

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Multisig Notary address

f02049625

Client address

f1kyaa43uqypbcsv5isyd4xw7lvgjcumfzqsxag7y

DataCap allocation requested

250TiB

Id

e29f9f49-300c-480e-a37d-5d7d325cbfa2

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report[^1]

There is no previous allocation for this issue.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

mjroddy commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacec5vdqxhm7d5nyngjqcy3iyudms25y5ekm7e2gvjkcjjog3p4dwqg

Address

f1kyaa43uqypbcsv5isyd4xw7lvgjcumfzqsxag7y

Datacap Allocated

250.00TiB

Signer Address

f1ystxl2ootvpirpa7ebgwl7vlhwkbx2r4zjxwe5i

Id

e29f9f49-300c-480e-a37d-5d7d325cbfa2

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacec5vdqxhm7d5nyngjqcy3iyudms25y5ekm7e2gvjkcjjog3p4dwqg

kernelogic commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacedir4wepesekcm726k5ex6sdurnmycs6kfkoivpe32eo6zt3s3aua

Address

f1kyaa43uqypbcsv5isyd4xw7lvgjcumfzqsxag7y

Datacap Allocated

250.00TiB

Signer Address

f1yjhnsoga2ccnepb7t3p3ov5fzom3syhsuinxexa

Id

e29f9f49-300c-480e-a37d-5d7d325cbfa2

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedir4wepesekcm726k5ex6sdurnmycs6kfkoivpe32eo6zt3s3aua

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 2

Multisig Notary address

f02049625

Client address

f1kyaa43uqypbcsv5isyd4xw7lvgjcumfzqsxag7y

DataCap allocation requested

500TiB

Id

4dae2fb2-ee7e-4a07-a78d-089338bb7844

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f01858410

Client address

f1kyaa43uqypbcsv5isyd4xw7lvgjcumfzqsxag7y

Last two approvers

kernelogic & megtei

Rule to calculate the allocation request amount

100% of weekly dc amount requested

DataCap allocation requested

500TiB

Total DataCap granted for client so far

32GiB

Datacap to be granted to reach the total amount requested by the client (5PiB)

4.99PiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
0 0 250TiB 0 42.49GiB
filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report[^1]

Storage Provider Distribution

The below table shows the distribution of storage providers that have stored data for this client.

If this is the first time a provider takes verified deal, it will be marked as new.

For most of the datacap application, below restrictions should apply.

Since this is the 3rd allocation, the following restrictions have been relaxed:

⚠️ 28.03% of total deal sealed by f022352 are duplicate data.

⚠️ 21.61% of total deal sealed by f0855584 are duplicate data.

⚠️ 68.57% of total deal sealed by f01385207 are duplicate data.

⚠️ 50.12% of total deal sealed by f01199430 are duplicate data.

⚠️ 41.64% of total deal sealed by f01720359 are duplicate data.

⚠️ 61.24% of total deal sealed by f01771403 are duplicate data.

⚠️ 31.56% of total deal sealed by f01937642 are duplicate data.

Provider Location Total Deals Sealed Percentage Unique Data Duplicate Deals
f022352 Oslo, Oslo, NO
Blix Solutions AS
326.11 TiB 8.04% 234.70 TiB 28.03%
f01240218new Chengdu, Sichuan, CN
China Mobile Communications Group Co., Ltd.
53.88 TiB 1.33% 53.88 TiB 0.00%
f01907545 Xi’an, Shaanxi, CN
China Telecom (Group)
145.06 TiB 3.57% 145.06 TiB 0.00%
f01989866new Xi’an, Shaanxi, CN
China Telecom (Group)
63.84 TiB 1.57% 63.84 TiB 0.00%
f01901765 Jiaxing, Zhejiang, CN
CHINANET-BACKBONE
674.61 TiB 16.62% 674.61 TiB 0.00%
f01906874new Plano, Texas, US
Cogent Communications
225.58 TiB 5.56% 225.58 TiB 0.00%
f01345523 Antwerpen, Flanders, BE
Cogent Communications
216.68 TiB 5.34% 213.55 TiB 1.45%
f01392893 Amsterdam, North Holland, NL
Fusix Networks B.V.
360.85 TiB 8.89% 352.12 TiB 2.42%
f01926686 Hangzhou, Zhejiang, CN
Jiangxi Jiujiang IDC
86.74 TiB 2.14% 86.74 TiB 0.00%
f01908299new Lincoln, Nebraska, US
LightEdge Solutions
216.69 TiB 5.34% 174.74 TiB 19.36%
f0855584 Lincoln, Nebraska, US
LightEdge Solutions
211.97 TiB 5.22% 166.16 TiB 21.61%
f01091851 Lincoln, Nebraska, US
LightEdge Solutions
151.74 TiB 3.74% 151.74 TiB 0.00%
f01736668new Lincoln, Nebraska, US
LightEdge Solutions
128.75 TiB 3.17% 128.75 TiB 0.00%
f01385207 Lincoln, Nebraska, US
LightEdge Solutions
1.09 TiB 0.03% 352.00 GiB 68.57%
f01933917 Hong Kong, Central and Western, HK
OneAsia Network Limited
449.09 TiB 11.07% 447.77 TiB 0.29%
f01926635 Hong Kong, Central and Western, HK
OneAsia Network Limited
296.30 TiB 7.30% 296.30 TiB 0.00%
f01885088 Hong Kong, Central and Western, HK
OneAsia Network Limited
130.97 TiB 3.23% 130.97 TiB 0.00%
f01199430 Heerhugowaard, North Holland, NL
Wijnand Schouten trading as Speedium
113.86 TiB 2.81% 56.80 TiB 50.12%
f01720359 Heerhugowaard, North Holland, NL
Wijnand Schouten trading as Speedium
95.00 TiB 2.34% 55.44 TiB 41.64%
f01786387 Heerhugowaard, North Holland, NL
Wijnand Schouten trading as Speedium
89.79 TiB 2.21% 89.79 TiB 0.00%
f01771403 Heerhugowaard, North Holland, NL
Wijnand Schouten trading as Speedium
10.56 TiB 0.26% 4.09 TiB 61.24%
f01937642 Heerhugowaard, North Holland, NL
Wijnand Schouten trading as Speedium
9.41 TiB 0.23% 6.44 TiB 31.56%

Provider Distribution

Deal Data Replication

The below table shows how each many unique data are replicated across storage providers.

Since this is the 3rd allocation, the following restrictions have been relaxed:

✔️ Data replication looks healthy.

Unique Data Size Total Deals Made Number of Providers Deal Percentage
49.67 TiB 50.08 TiB 1 1.23%
198.93 TiB 399.30 TiB 2 9.84%
99.62 TiB 301.56 TiB 3 7.43%
128.73 TiB 517.63 TiB 4 12.75%
145.50 TiB 730.29 TiB 5 17.99%
114.63 TiB 690.66 TiB 6 17.02%
55.58 TiB 397.41 TiB 7 9.79%
42.83 TiB 374.53 TiB 8 9.23%
20.95 TiB 243.77 TiB 9 6.01%
13.31 TiB 231.72 TiB 10 5.71%
1.75 TiB 79.44 TiB 11 1.96%
800.00 GiB 38.72 TiB 12 0.95%
64.00 GiB 3.47 TiB 13 0.09%

Replication Distribution

Deal Data Shared with other Clients

The below table shows how many unique data are shared with other clients. Usually different applications owns different data and should not resolve to the same CID.

However, this could be possible if all below clients use same software to prepare for the exact same dataset or they belong to a series of LDN applications for the same dataset.

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

xinaxu commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzaceavbwx6xdbbsbllkyuow2cmiu4276sd5yvf7jaldu7szmmp72iylk

Address

f1kyaa43uqypbcsv5isyd4xw7lvgjcumfzqsxag7y

Datacap Allocated

500.00TiB

Signer Address

f1k3ysofkrrmqcot6fkx4wnezpczlltpirmrpsgui

Id

4dae2fb2-ee7e-4a07-a78d-089338bb7844

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceavbwx6xdbbsbllkyuow2cmiu4276sd5yvf7jaldu7szmmp72iylk

mjroddy commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaceb2b3dnsa5m7aagr2dgtw64qwia27cg3iiazua7rdni7ky2olrmt2

Address

f1kyaa43uqypbcsv5isyd4xw7lvgjcumfzqsxag7y

Datacap Allocated

500.00TiB

Signer Address

f1ystxl2ootvpirpa7ebgwl7vlhwkbx2r4zjxwe5i

Id

4dae2fb2-ee7e-4a07-a78d-089338bb7844

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceb2b3dnsa5m7aagr2dgtw64qwia27cg3iiazua7rdni7ky2olrmt2

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 3

Multisig Notary address

f02049625

Client address

f1kyaa43uqypbcsv5isyd4xw7lvgjcumfzqsxag7y

DataCap allocation requested

1000.0TiB

Id

ed574fb9-8130-4fdf-83a5-8687e45b9595

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f01858410

Client address

f1kyaa43uqypbcsv5isyd4xw7lvgjcumfzqsxag7y

Last two approvers

megtei & xinaxu

Rule to calculate the allocation request amount

200% of weekly dc amount requested

DataCap allocation requested

1000.0TiB

Total DataCap granted for client so far

4.26PiB

Datacap to be granted to reach the total amount requested by the client (5PiB)

750.01TiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
160661 22 500TiB 15.68 10.49GiB
filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report[^1]

Storage Provider Distribution

The below table shows the distribution of storage providers that have stored data for this client.

If this is the first time a provider takes verified deal, it will be marked as new.

For most of the datacap application, below restrictions should apply.

⚠️ 28.03% of total deal sealed by f022352 are duplicate data.

⚠️ 21.61% of total deal sealed by f0855584 are duplicate data.

⚠️ 68.57% of total deal sealed by f01385207 are duplicate data.

⚠️ 50.12% of total deal sealed by f01199430 are duplicate data.

⚠️ 41.64% of total deal sealed by f01720359 are duplicate data.

⚠️ 61.24% of total deal sealed by f01771403 are duplicate data.

⚠️ 31.56% of total deal sealed by f01937642 are duplicate data.

Provider Location Total Deals Sealed Percentage Unique Data Duplicate Deals
f022352 Oslo, Oslo, NO
Blix Solutions AS
326.11 TiB 7.57% 234.70 TiB 28.03%
f01240218new Chengdu, Sichuan, CN
China Mobile Communications Group Co., Ltd.
53.88 TiB 1.25% 53.88 TiB 0.00%
f01907545 Xi’an, Shaanxi, CN
China Telecom (Group)
145.06 TiB 3.37% 145.06 TiB 0.00%
f01989866new Xi’an, Shaanxi, CN
China Telecom (Group)
63.84 TiB 1.48% 63.84 TiB 0.00%
f01901765 Jiaxing, Zhejiang, CN
CHINANET-BACKBONE
674.61 TiB 15.67% 674.61 TiB 0.00%
f01345523 Antwerpen, Flanders, BE
Cogent Communications
231.48 TiB 5.38% 228.34 TiB 1.36%
f01906874new Plano, Texas, US
Cogent Communications
225.58 TiB 5.24% 225.58 TiB 0.00%
f01392893 Amsterdam, North Holland, NL
Fusix Networks B.V.
373.30 TiB 8.67% 363.88 TiB 2.52%
f01926686 Hangzhou, Zhejiang, CN
Jiangxi Jiujiang IDC
152.20 TiB 3.53% 152.20 TiB 0.00%
f01908299new Lincoln, Nebraska, US
LightEdge Solutions
216.69 TiB 5.03% 174.74 TiB 19.36%
f0855584 Lincoln, Nebraska, US
LightEdge Solutions
211.97 TiB 4.92% 166.16 TiB 21.61%
f01091851 Lincoln, Nebraska, US
LightEdge Solutions
151.74 TiB 3.52% 151.74 TiB 0.00%
f01736668new Lincoln, Nebraska, US
LightEdge Solutions
128.75 TiB 2.99% 128.75 TiB 0.00%
f01385207 Lincoln, Nebraska, US
LightEdge Solutions
1.09 TiB 0.03% 352.00 GiB 68.57%
f01933917 Hong Kong, Central and Western, HK
OneAsia Network Limited
523.11 TiB 12.15% 521.80 TiB 0.25%
f01926635 Hong Kong, Central and Western, HK
OneAsia Network Limited
377.14 TiB 8.76% 377.14 TiB 0.00%
f01885088 Hong Kong, Central and Western, HK
OneAsia Network Limited
130.97 TiB 3.04% 130.97 TiB 0.00%
f01199430 Heerhugowaard, North Holland, NL
Wijnand Schouten trading as Speedium
113.86 TiB 2.64% 56.80 TiB 50.12%
f01720359 Heerhugowaard, North Holland, NL
Wijnand Schouten trading as Speedium
95.00 TiB 2.21% 55.44 TiB 41.64%
f01786387 Heerhugowaard, North Holland, NL
Wijnand Schouten trading as Speedium
89.79 TiB 2.09% 89.79 TiB 0.00%
f01771403 Heerhugowaard, North Holland, NL
Wijnand Schouten trading as Speedium
10.56 TiB 0.25% 4.09 TiB 61.24%
f01937642 Heerhugowaard, North Holland, NL
Wijnand Schouten trading as Speedium
9.41 TiB 0.22% 6.44 TiB 31.56%

Provider Distribution

Deal Data Replication

The below table shows how each many unique data are replicated across storage providers.

✔️ Data replication looks healthy.

Unique Data Size Total Deals Made Number of Providers Deal Percentage
44.10 TiB 44.48 TiB 1 1.03%
144.05 TiB 289.10 TiB 2 6.71%
85.09 TiB 256.23 TiB 3 5.95%
139.98 TiB 561.96 TiB 4 13.05%
177.57 TiB 892.37 TiB 5 20.72%
137.78 TiB 830.95 TiB 6 19.30%
63.77 TiB 455.16 TiB 7 10.57%
43.36 TiB 378.78 TiB 8 8.80%
20.95 TiB 243.77 TiB 9 5.66%
13.31 TiB 231.72 TiB 10 5.38%
1.75 TiB 79.44 TiB 11 1.84%
800.00 GiB 38.72 TiB 12 0.90%
64.00 GiB 3.47 TiB 13 0.08%

Replication Distribution

Deal Data Shared with other Clients

The below table shows how many unique data are shared with other clients. Usually different applications owns different data and should not resolve to the same CID.

However, this could be possible if all below clients use same software to prepare for the exact same dataset or they belong to a series of LDN applications for the same dataset.

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

NiwanDao commented 1 year ago

Greeting! The geographic distribution of Data looks good to me. Can you share your plan on the number of duplicates and the size in Datacap of a single copy?

herrehesse commented 1 year ago

Hello @xingjitansuo ! Thank you for your question.

Initially we have stored a bunch of CID duplicates due to bugs in singularity (#487). This has now been resolved by PL and our database has been purged of any duplicate CID records.

The plan is to distribute 10 to 12 full copies of the dataset, which comprises 500 TiB of carfiles. To store all of these files, approximately 900 TiB of datacap is needed per copy.

The total datacap required for 12 copies is 10.8 PiB, with the third tranche serving as a backup for any errors or duplicates and for the possibility of an SP slashing sectors. It should be noted that any unused datacap will not be refunded to our wallet. Measures will be taken to ensure that all copies are stored on-chain and the datacap from the third tranche will be reclaimed upon completion if not used.

Let me know if I can answer some more questions! Here to help.

kernelogic commented 1 year ago

@herrehesse you can use lotus filplus remove-expired-allocations command to recover DC caused by expiration (slashed I'm not sure).

kernelogic commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzaceaqujljuzezaq5kjm7tw47tftpxkpfyir6txsnhb3xllchzoocrrc

Address

f1kyaa43uqypbcsv5isyd4xw7lvgjcumfzqsxag7y

Datacap Allocated

1000.00TiB

Signer Address

f1yjhnsoga2ccnepb7t3p3ov5fzom3syhsuinxexa

Id

ed574fb9-8130-4fdf-83a5-8687e45b9595

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceaqujljuzezaq5kjm7tw47tftpxkpfyir6txsnhb3xllchzoocrrc

mjroddy commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacea3d4iybdstghk4vs6ttbppv3fckfaskz663nljjsw5c3f6pmnaka

Address

f1kyaa43uqypbcsv5isyd4xw7lvgjcumfzqsxag7y

Datacap Allocated

1000.00TiB

Signer Address

f1ystxl2ootvpirpa7ebgwl7vlhwkbx2r4zjxwe5i

Id

ed574fb9-8130-4fdf-83a5-8687e45b9595

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacea3d4iybdstghk4vs6ttbppv3fckfaskz663nljjsw5c3f6pmnaka

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 4

Multisig Notary address

f02049625

Client address

f1kyaa43uqypbcsv5isyd4xw7lvgjcumfzqsxag7y

DataCap allocation requested

1.95PiB

Id

3bf7d37c-3877-4107-a6be-883d4c7f40df

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f01858410

Client address

f1kyaa43uqypbcsv5isyd4xw7lvgjcumfzqsxag7y

Last two approvers

megtei & kernelogic

Rule to calculate the allocation request amount

400% of weekly dc amount requested

DataCap allocation requested

1.95PiB

Total DataCap granted for client so far

5.73PiB

Datacap to be granted to reach the total amount requested by the client (5PiB)

-824622725715722B

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
202025 26 1000.0TiB 12.86 230.99TiB
filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report[^1]

Storage Provider Distribution

The below table shows the distribution of storage providers that have stored data for this client.

If this is the first time a provider takes verified deal, it will be marked as new.

For most of the datacap application, below restrictions should apply.

⚠️ 28.03% of total deal sealed by f022352 are duplicate data.

⚠️ 21.64% of total deal sealed by f0855584 are duplicate data.

⚠️ 68.57% of total deal sealed by f01385207 are duplicate data.

⚠️ 50.12% of total deal sealed by f01199430 are duplicate data.

⚠️ 41.64% of total deal sealed by f01720359 are duplicate data.

⚠️ 61.24% of total deal sealed by f01771403 are duplicate data.

⚠️ 31.56% of total deal sealed by f01937642 are duplicate data.

Provider Location Total Deals Sealed Percentage Unique Data Duplicate Deals
f022352 Oslo, Oslo, NO
Blix Solutions AS
326.11 TiB 6.02% 234.70 TiB 28.03%
f01240218new Chengdu, Sichuan, CN
China Mobile Communications Group Co., Ltd.
53.88 TiB 0.99% 53.88 TiB 0.00%
f01907545 Xi’an, Shaanxi, CN
China Telecom (Group)
228.25 TiB 4.21% 228.25 TiB 0.00%
f01989866new Xi’an, Shaanxi, CN
China Telecom (Group)
63.84 TiB 1.18% 63.84 TiB 0.00%
f01901765 Jiaxing, Zhejiang, CN
CHINANET-BACKBONE
674.61 TiB 12.44% 674.61 TiB 0.00%
f01345523 Antwerpen, Flanders, BE
Cogent Communications
261.51 TiB 4.82% 254.12 TiB 2.83%
f01906874new Plano, Texas, US
Cogent Communications
225.58 TiB 4.16% 225.58 TiB 0.00%
f01992630 Dallas, Texas, US
Flexential Colorado Corp.
224.96 TiB 4.15% 224.96 TiB 0.00%
f01971600 Dallas, Texas, US
Flexential Colorado Corp.
206.30 TiB 3.81% 206.30 TiB 0.00%
f01392893 Amsterdam, North Holland, NL
Fusix Networks B.V.
403.52 TiB 7.44% 384.50 TiB 4.71%
f01896422 Fremont, California, US
Hurricane Electric LLC
14.55 TiB 0.27% 14.55 TiB 0.00%
f01926686 Hangzhou, Zhejiang, CN
Jiangxi Jiujiang IDC
258.75 TiB 4.77% 258.75 TiB 0.00%
f01908299new Lincoln, Nebraska, US
LightEdge Solutions
216.69 TiB 4.00% 174.74 TiB 19.36%
f0855584 Lincoln, Nebraska, US
LightEdge Solutions
211.66 TiB 3.90% 165.85 TiB 21.64%
f01091851 Lincoln, Nebraska, US
LightEdge Solutions
151.74 TiB 2.80% 151.74 TiB 0.00%
f01736668new Lincoln, Nebraska, US
LightEdge Solutions
128.75 TiB 2.38% 128.75 TiB 0.00%
f01385207 Lincoln, Nebraska, US
LightEdge Solutions
1.09 TiB 0.02% 352.00 GiB 68.57%
f01933917 Hong Kong, Central and Western, HK
OneAsia Network Limited
708.78 TiB 13.07% 706.34 TiB 0.34%
f01926635 Hong Kong, Central and Western, HK
OneAsia Network Limited
517.91 TiB 9.55% 517.91 TiB 0.00%
f01885088 Hong Kong, Central and Western, HK
OneAsia Network Limited
181.35 TiB 3.35% 181.35 TiB 0.00%
f01989888 Hong Kong, Central and Western, HK
OneAsia Network Limited
42.58 TiB 0.79% 42.58 TiB 0.00%
f01199430 Heerhugowaard, North Holland, NL
Wijnand Schouten trading as Speedium
113.86 TiB 2.10% 56.80 TiB 50.12%
f01720359 Heerhugowaard, North Holland, NL
Wijnand Schouten trading as Speedium
95.00 TiB 1.75% 55.44 TiB 41.64%
f01786387 Heerhugowaard, North Holland, NL
Wijnand Schouten trading as Speedium
89.79 TiB 1.66% 89.79 TiB 0.00%
f01771403 Heerhugowaard, North Holland, NL
Wijnand Schouten trading as Speedium
10.56 TiB 0.19% 4.09 TiB 61.24%
f01937642 Heerhugowaard, North Holland, NL
Wijnand Schouten trading as Speedium
9.41 TiB 0.17% 6.44 TiB 31.56%

Provider Distribution

Deal Data Replication

The below table shows how each many unique data are replicated across storage providers.

✔️ Data replication looks healthy.

Unique Data Size Total Deals Made Number of Providers Deal Percentage
18.25 TiB 18.25 TiB 1 0.34%
28.88 TiB 57.86 TiB 2 1.07%
85.65 TiB 257.51 TiB 3 4.75%
178.67 TiB 715.75 TiB 4 13.20%
124.23 TiB 623.56 TiB 5 11.50%
136.52 TiB 825.48 TiB 6 15.23%
136.03 TiB 962.34 TiB 7 17.75%
84.82 TiB 689.23 TiB 8 12.71%
51.53 TiB 483.72 TiB 9 8.92%
33.30 TiB 384.80 TiB 10 7.10%
13.41 TiB 235.84 TiB 11 4.35%
2.72 TiB 125.63 TiB 12 2.32%
768.00 GiB 37.53 TiB 13 0.69%
64.00 GiB 3.53 TiB 14 0.07%

Replication Distribution

Deal Data Shared with other Clients

The below table shows how many unique data are shared with other clients. Usually different applications owns different data and should not resolve to the same CID.

However, this could be possible if all below clients use same software to prepare for the exact same dataset or they belong to a series of LDN applications for the same dataset.

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

herrehesse commented 1 year ago

Kindly requesting the assistance of a Notary in signing the next tranche of the datacap. @kernelogic @s0nik42 @steven004 @GaryGJG @mjroddy @xinaxu @xingjitansuo

kernelogic commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacedryv57boabih6pmz4tgcycqrr6o7vmririmmr4czgewzqngfob5u

Address

f1kyaa43uqypbcsv5isyd4xw7lvgjcumfzqsxag7y

Datacap Allocated

1.95PiB

Signer Address

f1yjhnsoga2ccnepb7t3p3ov5fzom3syhsuinxexa

Id

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedryv57boabih6pmz4tgcycqrr6o7vmririmmr4czgewzqngfob5u

mjroddy commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaced3yb3a7caodeyrhhybcnhag6bppp63uivoxf7cw3rxnymn4hvf3y

Address

f1kyaa43uqypbcsv5isyd4xw7lvgjcumfzqsxag7y

Datacap Allocated

1.95PiB

Signer Address

f1ystxl2ootvpirpa7ebgwl7vlhwkbx2r4zjxwe5i

Id

3bf7d37c-3877-4107-a6be-883d4c7f40df

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaced3yb3a7caodeyrhhybcnhag6bppp63uivoxf7cw3rxnymn4hvf3y

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 5

Multisig Notary address

f02049625

Client address

f1kyaa43uqypbcsv5isyd4xw7lvgjcumfzqsxag7y

DataCap allocation requested

1.34PiB

Id

679a25cf-f96e-463d-b6a5-1c762000be62

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f01858410

Client address

f1kyaa43uqypbcsv5isyd4xw7lvgjcumfzqsxag7y

Last two approvers

megtei & kernelogic

Rule to calculate the allocation request amount

800% of weekly dc amount requested

DataCap allocation requested

1.34PiB

Total DataCap granted for client so far

5.73PiB

Datacap to be granted to reach the total amount requested by the client (5PiB)

-824622725715722B

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
213842 26 1.95PiB 12.74 7.99GiB
filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report[^1]

Storage Provider Distribution

The below table shows the distribution of storage providers that have stored data for this client.

If this is the first time a provider takes verified deal, it will be marked as new.

For most of the datacap application, below restrictions should apply.

⚠️ 28.03% of total deal sealed by f022352 are duplicate data.

⚠️ 21.64% of total deal sealed by f0855584 are duplicate data.

⚠️ 68.57% of total deal sealed by f01385207 are duplicate data.

⚠️ 50.12% of total deal sealed by f01199430 are duplicate data.

⚠️ 41.64% of total deal sealed by f01720359 are duplicate data.

⚠️ 61.24% of total deal sealed by f01771403 are duplicate data.

⚠️ 31.56% of total deal sealed by f01937642 are duplicate data.

Provider Location Total Deals Sealed Percentage Unique Data Duplicate Deals
f022352 Oslo, Oslo, NO
Blix Solutions AS
326.11 TiB 5.64% 234.70 TiB 28.03%
f01240218new Chengdu, Sichuan, CN
China Mobile Communications Group Co., Ltd.
53.88 TiB 0.93% 53.88 TiB 0.00%
f01907545 Xi’an, Shaanxi, CN
China Telecom (Group)
228.25 TiB 3.95% 228.25 TiB 0.00%
f01989866new Xi’an, Shaanxi, CN
China Telecom (Group)
63.84 TiB 1.10% 63.84 TiB 0.00%
f01901765 Jiaxing, Zhejiang, CN
CHINANET-BACKBONE
674.61 TiB 11.67% 674.61 TiB 0.00%
f01345523 Antwerpen, Flanders, BE
Cogent Communications
261.51 TiB 4.52% 254.12 TiB 2.83%
f01906874new Plano, Texas, US
Cogent Communications
225.58 TiB 3.90% 225.58 TiB 0.00%
f01992630 Dallas, Texas, US
Flexential Colorado Corp.
285.34 TiB 4.94% 285.34 TiB 0.00%
f01971600 Dallas, Texas, US
Flexential Colorado Corp.
236.13 TiB 4.08% 236.13 TiB 0.00%
f01392893 Amsterdam, North Holland, NL
Fusix Networks B.V.
403.52 TiB 6.98% 384.50 TiB 4.71%
f01896422 Fremont, California, US
Hurricane Electric LLC
24.78 TiB 0.43% 24.78 TiB 0.00%
f01926686 Hangzhou, Zhejiang, CN
Jiangxi Jiujiang IDC
275.64 TiB 4.77% 275.64 TiB 0.00%
f01908299new Lincoln, Nebraska, US
LightEdge Solutions
216.69 TiB 3.75% 174.74 TiB 19.36%
f0855584 Lincoln, Nebraska, US
LightEdge Solutions
211.66 TiB 3.66% 165.85 TiB 21.64%
f01091851 Lincoln, Nebraska, US
LightEdge Solutions
151.74 TiB 2.62% 151.74 TiB 0.00%
f01736668new Lincoln, Nebraska, US
LightEdge Solutions
128.75 TiB 2.23% 128.75 TiB 0.00%
f01385207 Lincoln, Nebraska, US
LightEdge Solutions
1.09 TiB 0.02% 352.00 GiB 68.57%
f01933917 Hong Kong, Central and Western, HK
OneAsia Network Limited
753.06 TiB 13.03% 750.62 TiB 0.32%
f01926635 Hong Kong, Central and Western, HK
OneAsia Network Limited
583.34 TiB 10.09% 583.34 TiB 0.00%
f01885088 Hong Kong, Central and Western, HK
OneAsia Network Limited
244.69 TiB 4.23% 244.69 TiB 0.00%
f01989888 Hong Kong, Central and Western, HK
OneAsia Network Limited
112.12 TiB 1.94% 112.12 TiB 0.00%
f01199430 Heerhugowaard, North Holland, NL
Wijnand Schouten trading as Speedium
113.86 TiB 1.97% 56.80 TiB 50.12%
f01720359 Heerhugowaard, North Holland, NL
Wijnand Schouten trading as Speedium
95.00 TiB 1.64% 55.44 TiB 41.64%
f01786387 Heerhugowaard, North Holland, NL
Wijnand Schouten trading as Speedium
89.79 TiB 1.55% 89.79 TiB 0.00%
f01771403 Heerhugowaard, North Holland, NL
Wijnand Schouten trading as Speedium
10.56 TiB 0.18% 4.09 TiB 61.24%
f01937642 Heerhugowaard, North Holland, NL
Wijnand Schouten trading as Speedium
9.41 TiB 0.16% 6.44 TiB 31.56%

Provider Distribution

Deal Data Replication

The below table shows how each many unique data are replicated across storage providers.

✔️ Data replication looks healthy.

Unique Data Size Total Deals Made Number of Providers Deal Percentage
82.73 TiB 82.73 TiB 1 1.43%
23.57 TiB 47.14 TiB 2 0.82%
51.52 TiB 154.58 TiB 3 2.67%
176.73 TiB 708.25 TiB 4 12.25%
146.05 TiB 732.30 TiB 5 12.67%
109.45 TiB 661.33 TiB 6 11.44%
150.96 TiB 1.04 PiB 7 18.44%
111.92 TiB 903.68 TiB 8 15.63%
53.55 TiB 487.22 TiB 9 8.43%
37.23 TiB 389.17 TiB 10 6.73%
17.09 TiB 229.69 TiB 11 3.97%
3.78 TiB 133.88 TiB 12 2.32%
2.75 TiB 138.13 TiB 13 2.39%
832.00 GiB 43.06 TiB 14 0.74%
64.00 GiB 3.59 TiB 15 0.06%

Replication Distribution

Deal Data Shared with other Clients

The below table shows how many unique data are shared with other clients. Usually different applications owns different data and should not resolve to the same CID.

However, this could be possible if all below clients use same software to prepare for the exact same dataset or they belong to a series of LDN applications for the same dataset.

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

NiwanDao commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacedc4s7stelgdv4zin7jipr7tn7oanrmakdvjrxtu4codkogzsbhv2

Address

f1kyaa43uqypbcsv5isyd4xw7lvgjcumfzqsxag7y

Datacap Allocated

1.34PiB

Signer Address

f1a2lia2cwwekeubwo4nppt4v4vebxs2frozarz3q

Id

679a25cf-f96e-463d-b6a5-1c762000be62

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedc4s7stelgdv4zin7jipr7tn7oanrmakdvjrxtu4codkogzsbhv2

kernelogic commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaceavwkqw6xjmarsyn7dkbb6atofpqbxjkwksrsq3v2gsc3zap7utn2

Address

f1kyaa43uqypbcsv5isyd4xw7lvgjcumfzqsxag7y

Datacap Allocated

1.34PiB

Signer Address

f1yjhnsoga2ccnepb7t3p3ov5fzom3syhsuinxexa

Id

679a25cf-f96e-463d-b6a5-1c762000be62

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceavwkqw6xjmarsyn7dkbb6atofpqbxjkwksrsq3v2gsc3zap7utn2

large-datacap-requests[bot] commented 1 year ago

Thanks for your request!

Heads up, you’re requesting more than the typical weekly onboarding rate of DataCap!
filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report[^1]

No client address found for this issue.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

Wednesdaysof commented 1 year ago

图片

图片

lotus client retrieve --provider f01989888 --pieceCid baga6ea4seaqk7hjam27qux6dnynxseyqidjzkp2iuirsi2bwvj6sfdt4s26mika baga6ea4seaqk7hjam27qux6dnynxseyqidjzkp2iuirsi2bwvj6sfdt4s26mika test.file ERROR: offer error: retrieval query offer errored: failed to fetch piece to retrieve from: getting pieces for cid baga6ea4seaqk7hjam27qux6dnynxseyqidjzkp2iuirsi2bwvj6sfdt4s26mika: getting pieces containing block baga6ea4seaqk7hjam27qux6dnynxseyqidjzkp2iuirsi2bwvj6sfdt4s26mika: failed to lookup index for mh 922020af9d2066bf0a5fc36e1b79131040d3953f48a223246836aa7d228e7c96bcc428, err: datastore: key not found

herrehesse commented 1 year ago

@Wednesdaysof Thank you for showing us a failed retrieval. Tagging the corresponding SP below here for full transparency. Will update you below when the deal(s) from this SP are retrievable again.

@Feellin Please check minerID f01989888

xinaxu commented 1 year ago

Hidde explained to me that the failed retrieval was an improperly set pieceCid during deal making in the early phase and has now been fixed. The pieceCid should start with bafy instead of baga. @Wednesdaysof have you tried retrievals for the correct deals and do you see any concern here?

herrehesse commented 1 year ago

@xinaxu should be in order now. Will ask @lvschouwen and @cryptowhizzard to double check on Monday morning and show you the results.

Enjoy your weekend.

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 6

Multisig Notary address

f02049625

Client address

f1kyaa43uqypbcsv5isyd4xw7lvgjcumfzqsxag7y

DataCap allocation requested

1.03TiB

Id

0cc99c66-3fe2-49e5-8a29-f2cd0c4183d9

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f01858410

Client address

f1kyaa43uqypbcsv5isyd4xw7lvgjcumfzqsxag7y

Last two approvers

kernelogic & xingjitansuo

Rule to calculate the allocation request amount

800% of weekly dc amount requested

DataCap allocation requested

1.03TiB

Total DataCap granted for client so far

9.02PiB

Datacap to be granted to reach the total amount requested by the client (5PiB)

-4528833419227956B

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
318292 30 1.34PiB 12.11 327.52TiB
filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

⚠️ 6 storage providers sealed too much duplicate data - f022352: 27.13%, f0855584: 21.64%, f01385207: 68.57%, f01199430: 50.12%, f01720359: 41.64%, f01937642: 31.56%

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the full report.

Joss-Hua commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacedvkzpjcpoywrcxem5bw4gxdxezqdkcj4nc5us4tpteymzflmlss4

Address

f1kyaa43uqypbcsv5isyd4xw7lvgjcumfzqsxag7y

Datacap Allocated

1.03TiB

Signer Address

f1tfg54zzscugttejv336vivknmsnzzmyudp3t7wi

Id

0cc99c66-3fe2-49e5-8a29-f2cd0c4183d9

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedvkzpjcpoywrcxem5bw4gxdxezqdkcj4nc5us4tpteymzflmlss4

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

⚠️ 5 storage providers sealed too much duplicate data - f0855584: 21.64%, f01385207: 68.57%, f01199430: 50.12%, f01720359: 41.64%, f01937642: 31.56%

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the full report.

psh0691 commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacecgtet2honrqucdhp7inabcl64dz3tbonaed7zfo7jrdyow5bmqxg

Address

f1kyaa43uqypbcsv5isyd4xw7lvgjcumfzqsxag7y

Datacap Allocated

1.03TiB

Signer Address

f1qdko4jg25vo35qmyvcrw4ak4fmuu3f5rif2kc7i

Id

0cc99c66-3fe2-49e5-8a29-f2cd0c4183d9

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacecgtet2honrqucdhp7inabcl64dz3tbonaed7zfo7jrdyow5bmqxg

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 14 days, so for now it is being closed. Please feel free to contact the Fil+ Gov team to re-open the application if it is still being processed. Thank you!

data-programs commented 1 year ago
KYC

This user’s identity has been verified through filplus.storage