search

filecoin-project / filecoin-plus-large-datasets

Hub for client applications for DataCap at a large scale
110 stars 62 forks source link

[DataCap Application] FileDrive Labs - Datasets Landing Plan - [2/3] #1267

Closed laurarenpanda closed 1 year ago

laurarenpanda commented 2 years ago

Large Dataset Notary Application

To apply for DataCap to onboard your dataset to Filecoin, please fill out the following.

Core Information

Please respond to the questions below by replacing the text saying "Please answer here". Include as much detail as you can in your answer.

Project details

Share a brief history of your project and organization.

FileDrive Datasets Landing Plan is a project for onboarding more valuable public datasets onto the Filecoin network. Through several phases, we plan to bring 10 PiB data and promote 100 PiB storage power growth to Filecoin. 

About FileDrive Datasets

FileDrive Datasets is a platform to effectively connect the huge storage market that Filecoin has built with publishers of public datasets.
The Filecoin network provides reliable, secure, and affordable decentralized storage services, and FileDrive Labs wants to deliver these benefits to end-users by building a public dataset platform.
It is challenging to attract traditional Cloud Storage and Object-base Storage users to the Filecoin network and benefit from it. Developers in the Felicoin ecosystem, such as FileDrive Labs, need to face this challenge together.
As a member of the Filecoin ecosystem, FileDrive Labs has been insisting on developing useful tools to make it easier for users to store their data onto the Filecoin network. 

FileDrive Datasets has integrated a group of tools to provide storage service with the compatibility of both Cloud Storage and Object-base Storage and better user experience to attract more users.
Projects(ongoing) behind:
- Go-Graphsplit: https://github.com/filedrive-team/go-graphsplit
- DS-Cluster: https://github.com/filedrive-team/go-ds-cluster
- Filejoy: https://github.com/filedrive-team/filejoy

Article about FileDrive Datasets on Filecoin Blog:
- Large Datasets: FileDrive: https://filecoin.io/blog/posts/large-datasets-filedrive/

About FileDrive Labs

FileDrive Labs has always defined ourselves as tool developers and infrastructure builders in the Filecoin ecosystem. From 2019, we continuously focus on technical solutions and development based on IPFS protocol and the Filecoin network and do our best to contribute to the community.
Over 80% of our team are qualified engineers, and half of them have more than 10-year development experience in multiple industries, including Communication, the Internet, and blockchain.
Since 2020, we have participated in Slingshot Competition, become one of the top teams, and stored over 5 PiB useful data from public datasets to the Filecoin network.
To contribute to the Filecoin Community, we developed an open-source data prep tool Graphsplit, FIL+ project dashboard filplus.info and storage provider discovery platform filfind,info.
Besides, we have also hold weekly online virtual events named FileDrive Meetup from March 2022, which aims to provide a platform for community members to grasp the latest trends of the Filecoin network and our work and research.

Please check the following links for more details.
- GitHub: https://github.com/filedrive-team
- Twitter: https://twitter.com/FileDrive1
- Eventbrite: https://www.eventbrite.hk/o/filedrive-labs-42456337463
- YouTube Channel: https://www.youtube.com/channel/UCxcZC1dtBUlQvZY7DX13W1w
- Medium: https://medium.com/@FileDrive1

What is the primary source of funding for this project?

FileDriven Labs, rewards from the Slingshot Competition, Filecoin DevGrants, Mircogrants and a series of Hackathons.

What other projects/ecosystem stakeholders is this project associated with?

FileDrive Dataset is an open dataset platform on IPFS Network, and all data will store on Filecoin Network through the Filecoin Plus project. Since that the primary ecosystem stakeholders are IPFS and Filecoin.

Use-case details

Describe the data being stored onto Filecoin

FileDrive Datasets Landing Plan #1
- Datasets: 6
- Total data capacity: 2451.1TiB

List of Datasets in #1:

1. ZINC Database
- 3D models for molecular docking screens.
- Size: 924.5 TiB

2. Transiting Exoplanet Survey Satellite (TESS)
- The Transiting Exoplanet Survey Satellite (TESS) is a multi-year survey that will discover exoplanets in orbit around bright stars across the entire sky using high-precision photometry. The survey will also enable a wide variety of stellar astrophysics, solar system science, and extragalactic variability studies. More information about TESS is available at MAST and the TESS Science Support Center.
- Size: 285.6 TiB

3. Smithsonian Open Access
- The Smithsonian’s mission is the "increase and diffusion of knowledge" and has been collecting since 1846. The Smithsonian, through its efforts to digitize its multidisciplinary collections, has created millions of digital assets and related metadata describing the collection objects. On February 25th, 2020, the Smithsonian released over 2.8 million CC0 interdisciplinary 2-D and 3-D images, related metadata, and additionally, research data from researches across the Smithsonian. The 2.8 million "open access" collections are a subset of the Smithsonian’s 155 million objects, 2.1 million library volumes and 156,000 cubic feet of archival collections held in 19 museums, 9 research centers, libraries, archives and the National Zoo. Digitization of collections is ongoing.
- Size: 621.2 TiB

4. Community Earth System Model v2 ARISE (CESM2 ARISE)
- Data from ARISE-SAI Experiments with CESM2
- Size: 263.5 TiB

5. 3DCoMPaT: Composition of Materials on Parts of 3D Things
- 3D CoMPaT is a richly annotated large-scale dataset of rendered compositions of Materials on Parts of thousands of unique 3D Models. This dataset primarily focuses on stylizing 3D shapes at part-level with compatible materials. Each object with the applied part-material compositions is rendered from four equally spaced views as well as four randomized views. We introduce a new task, called Grounded CoMPaT Recognition (GCR), to collectively recognize and ground compositions of materials on parts of 3D objects. We present two variations of this task and adapt state-of-art 2D/3D deep learning methods to solve the problem as baselines for future research. We hope our work will help ease future research on compositional 3D Vision.
- Size: 42.8 TiB

6. Reference Elevation Model of Antarctica (REMA)
- The Reference Elevation Model of Antarctica - 2m GSD Digital Elevation Models (DEMs) and mosaics from 2009 to the present. The REMA project seeks to fill the need for high-resolution time-series elevation data in the Antarctic. The time-dependent nature of the strip DEM files allows users to perform change detection analysis and to compare observations of topography data acquired in different seasons or years. The mosaic DEM tiles are assembled from multiple strip DEMs with the intention of providing a more consistent and comprehensive product over large areas. REMA data is constructed from in-track and cross-track high-resolution (~0.5 meter) imagery acquired by the Maxar constellation of optical imaging satellites.
- Size: 313.5 TiB

Where was the data in this dataset sourced from?

All data is from public open datasets.

Can you share a sample of the data? A link to a file, an image, a table, etc., are good ways to do this. 

FileDrive Datasets: 
https://datasets.filedrive.io/

Original Source:
https://registry.opendata.aws/zinc15/
https://registry.opendata.aws/tess/
https://registry.opendata.aws/smithsonian-open-access/
https://registry.opendata.aws/ncar-cesm2-arise/
https://registry.opendata.aws/3dcompat/
https://registry.opendata.aws/pgc-rema/

Confirm that this is a public dataset that can be retrieved by anyone on the Network (i.e., no specific permissions or access rights are required to view the data).

Yes, it is. All data can be retrieved by anyone on Filecoin Network.

What is the expected retrieval frequency for this data?

The data of FileDrive Dataset will be pinned on IPFS Network before being stored on Filecoin, which means users could mainly have two different ways to retrieve data, thought IPFS or Filecoin. So the retrieval frequency depends on users' needs.

For how long do you plan to keep this dataset stored on Filecoin?

This data will be stored for at least 1 year on Filecoin, so the verified deals will use a 1-year minimum deal duration (from 356 to 530 days).
Ideally, this project will be a permanent archival on the Filecoin network, as long as there are actual users and data requirements.

DataCap allocation plan

In which geographies (countries, regions) do you plan on making storage deals?

All regions, as long as data transmission can be stable and successful.

How will you be distributing your data to storage providers? Is there an offline data transfer process?

Through both online/offline storage deals. Data transfer strategy might be considered in certain situations.
The expected data onboarding rate is 500TiB per week, which is around 72TiB per day. 
However, it will be influenced by factors such as the speed of data transmission(especially inter-regional transmission), daily power growth of storage providers, base fees, equipment performance, etc. For these reasons, the actual data onboarding rate may differ from our expectations.

How do you plan on choosing the storage providers with whom you will be making deals? This should include a plan to ensure the data is retrievable in the future both by you and others.

Major characteristics about choosing the storage providers:
- location
- transmission speed
- deal success rate
- previous experience of real data storage
- stability of their nodes
- reputation score

How will you be distributing deals across storage providers?

All storage deals will be verified if they have enough FIL to pledge and their equipment can handle.

Do you have the resources/funding to start making deals as soon as you receive DataCap? What support from the community would help you onboard onto Filecoin?

Yes, we do.
Suggestions and feedbacks can help us optimate this project and cooperate with more great storage providers from all over the world.
large-datacap-requests[bot] commented 2 years ago

Thanks for your request!

Heads up, you’re requesting more than the typical weekly onboarding rate of DataCap!
large-datacap-requests[bot] commented 2 years ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

raghavrmadya commented 2 years ago

Datacap Request Trigger

Total DataCap requested

5PiB

Expected weekly DataCap usage rate

500TiB

Client address

f14uhjnqrocqcenbjfaergw2uvaimysi4snv2oepy

large-datacap-requests[bot] commented 2 years ago

DataCap Allocation requested

Multisig Notary address

f02049625

Client address

f14uhjnqrocqcenbjfaergw2uvaimysi4snv2oepy

DataCap allocation requested

250TiB

Id

f0608bb3-9102-4064-8a08-fc6afd65a477

Joss-Hua commented 2 years ago

I have some knowledge about FileDrive Labs and related products, and have conducted face-to-face visit with the team about the LDN. At present, I have confirmed that the above information is reliable, so as to start the first allocation.

Joss-Hua commented 2 years ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzaceckf5m4ck6x2gypwrw2n3jgr76cnhhznr6magxe56vsvi6wm2jixk

Address

f14uhjnqrocqcenbjfaergw2uvaimysi4snv2oepy

Datacap Allocated

250.00TiB

Signer Address

f1tfg54zzscugttejv336vivknmsnzzmyudp3t7wi

Id

f0608bb3-9102-4064-8a08-fc6afd65a477

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceckf5m4ck6x2gypwrw2n3jgr76cnhhznr6magxe56vsvi6wm2jixk

newwebgroup commented 1 year ago

Meet and discuss this LDN with the FileDrive Labs team through Zoom, and are willing to support them in the first round.

newwebgroup commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacead2hs2hexuij6j6x2u3yudvwkjw2wmbrs2gvw6ie6r6vbsosrbdo

Address

f14uhjnqrocqcenbjfaergw2uvaimysi4snv2oepy

Datacap Allocated

250.00TiB

Signer Address

f1e77zuityhvvw6u2t6tb5qlnsegy2s67qs4lbbbq

Id

f0608bb3-9102-4064-8a08-fc6afd65a477

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacead2hs2hexuij6j6x2u3yudvwkjw2wmbrs2gvw6ie6r6vbsosrbdo

filplus-checker commented 1 year ago

DataCap and CID Checker Report[^1]

If this is the first time a provider takes verified deal, it will be marked as new.

For most of the datacap application, below restrictions should apply.

⚠️ f01227975 has sealed 44.46% of total datacap.

⚠️ f01228008 has sealed 29.79% of total datacap.

⚠️ f0522948 has sealed 25.13% of total datacap.

Provider Location Total Deals Sealed Percentage Unique Data Duplicate Deals
f01227975 Hong Kong, Central and Western, HK 45.33 TiB 44.46% 45.33 TiB 0.00%
f01228008 Sydney, New South Wales, AU 30.38 TiB 29.79% 30.38 TiB 0.00%
f0522948 Singapore, Singapore, SG 25.63 TiB 25.13% 25.63 TiB 0.00%
f0134516 Hong Kong, Central and Western, HK 640.00 GiB 0.61% 640.00 GiB 0.00%

Provider Distribution

Deal Data Replication

The below table shows how each many unique data are replicated across storage providers.

⚠️ 97.55% of deals are for data replicated across less than 4 storage providers.

Unique Data Size Total Deals Made Number of Providers Deal Percentage
15.36 TiB 15.36 TiB 1 15.07%
4.59 TiB 9.19 TiB 2 9.01%
24.97 TiB 74.91 TiB 3 73.47%
640.00 GiB 2.50 TiB 4 2.45%

Replication Distribution

Deal Data Shared with other Clients

The below table shows how many unique data are shared with other clients. Usually different applications owns different data and should not resolve to the same CID.

⚠️ CID sharing has been observed.

Other Client Application Total Deals Affected Unique CIDs Verifier
f1bycr5r3ymkgqvkuxoemgsmnuawyawptwj44mqdi FileDrive Labs 195.81 TiB 1,600 LDN v3 multisig
f1sejgqbuwsf74qifuxqykwotyu5aswuwhubxghqa FileDrive Labs 49.97 TiB 820 LDN v3 multisig

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 2

Multisig Notary address

f02049625

Client address

f14uhjnqrocqcenbjfaergw2uvaimysi4snv2oepy

DataCap allocation requested

500TiB

Id

c4d39f80-3a31-40ca-8e7e-382cca1334fb

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f01858410

Client address

f14uhjnqrocqcenbjfaergw2uvaimysi4snv2oepy

Last two approvers

newwebgroup & Joss-Hua

Rule to calculate the allocation request amount

100% of weekly dc amount requested

DataCap allocation requested

500TiB

Total DataCap granted for client so far

250TiB

Datacap to be granted to reach the total amount requested by the client (5 PiB)

4.75PiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
6045 4 250TiB 45.01 42.10TiB
filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report[^1]

If this is the first time a provider takes verified deal, it will be marked as new.

For most of the datacap application, below restrictions should apply.

Since this is the 3rd allocation, the following restrictions have been relaxed:

✔️ Storage provider distribution looks healthy.

Provider Location Total Deals Sealed Percentage Unique Data Duplicate Deals
f01227975 Hong Kong, Central and Western, HK 64.56 TiB 44.79% 64.56 TiB 0.00%
f01228008 Sydney, New South Wales, AU 41.39 TiB 28.72% 41.39 TiB 0.00%
f0522948 Singapore, Singapore, SG 37.56 TiB 26.06% 37.56 TiB 0.00%
f0134516 Hong Kong, Central and Western, HK 640.00 GiB 0.43% 640.00 GiB 0.00%

Provider Distribution

Deal Data Replication

The below table shows how each many unique data are replicated across storage providers.

Since this is the 3rd allocation, the following restrictions have been relaxed:

✔️ Data replication looks healthy.

Unique Data Size Total Deals Made Number of Providers Deal Percentage
23.23 TiB 23.23 TiB 1 16.12%
4.17 TiB 8.34 TiB 2 5.79%
36.69 TiB 110.06 TiB 3 76.36%
640.00 GiB 2.50 TiB 4 1.73%

Replication Distribution

Deal Data Shared with other Clients

The below table shows how many unique data are shared with other clients. Usually different applications owns different data and should not resolve to the same CID.

⚠️ CID sharing has been observed.

Other Client Application Total Deals Affected Unique CIDs Verifier
f1bycr5r3ymkgqvkuxoemgsmnuawyawptwj44mqdi FileDrive Labs 212.83 TiB 2,040 LDN v3 multisig
f1sejgqbuwsf74qifuxqykwotyu5aswuwhubxghqa FileDrive Labs 87.41 TiB 1,420 LDN v3 multisig

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

kernelogic commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacedsle3u3mp75lutzro4f2odbwrgm74iunqkwifobbc4gc7d36i53o

Address

f14uhjnqrocqcenbjfaergw2uvaimysi4snv2oepy

Datacap Allocated

500.00TiB

Signer Address

f1yjhnsoga2ccnepb7t3p3ov5fzom3syhsuinxexa

Id

c4d39f80-3a31-40ca-8e7e-382cca1334fb

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedsle3u3mp75lutzro4f2odbwrgm74iunqkwifobbc4gc7d36i53o

kernelogic commented 1 year ago

FileDrive is long time community participant, willing to support.

steven004 commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacedykwneqdhqcw6dz42htvfzln544rkzsr2c5oaobceuycbdbe57tk

Address

f14uhjnqrocqcenbjfaergw2uvaimysi4snv2oepy

Datacap Allocated

500.00TiB

Signer Address

f1w2vyp4w6df44gbh4vxqle4w65zfrfnwhrl3hojy

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedykwneqdhqcw6dz42htvfzln544rkzsr2c5oaobceuycbdbe57tk

large-datacap-requests[bot] commented 1 year ago

We have found some problems in the information provided in the Approved Comment. We could not find the Filecoin address in the information provided in the comment We could not find the Datacap** allocated in the information provided in the comment

Please, take a look at the comment and edit the body of the comment providing all the required information.
steven004 commented 1 year ago

Well done, FileDrive. Willing to support FileDrive for the contribution to the ecosystem and community.

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 3

Multisig Notary address

f02049625

Client address

f14uhjnqrocqcenbjfaergw2uvaimysi4snv2oepy

DataCap allocation requested

1000.0TiB

Id

15273a41-9ae1-47f6-af56-b646a5dad2eb

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report[^1]

If this is the first time a provider takes verified deal, it will be marked as new.

For most of the datacap application, below restrictions should apply.

Since this is the 3rd allocation, the following restrictions have been relaxed:

✔️ Storage provider distribution looks healthy.

Provider Location Total Deals Sealed Percentage Unique Data Duplicate Deals
f01227975 Hong Kong, Central and Western, HK 99.85 TiB 39.99% 99.85 TiB 0.00%
f01228008 Sydney, New South Wales, AU 99.16 TiB 39.71% 99.16 TiB 0.00%
f0522948 Singapore, Singapore, SG 50.05 TiB 20.04% 50.05 TiB 0.00%
f0134516 Hong Kong, Central and Western, HK 640.00 GiB 0.25% 640.00 GiB 0.00%

Provider Distribution

Deal Data Replication

The below table shows how each many unique data are replicated across storage providers.

Since this is the 3rd allocation, the following restrictions have been relaxed:

✔️ Data replication looks healthy.

Unique Data Size Total Deals Made Number of Providers Deal Percentage
704.00 GiB 704.00 GiB 1 0.28%
49.11 TiB 98.23 TiB 2 39.34%
49.42 TiB 148.27 TiB 3 59.38%
640.00 GiB 2.50 TiB 4 1.00%

Replication Distribution

Deal Data Shared with other Clients

The below table shows how many unique data are shared with other clients. Usually different applications owns different data and should not resolve to the same CID.

⚠️ CID sharing has been observed.

Other Client Application Total Deals Affected Unique CIDs Verifier
f1bycr5r3ymkgqvkuxoemgsmnuawyawptwj44mqdi FileDrive Labs 249.05 TiB 3,200 LDN v3 multisig
f1sejgqbuwsf74qifuxqykwotyu5aswuwhubxghqa FileDrive Labs 99.91 TiB 1,620 LDN v3 multisig
f3wgfwtrs5p6jrkwfl2mksqa2ivgbgdjjrhjbefy3
n7qzvotc3y6sazmp5gfyj7um6jlgdvlbiepzawnc6
wxtq		 FileDrive Labs 1.25 TiB 20 LDN v3 multisig

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

NDLABS-Leo commented 1 year ago

The storage is in good condition and willing to support

NDLABS-Leo commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacebq4lalhoacalonj56w4j2v25sx5jph74wlkyuwyib6777e5yzlra

Address

f14uhjnqrocqcenbjfaergw2uvaimysi4snv2oepy

Datacap Allocated

1000.00TiB

Signer Address

f1yayfsv6whu3rheviucvventj3y6t542xfpb47ei

Id

15273a41-9ae1-47f6-af56-b646a5dad2eb

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebq4lalhoacalonj56w4j2v25sx5jph74wlkyuwyib6777e5yzlra

1ane-1 commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacedsrtwejo6rpomgpp2cqzl44eye7czn52crf2ojoxrlhckm6eifg6

Address

f14uhjnqrocqcenbjfaergw2uvaimysi4snv2oepy

Datacap Allocated

1000.00TiB

Signer Address

f1mdk7s2vntzm6hu35yuo6vjubtrpfnb2awhgvrri

Id

15273a41-9ae1-47f6-af56-b646a5dad2eb

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedsrtwejo6rpomgpp2cqzl44eye7czn52crf2ojoxrlhckm6eifg6

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 4

Multisig Notary address

f02049625

Client address

f14uhjnqrocqcenbjfaergw2uvaimysi4snv2oepy

DataCap allocation requested

1.95PiB

Id

1c0fad94-1821-42a6-9fad-c74209f62b29

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f01858410

Client address

f14uhjnqrocqcenbjfaergw2uvaimysi4snv2oepy

Last two approvers

1ane-1 & not found

Rule to calculate the allocation request amount

400% of weekly dc amount requested

DataCap allocation requested

1.95PiB

Total DataCap granted for client so far

750TiB

Datacap to be granted to reach the total amount requested by the client (5 PiB)

4.26PiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
22317 9 1000.0TiB 30.92 91.21TiB
filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report[^1]

Storage Provider Distribution

The below table shows the distribution of storage providers that have stored data for this client.

If this is the first time a provider takes verified deal, it will be marked as new.

For most of the datacap application, below restrictions should apply.

⚠️ f01227975 has sealed 33.08% of total datacap.

Provider Location Total Deals Sealed Percentage Unique Data Duplicate Deals
f01227975 Hong Kong, Central and Western, HK
Alibaba (US) Technology Co., Ltd.		 199.86 TiB 33.08% 199.86 TiB 0.00%
f01228008 Sydney, New South Wales, AU
Alibaba (US) Technology Co., Ltd.		 99.19 TiB 16.42% 99.19 TiB 0.00%
f01228100 San Jose, California, US
Alibaba (US) Technology Co., Ltd.		 70.04 TiB 11.59% 70.04 TiB 0.00%
f01228105 Hong Kong, Central and Western, HK
Alibaba (US) Technology Co., Ltd.		 68.63 TiB 11.36% 68.63 TiB 0.00%
f0522948 Singapore, Singapore, SG
Alibaba (US) Technology Co., Ltd.		 50.05 TiB 8.28% 50.05 TiB 0.00%
f01228089 Frankfurt am Main, Hesse, DE
Alibaba (US) Technology Co., Ltd.		 46.79 TiB 7.74% 46.79 TiB 0.00%
f01228087 London, England, GB
Alibaba (US) Technology Co., Ltd.		 41.77 TiB 6.91% 41.77 TiB 0.00%
f0134516 Hong Kong, Central and Western, HK
Alibaba (US) Technology Co., Ltd.		 640.00 GiB 0.10% 640.00 GiB 0.00%
f01984576new Singapore, Singapore, SG
HUAWEI CLOUDS		 27.25 TiB 4.51% 27.25 TiB 0.00%

Provider Distribution

Deal Data Replication

The below table shows how each many unique data are replicated across storage providers.

⚠️ 86.35% of deals are for data replicated across less than 4 storage providers.

Unique Data Size Total Deals Made Number of Providers Deal Percentage
129.61 TiB 129.61 TiB 1 21.45%
49.39 TiB 98.79 TiB 2 16.35%
97.78 TiB 293.34 TiB 3 48.55%
20.62 TiB 82.47 TiB 4 13.65%

Replication Distribution

Deal Data Shared with other Clients

The below table shows how many unique data are shared with other clients. Usually different applications owns different data and should not resolve to the same CID.

However, this could be possible if all below clients use same software to prepare for the exact same dataset or they belong to a series of LDN applications for the same dataset.

⚠️ CID sharing has been observed.

Other Client Application Total Deals Affected Unique CIDs Approvers
f3wgfwtrs5p6jrkwfl2mksqa2ivgbgdjjrhjbefy3
n7qzvotc3y6sazmp5gfyj7um6jlgdvlbiepzawnc6
wxtq		 FileDrive Labs 653.94 TiB 3,020 1GaryGJG
1IreneYoung
3Joss-Hua
1liyunzhi-666
1MegTei
1MetaWaveInfo
3newwebgroup
2psh0691
f1bycr5r3ymkgqvkuxoemgsmnuawyawptwj44mqdi FileDrive Labs 384.44 TiB 6,525 11ane-1
1Joss-Hua
1kernelogic
1NDLABS-OFFICE
1newwebgroup
1steven004
f1sejgqbuwsf74qifuxqykwotyu5aswuwhubxghqa FileDrive Labs 99.91 TiB 1,620 1Joss-Hua
1kernelogic
2newwebgroup

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report[^1]

Storage Provider Distribution

The below table shows the distribution of storage providers that have stored data for this client.

If this is the first time a provider takes verified deal, it will be marked as new.

For most of the datacap application, below restrictions should apply.

⚠️ f01227975 has sealed 33.08% of total datacap.

Provider Location Total Deals Sealed Percentage Unique Data Duplicate Deals
f01227975 Hong Kong, Central and Western, HK
Alibaba (US) Technology Co., Ltd.		 199.86 TiB 33.08% 199.86 TiB 0.00%
f01228008 Sydney, New South Wales, AU
Alibaba (US) Technology Co., Ltd.		 99.19 TiB 16.42% 99.19 TiB 0.00%
f01228100 San Jose, California, US
Alibaba (US) Technology Co., Ltd.		 70.04 TiB 11.59% 70.04 TiB 0.00%
f01228105 Hong Kong, Central and Western, HK
Alibaba (US) Technology Co., Ltd.		 68.63 TiB 11.36% 68.63 TiB 0.00%
f0522948 Singapore, Singapore, SG
Alibaba (US) Technology Co., Ltd.		 50.05 TiB 8.28% 50.05 TiB 0.00%
f01228089 Frankfurt am Main, Hesse, DE
Alibaba (US) Technology Co., Ltd.		 46.79 TiB 7.74% 46.79 TiB 0.00%
f01228087 London, England, GB
Alibaba (US) Technology Co., Ltd.		 41.77 TiB 6.91% 41.77 TiB 0.00%
f0134516 Hong Kong, Central and Western, HK
Alibaba (US) Technology Co., Ltd.		 640.00 GiB 0.10% 640.00 GiB 0.00%
f01984576new Singapore, Singapore, SG
HUAWEI CLOUDS		 27.25 TiB 4.51% 27.25 TiB 0.00%

Provider Distribution

Deal Data Replication

The below table shows how each many unique data are replicated across storage providers.

⚠️ 86.35% of deals are for data replicated across less than 4 storage providers.

Unique Data Size Total Deals Made Number of Providers Deal Percentage
129.61 TiB 129.61 TiB 1 21.45%
49.39 TiB 98.79 TiB 2 16.35%
97.78 TiB 293.34 TiB 3 48.55%
20.62 TiB 82.47 TiB 4 13.65%

Replication Distribution

Deal Data Shared with other Clients

The below table shows how many unique data are shared with other clients. Usually different applications owns different data and should not resolve to the same CID.

However, this could be possible if all below clients use same software to prepare for the exact same dataset or they belong to a series of LDN applications for the same dataset.

⚠️ CID sharing has been observed.

Other Client Application Total Deals Affected Unique CIDs Approvers
f3wgfwtrs5p6jrkwfl2mksqa2ivgbgdjjrhjbefy3
n7qzvotc3y6sazmp5gfyj7um6jlgdvlbiepzawnc6
wxtq		 FileDrive Labs 653.94 TiB 3,020 1GaryGJG
1IreneYoung
3Joss-Hua
1liyunzhi-666
1MegTei
1MetaWaveInfo
3newwebgroup
2psh0691
f1bycr5r3ymkgqvkuxoemgsmnuawyawptwj44mqdi FileDrive Labs 384.44 TiB 6,525 11ane-1
1Joss-Hua
1kernelogic
1NDLABS-OFFICE
1newwebgroup
1steven004
f1sejgqbuwsf74qifuxqykwotyu5aswuwhubxghqa FileDrive Labs 99.91 TiB 1,620 1Joss-Hua
1kernelogic
2newwebgroup

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

Joss-Hua commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report[^1]

Storage Provider Distribution

The below table shows the distribution of storage providers that have stored data for this client.

If this is the first time a provider takes verified deal, it will be marked as new.

For most of the datacap application, below restrictions should apply.

✔️ Storage provider distribution looks healthy.

Provider Location Total Deals Sealed Percentage Unique Data Duplicate Deals
f01227975 Hong Kong, Central and Western, HK
Alibaba (US) Technology Co., Ltd.		 199.86 TiB 18.42% 199.86 TiB 0.00%
f01228008 Sydney, New South Wales, AU
Alibaba (US) Technology Co., Ltd.		 154.30 TiB 14.22% 154.30 TiB 0.00%
f01228100 San Jose, California, US
Alibaba (US) Technology Co., Ltd.		 128.38 TiB 11.83% 128.38 TiB 0.00%
f01228105 Hong Kong, Central and Western, HK
Alibaba (US) Technology Co., Ltd.		 99.98 TiB 9.21% 99.98 TiB 0.00%
f01228087 London, England, GB
Alibaba (US) Technology Co., Ltd.		 99.98 TiB 9.21% 99.98 TiB 0.00%
f0522948 Singapore, Singapore, SG
Alibaba (US) Technology Co., Ltd.		 92.50 TiB 8.52% 92.50 TiB 0.00%
f0867300 Tokyo, Tokyo, JP
Alibaba (US) Technology Co., Ltd.		 50.64 TiB 4.67% 50.64 TiB 0.00%
f01228000 Seoul, Seoul, KR
Alibaba (US) Technology Co., Ltd.		 47.02 TiB 4.33% 47.02 TiB 0.00%
f01228089 Frankfurt am Main, Hesse, DE
Alibaba (US) Technology Co., Ltd.		 46.79 TiB 4.31% 46.79 TiB 0.00%
f0134516 Hong Kong, Central and Western, HK
Alibaba (US) Technology Co., Ltd.		 640.00 GiB 0.06% 640.00 GiB 0.00%
f01993388 Boardman, Oregon, US
Amazon.com, Inc.		 99.79 TiB 9.20% 99.75 TiB 0.03%
f01993339 Singapore, Singapore, SG
Amazon.com, Inc.		 6.25 TiB 0.58% 6.25 TiB 0.00%
f01984576new Singapore, Singapore, SG
HUAWEI CLOUDS		 53.19 TiB 4.90% 53.19 TiB 0.00%
f01984580 Singapore, Singapore, SG
HUAWEI CLOUDS		 5.84 TiB 0.54% 5.84 TiB 0.00%

Provider Distribution

Deal Data Replication

The below table shows how each many unique data are replicated across storage providers.

⚠️ 40.32% of deals are for data replicated across less than 4 storage providers.

Unique Data Size Total Deals Made Number of Providers Deal Percentage
132.92 TiB 132.92 TiB 1 12.25%
69.27 TiB 138.53 TiB 2 12.77%
55.34 TiB 166.03 TiB 3 15.30%
107.27 TiB 429.09 TiB 4 39.54%
43.71 TiB 218.57 TiB 5 20.14%

Replication Distribution

Deal Data Shared with other Clients

The below table shows how many unique data are shared with other clients. Usually different applications owns different data and should not resolve to the same CID.

However, this could be possible if all below clients use same software to prepare for the exact same dataset or they belong to a series of LDN applications for the same dataset.

⚠️ CID sharing has been observed.

Other Client Application Total Deals Affected Unique CIDs Approvers
f3wgfwtrs5p6jrkwfl2mksqa2ivgbgdjjrhjbefy3
n7qzvotc3y6sazmp5gfyj7um6jlgdvlbiepzawnc6
wxtq		 FileDrive Labs 801.75 TiB 3,635 1GaryGJG
1IreneYoung
3Joss-Hua
1liyunzhi-666
1MegTei
1MetaWaveInfo
3newwebgroup
2psh0691
f1bycr5r3ymkgqvkuxoemgsmnuawyawptwj44mqdi FileDrive Labs 681.40 TiB 10,333 11ane-1
1Joss-Hua
1kernelogic
1NDLABS-OFFICE
1newwebgroup
1steven004
f1sejgqbuwsf74qifuxqykwotyu5aswuwhubxghqa FileDrive Labs 559.50 TiB 9,212 1Joss-Hua
1kernelogic
2newwebgroup

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

cryptowhizzard commented 1 year ago

Hello,

Finally someone who understands what retrievability is:

lotus client retrieve --provider f01984580 QmUpP6z2iRwGvkx1DMkF994R3iceRFyepStNJmpTHXCPjd test.car Recv 0 B, Paid 0 FIL, Open (New), 0s [1673298242118094995|0] Recv 0 B, Paid 0 FIL, DealProposed (WaitForAcceptance), 4ms [1673298242118094995|0]

Instant reply, instant retrieval. Chapeau!

cryptowhizzard commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacebkwic4df5edhssffxjrcta47juorqerxlymyr6eke5ldtsrlojea

Address

f14uhjnqrocqcenbjfaergw2uvaimysi4snv2oepy

Datacap Allocated

1.95PiB

Signer Address

f1krmypm4uoxxf3g7okrwtrahlmpcph3y7rbqqgfa

Id

1c0fad94-1821-42a6-9fad-c74209f62b29

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebkwic4df5edhssffxjrcta47juorqerxlymyr6eke5ldtsrlojea

kernelogic commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacedmkk6h4ferwq46yied6w5go4pzq56v3yvi7rdvwt6rdu7pbo42jo

Address

f14uhjnqrocqcenbjfaergw2uvaimysi4snv2oepy

Datacap Allocated

1.95PiB

Signer Address

f1yjhnsoga2ccnepb7t3p3ov5fzom3syhsuinxexa

Id

1c0fad94-1821-42a6-9fad-c74209f62b29

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedmkk6h4ferwq46yied6w5go4pzq56v3yvi7rdvwt6rdu7pbo42jo

kernelogic commented 1 year ago

DD done in #1266

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 5

Multisig Notary address

f02049625

Client address

f14uhjnqrocqcenbjfaergw2uvaimysi4snv2oepy

DataCap allocation requested

1.34PiB

Id

693fbd37-bee1-44ca-8ea0-e6ddacec1945

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f01858410

Client address

f14uhjnqrocqcenbjfaergw2uvaimysi4snv2oepy

Last two approvers

kernelogic & cryptowhizzard

Rule to calculate the allocation request amount

800% of weekly dc amount requested

DataCap allocation requested

1.34PiB

Total DataCap granted for client so far

1.70PiB

Datacap to be granted to reach the total amount requested by the client (5 PiB)

3.29PiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
42329 14 1.95PiB 15.72 478.63TiB
filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report[^1]

Storage Provider Distribution

The below table shows the distribution of storage providers that have stored data for this client.

If this is the first time a provider takes verified deal, it will be marked as new.

For most of the datacap application, below restrictions should apply.

✔️ Storage provider distribution looks healthy.

Provider Location Total Deals Sealed Percentage Unique Data Duplicate Deals
f01227975 Hong Kong, Central and Western, HK
Alibaba (US) Technology Co., Ltd.		 199.86 TiB 16.91% 199.86 TiB 0.00%
f01228008 Sydney, New South Wales, AU
Alibaba (US) Technology Co., Ltd.		 168.27 TiB 14.24% 168.27 TiB 0.00%
f01228100 San Jose, California, US
Alibaba (US) Technology Co., Ltd.		 138.38 TiB 11.71% 138.38 TiB 0.00%
f0522948 Singapore, Singapore, SG
Alibaba (US) Technology Co., Ltd.		 106.00 TiB 8.97% 106.00 TiB 0.00%
f01228087 London, England, GB
Alibaba (US) Technology Co., Ltd.		 99.98 TiB 8.46% 99.98 TiB 0.00%
f01228105 Hong Kong, Central and Western, HK
Alibaba (US) Technology Co., Ltd.		 99.98 TiB 8.46% 99.98 TiB 0.00%
f0867300 Tokyo, Tokyo, JP
Alibaba (US) Technology Co., Ltd.		 65.27 TiB 5.52% 65.27 TiB 0.00%
f01228000 Seoul, Seoul, KR
Alibaba (US) Technology Co., Ltd.		 60.92 TiB 5.15% 60.92 TiB 0.00%
f01228089 Frankfurt am Main, Hesse, DE
Alibaba (US) Technology Co., Ltd.		 46.79 TiB 3.96% 46.79 TiB 0.00%
f0134516 Hong Kong, Central and Western, HK
Alibaba (US) Technology Co., Ltd.		 640.00 GiB 0.05% 640.00 GiB 0.00%
f01993388 Boardman, Oregon, US
Amazon.com, Inc.		 99.97 TiB 8.46% 99.94 TiB 0.03%
f01993339 Singapore, Singapore, SG
Amazon.com, Inc.		 21.94 TiB 1.86% 21.94 TiB 0.00%
f01984576new Singapore, Singapore, SG
HUAWEI CLOUDS		 53.19 TiB 4.50% 53.19 TiB 0.00%
f01984580 Singapore, Singapore, SG
HUAWEI CLOUDS		 20.81 TiB 1.76% 20.81 TiB 0.00%

Provider Distribution

Deal Data Replication

The below table shows how each many unique data are replicated across storage providers.

⚠️ 37.37% of deals are for data replicated across less than 4 storage providers.

Unique Data Size Total Deals Made Number of Providers Deal Percentage
122.98 TiB 122.98 TiB 1 10.40%
75.92 TiB 151.84 TiB 2 12.85%
55.63 TiB 166.88 TiB 3 14.12%
104.99 TiB 419.97 TiB 4 35.53%
57.64 TiB 288.25 TiB 5 24.39%
5.34 TiB 32.06 TiB 6 2.71%

Replication Distribution

Deal Data Shared with other Clients

The below table shows how many unique data are shared with other clients. Usually different applications owns different data and should not resolve to the same CID.

However, this could be possible if all below clients use same software to prepare for the exact same dataset or they belong to a series of LDN applications for the same dataset.

⚠️ CID sharing has been observed.

Other Client Application Total Deals Affected Unique CIDs Approvers
f3wgfwtrs5p6jrkwfl2mksqa2ivgbgdjjrhjbefy3
n7qzvotc3y6sazmp5gfyj7um6jlgdvlbiepzawnc6
wxtq		 FileDrive Labs 802.53 TiB 3,660 1GaryGJG
1IreneYoung
3Joss-Hua
1liyunzhi-666
1MegTei
1MetaWaveInfo
3newwebgroup
2psh0691
f1bycr5r3ymkgqvkuxoemgsmnuawyawptwj44mqdi FileDrive Labs 705.93 TiB 10,728 11ane-1
1cryptowhizzard
1Joss-Hua
2kernelogic
1NDLABS-OFFICE
1newwebgroup
1steven004
f1sejgqbuwsf74qifuxqykwotyu5aswuwhubxghqa FileDrive Labs 588.70 TiB 9,587 1cryptowhizzard
1Joss-Hua
2kernelogic
2newwebgroup

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

BDEio commented 1 year ago

@laurarenpanda Hi! Congratulations on your DataCap approval！ BDE is a verified deals auction house helping you to get paid storing your data with reliable storage providers. If you need any help, please get in touch.

cryptowhizzard commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacebpyc5pt4pcd4p5gsrx3dudrzfnwvwuxqokey4377cjfwhhwmx2dg

Address

f14uhjnqrocqcenbjfaergw2uvaimysi4snv2oepy

Datacap Allocated

1.34PiB

Signer Address

f1krmypm4uoxxf3g7okrwtrahlmpcph3y7rbqqgfa

Id

693fbd37-bee1-44ca-8ea0-e6ddacec1945

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebpyc5pt4pcd4p5gsrx3dudrzfnwvwuxqokey4377cjfwhhwmx2dg

kernelogic commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacedzn6nm4smgxgnvs2qpxjrueq7cykrg3pa463ydkeh3u4vgswt35a

Address

f14uhjnqrocqcenbjfaergw2uvaimysi4snv2oepy

Datacap Allocated

1.34PiB

Signer Address

f1yjhnsoga2ccnepb7t3p3ov5fzom3syhsuinxexa

Id

693fbd37-bee1-44ca-8ea0-e6ddacec1945

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedzn6nm4smgxgnvs2qpxjrueq7cykrg3pa463ydkeh3u4vgswt35a

cryptowhizzard commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacecflup3alqyuvqxajso6v76cwtthgph647fdyb3xuozzspqgb2yki

Address

f14uhjnqrocqcenbjfaergw2uvaimysi4snv2oepy

Datacap Allocated

1.34PiB

Signer Address

f1krmypm4uoxxf3g7okrwtrahlmpcph3y7rbqqgfa

Id

693fbd37-bee1-44ca-8ea0-e6ddacec1945

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacecflup3alqyuvqxajso6v76cwtthgph647fdyb3xuozzspqgb2yki

stcloudlisa commented 1 year ago

I would like to support them for the following reasons:

Data can be retrieved Reasonable SP distribution

stcloudlisa commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacedzgj3uxfgzsec6sypsju7wlgh3gux3lnabjkt7ntzq5esvzc5bpm

Address

f14uhjnqrocqcenbjfaergw2uvaimysi4snv2oepy

Datacap Allocated

1.34PiB

Signer Address

f1jvvltduw35u6inn5tr4nfualyd42bh3vjtylgci

Id

693fbd37-bee1-44ca-8ea0-e6ddacec1945

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedzgj3uxfgzsec6sypsju7wlgh3gux3lnabjkt7ntzq5esvzc5bpm

Sunnyiscoming commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

⚠️ 51.25% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

⚠️ CID sharing has been observed. (Top 3)

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the full report.

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 6

Multisig Notary address

f02049625

Client address

f14uhjnqrocqcenbjfaergw2uvaimysi4snv2oepy

DataCap allocation requested

1.03TiB

Id

7f74192e-2a21-4e83-81f5-957033f7fc4a

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

⚠️ 44.53% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

⚠️ CID sharing has been observed. (Top 3)

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the full report.

data-programs commented 1 year ago
KYC

This user’s identity has been verified through filplus.storage