search

filecoin-project / filecoin-plus-large-datasets

Hub for client applications for DataCap at a large scale
110 stars 62 forks source link

[DataCap Application] Zhixun #628

Closed ShanghaiZhixun closed 1 year ago

ShanghaiZhixun commented 2 years ago

Large Dataset Notary Application

To apply for DataCap to onboard your dataset to Filecoin, please fill out the following.

Core Information

Please respond to the questions below by replacing the text saying "Please answer here". Include as much detail as you can in your answer.

Project details

Share a brief history of your project and organization.

Zhixun was established in 2011, since its establishment, the company continues to take the road of professional development, with leading technical capabilities, excellent product quality, the whole integration of solutions to win the recognition of the market and customers. We focus on IT integration and services, specializing in DICT system integration and smart city construction in the field of computer, communication and broadband network, the company has a deep technical background and carrier-grade engineering capabilities to provide customers with excellent communication and network security solutions and full process delivery services. For a long time, zhixun has been dedicated to researching industry customer needs and tailoring total solutions for customers. Through industry best practices and advanced technologies, we are committed to ensuring our customers' networks are stable, secure and highly efficient. We help customers reduce operational costs, respond quickly to market changes, and leverage competitive advantages.

What is the primary source of funding for this project?

Company's own funding.

What other projects/ecosystem stakeholders is this project associated with?

No.

Use-case details

Describe the data being stored onto Filecoin

Huge amount of videos and pics.

Where was the data in this dataset sourced from?

It is from online exhibition hall.

Can you share a sample of the data? A link to a file, an image, a table, etc., are good ways to do this. 

[a.zip](https://github.com/filecoin-project/filecoin-plus-large-datasets/files/9298968/a.zip)
[b.zip](https://github.com/filecoin-project/filecoin-plus-large-datasets/files/9298970/b.zip)

         Confirm that this is a public dataset that can be retrieved by anyone on the Network (i.e., no specific permissions or access rights are required to view the data).

Confirm!

What is the expected retrieval frequency for this data?

Twice a year

For how long do you plan to keep this dataset stored on Filecoin?

540 days+

DataCap allocation plan

In which geographies (countries, regions) do you plan on making storage deals?

World Wide.

How will you be distributing your data to storage providers? Is there an offline data transfer process?

Both. A small part of data may be distributed via online data transfer process.

How do you plan on choosing the storage providers with whom you will be making deals? This should include a plan to ensure the data is retrievable in the future both by you and others.

1. SP with storage experience
2. SP with high transmission speed

How will you be distributing deals across storage providers?

Distibuting rules based on distance and Fil.

Do you have the resources/funding to start making deals as soon as you receive DataCap? What support from the community would help you onboard onto Filecoin?

Yes.
large-datacap-requests[bot] commented 2 years ago

Thanks for your request!

Heads up, you’re requesting more than the typical weekly onboarding rate of DataCap!
large-datacap-requests[bot] commented 2 years ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

large-datacap-requests[bot] commented 2 years ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

Sunnyiscoming commented 2 years ago

Can you provide more detailed information about storage providers distribution, such as you can list SPs you have contacted with at present?

Could you send an email to filplus-app-review@fil.org with your official domain in order to confirm your identity? Email name should includes the issue id #628.

ShanghaiZhixun commented 2 years ago

@Sunnyiscoming We welcome sp from all countries in the world to work with us, but we will try to choose sp that are closer to us. We are contacting f0104671 and will continue to contact 4-5 nodes afterwards. I've sent the email to you. 8b42-62789b517e8e 98d0-8c71e210ac4f

Sunnyiscoming commented 2 years ago

Received that.

raghavrmadya commented 2 years ago

"Huge amount of videos and pics." We are unsure what this means. Please provide more evidence for what you are storing by providing more data samples including evidence that the data is owned by you. If not, please share who is the data owner.

raghavrmadya commented 2 years ago

Also how many copies will you store and how much data do you have today

ShanghaiZhixun commented 2 years ago

@raghavrmadya Ok! All of this data comes from our projects, and I've given some more samples to show. At the moment we need to store 1.24PiB of data, 4 copies. https://drive.google.com/file/d/1EhMskyssNfSISAAGtOfPBtyJ7seosYOP/view?usp=sharing

raghavrmadya commented 2 years ago

Datacap Request Trigger

Total DataCap requested

5PiB

Expected weekly DataCap usage rate

100TiB

Client address

f1zav4f4rak2yvbrq357z3whj224wn4dipl7eiesy

large-datacap-requests[bot] commented 2 years ago

DataCap Allocation requested

Multisig Notary address

f02049625

Client address

f1zav4f4rak2yvbrq357z3whj224wn4dipl7eiesy

DataCap allocation requested

50TiB

kernelogic commented 2 years ago

How is this data useful for anyone outside of your company? I think this is more like an enterprise dataset.

raghavrmadya commented 2 years ago

Important to ascertain if the dataset is public. If not fully public, must declare and go through LDN exceptions process. Thanks for your question @kernelogic

jessie8o8 commented 2 years ago

Hi @ShanghaiZhixun! Could you send a google drive folder with viewable content in it? I am not comfortable downloading unknown .zip files.

ShanghaiZhixun commented 2 years ago

Hi @kernelogic ! Our data is about our projects of digital exhibition hall. As you know, the online exhibition is open to the public, and the data was carefully selected by us. I think our application is qualified for LDN. We are more than willing to store this data in filecoin!

ShanghaiZhixun commented 2 years ago

Hi @ShanghaiZhixun! Could you send a google drive folder with viewable content in it? I am not comfortable downloading unknown .zip files.

@jessie8o8 Hi! Could you try it again? I checked my google drive and made sure my setting was right. googledrive

PluskitOfficial commented 2 years ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacec6uwb6nqpfshuguavdujy4lr35uefk352gv2inogmebkq2t673du

Address

f1zav4f4rak2yvbrq357z3whj224wn4dipl7eiesy

Datacap Allocated

50.00TiB

Signer Address

f1tgnlhtcmhwipfm7thsftxhn5k52velyjlazpvka

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacec6uwb6nqpfshuguavdujy4lr35uefk352gv2inogmebkq2t673du

MRJAVAZHAO commented 2 years ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaced6dar4nmgy5j2jzl5lkak6u72jgxkc4kcosymf7wuplesux34oci

Address

f1zav4f4rak2yvbrq357z3whj224wn4dipl7eiesy

Datacap Allocated

50.00TiB

Signer Address

f14gme3f52prtyzk6pblogrdd6b6ivp4swc6qmesi

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaced6dar4nmgy5j2jzl5lkak6u72jgxkc4kcosymf7wuplesux34oci

BDE-io commented 2 years ago

@ShanghaiZhixun Hi! Great to see you have gotten approval for DataCap and advancing the mission of preserving humanity’s most important information. If you are looking for storage providers to store these data or have any questions, please visit #bigdata-exchange on Filecoin Slack or reply here.

We have strong demand from a diverse group of SPs, who are actively looking to onboard more data.

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 2

Multisig Notary address

f02049625

Client address

f1zav4f4rak2yvbrq357z3whj224wn4dipl7eiesy

DataCap allocation requested

100TiB

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f01858410

Client address

f1zav4f4rak2yvbrq357z3whj224wn4dipl7eiesy

Last two approvers

MRJAVAZHAO & PluskitOfficial

Rule to calculate the allocation request amount

100% of weekly dc amount requested

DataCap allocation requested

100TiB

Total DataCap granted for client so far

50TiB

Datacap to be granted to reach the total amount requested by the client (5PiB)

4.95PiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
800 2 50TiB 51.52 5.56TiB
UnionLabs2020 commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzaceamoubytlpnvda2mwpwbwy62x5erda53mtoa5tpegmab422nl4mhs

Address

f1zav4f4rak2yvbrq357z3whj224wn4dipl7eiesy

Datacap Allocated

100.00TiB

Signer Address

f17xdri3wunqgld7dm23e4f3eqsntjakwc47xjo6i

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceamoubytlpnvda2mwpwbwy62x5erda53mtoa5tpegmab422nl4mhs

MetaWaveInfo commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacecex5m6nlbv4xh4dj2fge6pkfkg2bo5kjuqnzqwr5oequvomj73hu

Address

f1zav4f4rak2yvbrq357z3whj224wn4dipl7eiesy

Datacap Allocated

100.00TiB

Signer Address

f1ktlkcxnmzxcdaoqfsunrg3vocfbmgv4n3mrn74a

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacecex5m6nlbv4xh4dj2fge6pkfkg2bo5kjuqnzqwr5oequvomj73hu

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 3

Multisig Notary address

f02049625

Client address

f1zav4f4rak2yvbrq357z3whj224wn4dipl7eiesy

DataCap allocation requested

200TiB

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f01858410

Client address

f1zav4f4rak2yvbrq357z3whj224wn4dipl7eiesy

Last two approvers

MetaWaveInfo & UnionLabs2020

Rule to calculate the allocation request amount

200% of weekly dc amount requested

DataCap allocation requested

200TiB

Total DataCap granted for client so far

150TiB

Datacap to be granted to reach the total amount requested by the client (5PiB)

4.85PiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
3786 5 100TiB 35.56 0B
Destore2023 commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacebpsbqo7wngswbbvpoie7o3pk2a3ldk2yphuugj3mninjurisp2d4

Address

f1zav4f4rak2yvbrq357z3whj224wn4dipl7eiesy

Datacap Allocated

200.00TiB

Signer Address

f1yh6q3nmsg7i2sys7f7dexcuajgoweudcqj2chfi

Id

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebpsbqo7wngswbbvpoie7o3pk2a3ldk2yphuugj3mninjurisp2d4

1475Notary commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaceagft5pj7ebpucm2ojx6czh5425jj6ayvbjc6b4usvmtzrk344c3s

Address

f1zav4f4rak2yvbrq357z3whj224wn4dipl7eiesy

Datacap Allocated

200.00TiB

Signer Address

f1ofq4mngy7ggcp755pfquq2gphjjnlydolf6awtq

Id

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceagft5pj7ebpucm2ojx6czh5425jj6ayvbjc6b4usvmtzrk344c3s

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 4

Multisig Notary address

f02049625

Client address

f1zav4f4rak2yvbrq357z3whj224wn4dipl7eiesy

DataCap allocation requested

400TiB

Id

3a168233-5958-4c7b-9b09-cc70407be0cb

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f01858410

Client address

f1zav4f4rak2yvbrq357z3whj224wn4dipl7eiesy

Last two approvers

1475Notary & swatchliu

Rule to calculate the allocation request amount

400% of weekly dc amount requested

DataCap allocation requested

400TiB

Total DataCap granted for client so far

350TiB

Datacap to be granted to reach the total amount requested by the client (5PiB)

4.65PiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
5787 6 200TiB 44.88 19.37TiB
MetaWaveInfo commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacebukpix6ys4pckmybm2owmtwm47tsh3buhwh7oulr6pb4dll5htlw

Address

f1zav4f4rak2yvbrq357z3whj224wn4dipl7eiesy

Datacap Allocated

400.00TiB

Signer Address

f1ktlkcxnmzxcdaoqfsunrg3vocfbmgv4n3mrn74a

Id

3a168233-5958-4c7b-9b09-cc70407be0cb

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebukpix6ys4pckmybm2owmtwm47tsh3buhwh7oulr6pb4dll5htlw

Destore2023 commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacedqsgmgkqkj3fbz4myru3umbvwjqbxmyrzqvw6filk36f2zzlfl5k

Address

f1zav4f4rak2yvbrq357z3whj224wn4dipl7eiesy

Datacap Allocated

400.00TiB

Signer Address

f1yh6q3nmsg7i2sys7f7dexcuajgoweudcqj2chfi

Id

3a168233-5958-4c7b-9b09-cc70407be0cb

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedqsgmgkqkj3fbz4myru3umbvwjqbxmyrzqvw6filk36f2zzlfl5k

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 5

Multisig Notary address

f02049625

Client address

f1zav4f4rak2yvbrq357z3whj224wn4dipl7eiesy

DataCap allocation requested

800TiB

Id

6b4bcd09-06d9-4b7a-99af-2ca2963da94f

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f01858410

Client address

f1zav4f4rak2yvbrq357z3whj224wn4dipl7eiesy

Last two approvers

swatchliu & MetaWaveInfo

Rule to calculate the allocation request amount

800% of weekly dc amount requested

DataCap allocation requested

800TiB

Total DataCap granted for client so far

750TiB

Datacap to be granted to reach the total amount requested by the client (5PiB)

4.26PiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
18020 11 400TiB 30.26 90.62TiB
filplus-checker commented 1 year ago

DataCap and CID Checker Report[^1]

If this is the first time a provider takes verified deal, it will be marked as new.

For most of the datacap application, below restrictions should apply.

⚠️ f01962589 has sealed 28.19% of total datacap.

⚠️ 89.78% of total deal sealed by f01962589 are duplicate data.

⚠️ f01962589 has unknown IP location.

⚠️ f01974746 has sealed 27.10% of total datacap.

⚠️ 89.99% of total deal sealed by f01974746 are duplicate data.

⚠️ f01974746 has unknown IP location.

⚠️ 88.46% of total deal sealed by f01887652 are duplicate data.

⚠️ 76.54% of total deal sealed by f01937964 are duplicate data.

⚠️ 80.68% of total deal sealed by f01828096 are duplicate data.

⚠️ 79.45% of total deal sealed by f01832632 are duplicate data.

⚠️ 77.46% of total deal sealed by f01926914 are duplicate data.

⚠️ 77.13% of total deal sealed by f01926802 are duplicate data.

⚠️ 74.86% of total deal sealed by f01926585 are duplicate data.

⚠️ 70.04% of total deal sealed by f01924416 are duplicate data.

⚠️ 86.87% of total deal sealed by f01889480 are duplicate data.

Provider Location Total Deals Sealed Percentage Unique Data Duplicate Deals
f01962589 Unknown 208.50 TiB 28.19% 21.31 TiB 89.78%
f01974746 Unknown 200.38 TiB 27.10% 20.06 TiB 89.99%
f01887652 Ashburn, Virginia, US 113.75 TiB 15.38% 13.13 TiB 88.46%
f01937964 Tokyo, Tokyo, JP 54.63 TiB 7.39% 12.81 TiB 76.54%
f01828096 San Jose, California, US 33.00 TiB 4.46% 6.38 TiB 80.68%
f01832632 San Jose, California, US 33.00 TiB 4.46% 6.78 TiB 79.45%
f01926914 Ashburn, Virginia, US 22.88 TiB 3.09% 5.16 TiB 77.46%
f01926802 Ashburn, Virginia, US 22.00 TiB 2.97% 5.03 TiB 77.13%
f01926585 Ashburn, Virginia, US 22.00 TiB 2.97% 5.53 TiB 74.86%
f01924416 Singapore, Singapore, SG 17.00 TiB 2.30% 5.09 TiB 70.04%
f01889480 Boardman, Oregon, US 12.38 TiB 1.67% 1.63 TiB 86.87%

Provider Distribution

Deal Data Replication

The below table shows how each many unique data are replicated across storage providers.

⚠️ 100.00% of deals are for data replicated across less than 4 storage providers.

Unique Data Size Total Deals Made Number of Providers Deal Percentage
84.75 TiB 660.41 TiB 1 89.30%
7.34 TiB 67.72 TiB 2 9.16%
1.16 TiB 11.38 TiB 3 1.54%

Replication Distribution

Deal Data Shared with other Clients

The below table shows how many unique data are shared with other clients. Usually different applications owns different data and should not resolve to the same CID.

⚠️ CID sharing has been observed.

Other Client Application Total Deals Affected Unique CIDs Verifier
f1xtnkj5ti77f5gbzhclnpj3xt6e5dwhfzive2yca Beacon Project 71.38 TiB 182 Project Beacon, 12 LDNs LDN # 444, 446, 447, 448, 474, 475, 476, 477, 479, 480, 481, 482
f1einobkrjcjk6gfc5ov6663vrri75hwdsjfs6pmq Cansoti 33.50 TiB 163 LDN v3 multisig
f1fmdka4yaspkjp3kw65azeygbmgp5dujl6wlghti Bole digital 17.31 TiB 113 LDN v3 multisig
f1dwhevm3p2slu6itmkssznscmpkpltdiztprnpya Hiscene 16.34 TiB 93 LDN v3 multisig
f1uuzet23wng5lxu3y4iicokksydynamgmptfekdq Explainer video 13.69 TiB 81 LDN # 65
f1mn5mw2dpqz6p3s7vxrlkhkh3okdtc6t7wqro4xa Unisound 12.38 TiB 77 LDN v3 multisig
f17qd6x3leh5pa7vh6ewdaed7qhbn2mgofrokuayy Drust 1.19 TiB 3 LDN v3 multisig
f1l2hkjqfh5abpd5x6lxj4tjrphaa44pnumt5ms5i Fly Brain Anatomy - Slingshot 736.00 GiB 4 LDN # 153
f1hamzyd3tx4pdchop3l4hfeay5q6s427pxga6flq Beacon Project 448.00 GiB 1 Project Beacon, 12 LDNs LDN # 444, 446, 447, 448, 474, 475, 476, 477, 479, 480, 481, 482
f1f5edx6ofn3lf6rk3yfpldbi2pzgq2egoyu6ubpi Samdata Trade 448.00 GiB 2 LDN v3 multisig
f1727w2vwjctfo7hflr5trgqkl3c7texh7pl4grzq GMverse 384.00 GiB 2 LDN v3 multisig
f1jdmx2cuswzjlt4pnsv3t7ipmesywzxsg6q56a2a Raysun Radar Electronic Technology Co.,Ltd 128.00 GiB 2 LDN v3 multisig

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 14 days, so for now it is being closed. Please feel free to contact the Fil+ Gov team to re-open the application if it is still being processed. Thank you!

aggregation-and-compliance-bot[bot] commented 10 months ago
Client f01902002 does not follow the datacap usage rules. More info here. This application has been failing the requirements for 29 days. Please take appropiate action to fix the following DataCap usage problems. Criteria Treshold Reason
Cid Checker score > 25% The client has a CID checker score of 1%. This should be greater than 25%. To find out more about CID checker score please look at this issue: https://github.com/filecoin-project/notary-governance/issues/986