search

filecoin-project / filecoin-plus-large-datasets

Hub for client applications for DataCap at a large scale
110 stars 62 forks source link

[DataCap Application]Qistone Information technology #324

Closed stonecopy closed 1 year ago

stonecopy commented 2 years ago

Large Dataset Notary Application

To apply for DataCap to onboard your dataset to Filecoin, please fill out the following.

Core Information

Please respond to the questions below by replacing the text saying "Please answer here". Include as much detail as you can in your answer.

Project details

Share a brief history of your project and organization. Qistone is an bigger software development outsourcing company, which including Human resource managenent and Office automation for business.
Qistone has end-to-end "software + service" comprehensive business capabilities and strong in-depth service advantages, can provide a comprehensive IT service portfolio, and multi-directional technology development capabilities, ensuring that we can meet the diverse needs of global customers. The company provides information technology services, consulting and solutions, cloud computing, big data and Internet services, creating value for customers in cities, industries, enterprises and other fields around the world.

What is the primary source of funding for this project?

from our share holders

What other projects/ecosystem stakeholders is this project associated with?

No

Use-case details

Describe the data being stored onto Filecoin

Since we have many customers, we need to upolad Compiled binaries code files ,resource files and project data of each customer project that can be disclosed;
Especially now, our current important customers, designers need a lot of storeage to save resource files, such photo,video etc.

Where was the data in this dataset sourced from?

Some are produced by ourselves, and most of the data are submitted by customers through our SAAS system

Can you share a sample of the data? A link to a file, an image, a table, etc., are good ways to do this. 

In the data sample, it is the content that one of our clients who makes a copy backup system for psychology books needs to store.
[psw(www.xlzx.com).PDF](https://github.com/filecoin-project/filecoin-plus-large-datasets/files/8884993/psw.www.xlzx.com.PDF)

Confirm that this is a public dataset that can be retrieved by anyone on the Network (i.e., no specific permissions or access rights are required to view the data).

Sure. These data is public and it can be retrieved by anyone.

What is the expected retrieval frequency for this data?

This is just one of our redundant backups, which does not require frequent access. Maybe one year.

For how long do you plan to keep this dataset stored on Filecoin?

I hope it's long-term.

DataCap allocation plan

In which geographies (countries, regions) do you plan on making storage deals?

Greater China or somewhere in Asia.

How will you be distributing your data to storage providers? Is there an offline data transfer process?

Both online and offline would be more suiteable

How do you plan on choosing the storage providers with whom you will be making deals? This should include a plan to ensure the data is retrievable in the future both by you and others.

We've contacted some asian storage providers by whatsapp, and they need datacap to give us a low price. If PL can find more storage providers for us, we are happy to cooperate with them. We may have more than 5 nodes to make deals.

How will you be distributing deals across storage providers?

Less than 25% data for every storage providers.

Do you have the resources/funding to start making deals as soon as you receive DataCap? What support from the community would help you onboard onto Filecoin?

Of course, the funds are enough and we can pay easily.
We also have lots of non-public data. If filecoin can accept non-public data, we are happy to continue to make more deals in the future.
large-datacap-requests[bot] commented 2 years ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

large-datacap-requests[bot] commented 2 years ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

large-datacap-requests[bot] commented 2 years ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

Sunnyiscoming commented 2 years ago

@stonecopy The link of data sample only shows a few photos. They are not enough for proving there are 1 PB data already. Can you share some detailed projects' data? image

large-datacap-requests[bot] commented 2 years ago

Thanks for your request!

Heads up, you’re requesting more than the typical weekly onboarding rate of DataCap!
large-datacap-requests[bot] commented 2 years ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

stonecopy commented 2 years ago

Thank you @sunnyiscoming ! As a professional technology outsourcing provider, we need to save a large number of system files and related data storage to help our customers quickly recover the system in case of uncontrollable failures. I also resubmitted the data samples to help the community understand our storage content more easily. In addition, the SPs we contacted are all ready to start the node upon the approval of datacap. Thank you for your question.

Sunnyiscoming commented 2 years ago

image Data sample cannot be opened, please view it.

stonecopy commented 2 years ago

PSW: www.xlzx.com, I have shared it in the name. In fact, as a software outsourcing service platform, our different clients we served will have completely different data types and samples. I hope the community can understand this and support us. Thank you very much.

Kakkouii commented 2 years ago

Could you please mail to filplus@fil.org with your company domain to verify you're representing Qistone?

stonecopy commented 2 years ago

1656311333711 Looking forward to seeing progress at an early date.

stonecopy commented 2 years ago

Any update? It's been three months.

raghavrmadya commented 2 years ago

Still not convinced with the justification for 5 PiBs. Please provide more insight into the SP allocation plan and SP ID's if possible to gain community's confidence.

stonecopy commented 2 years ago

Hi @raghavrmadya , thanks for your response! Maybe I have not described it clearly before.

I'm Paul, CTO of Qistone company. As a software development provider, the public data sets we can store mainly include development files and application data. Among them, multimedia files, especially games and video files, occupy more space. I can add some more samples to help understand. BTW, previously we contacted a few SPS in the community, including f0508328, f01862229, etc. 4-5 SPs, but it is really a little bit hard to cooperate without datacap.

Thanks for your understanding, and looking forward to your support! datasample.zip https://user-images.githubusercontent.com/103362492/184180372-ebe56938-7401-4b22-87d3-e2cff1a863a4.mp4

tobbytheelephant commented 2 years ago

What exactly do you do? yOU said you do psychology textbooks, then games and video. Very suspicious

stonecopy commented 2 years ago

@tobbytheelephant Thank you for your attention. Software development enterprises need to face customers in various industries, so there are many forms of data samples. This is a very real situation. Hope to get the support of the community. @raghavrmadya

stonecopy commented 2 years ago

Hi @raghavrmadya Could you help me push this process? Thank you!

raghavrmadya commented 2 years ago

Datacap Request Trigger

Total DataCap requested

5 PiB

Expected weekly DataCap usage rate

300 TiB

Client address

f1pediuk4kncwp4qxawlope7hzfmd2ran35w54o7y

large-datacap-requests[bot] commented 2 years ago

DataCap Allocation requested

Multisig Notary address

f01858410

Client address

f1pediuk4kncwp4qxawlope7hzfmd2ran35w54o7y

DataCap allocation requested

150TiB

stonecopy commented 2 years ago

Thanks@raghavrmadya ,Could any notaries help sign this application?

MRJAVAZHAO commented 2 years ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacebwelztei5vifa6rrm6zryuvr6qbie2ozrbplebgfnqelw5jym4aq

Address

f1pediuk4kncwp4qxawlope7hzfmd2ran35w54o7y

Datacap Allocated

150.00TiB

Signer Address

f14gme3f52prtyzk6pblogrdd6b6ivp4swc6qmesi

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebwelztei5vifa6rrm6zryuvr6qbie2ozrbplebgfnqelw5jym4aq

PluskitOfficial commented 2 years ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacecsdbzzdbi7lqnzrr2zewc2kdvnpaznlmj3u3vdz3podnkjjkaudk

Address

f1pediuk4kncwp4qxawlope7hzfmd2ran35w54o7y

Datacap Allocated

150.00TiB

Signer Address

f1tgnlhtcmhwipfm7thsftxhn5k52velyjlazpvka

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacecsdbzzdbi7lqnzrr2zewc2kdvnpaznlmj3u3vdz3podnkjjkaudk

BDE-io commented 2 years ago

@stonecopy Hi! Great to see you have gotten approval for DataCap. If you are looking for storage providers to store these data or have any questions, please visit #bigdata-exchange on Filecoin Slack or reply here.

We have strong demand from a diverse group of SPs, who are actively looking to onboard more data.

large-datacap-requests[bot] commented 2 years ago

Thanks for your request!

Heads up, you’re requesting more than the typical weekly onboarding rate of DataCap!
dkkapur commented 2 years ago

OK - this one did not apply - see ErrorCode = 16. Re-requesting on behalf of client.

dkkapur commented 2 years ago

DataCap Allocation requested

Multisig Notary address

f01858410

Client address

f1pediuk4kncwp4qxawlope7hzfmd2ran35w54o7y

DataCap allocation requested

150TiB

MatrixStorage commented 2 years ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzaceaa2vorjsqr5lbkaajqhk33ljvrdbimoaia7s6bpw7hhhw6elzkuq

Address

f1pediuk4kncwp4qxawlope7hzfmd2ran35w54o7y

Datacap Allocated

150.00TiB

Signer Address

f1tbxqwjxfyv7swsdin4einirlsfquv3vnmlapley

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceaa2vorjsqr5lbkaajqhk33ljvrdbimoaia7s6bpw7hhhw6elzkuq

PluskitOfficial commented 2 years ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacebypwfcvkf2ykp7hdtt2c5ggzzxzbmjqlkqdnj2itrsf3r4g6wpeo

Address

f1pediuk4kncwp4qxawlope7hzfmd2ran35w54o7y

Datacap Allocated

150.00TiB

Signer Address

f1tgnlhtcmhwipfm7thsftxhn5k52velyjlazpvka

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebypwfcvkf2ykp7hdtt2c5ggzzxzbmjqlkqdnj2itrsf3r4g6wpeo

large-datacap-requests[bot] commented 2 years ago

DataCap Allocation requested

Request number 3

Multisig Notary address

f01858410

Client address

f1pediuk4kncwp4qxawlope7hzfmd2ran35w54o7y

DataCap allocation requested

600TiB

large-datacap-requests[bot] commented 2 years ago

Stats & Info for DataCap Allocation

Multisig Notary address

f01858410

Client address

f1pediuk4kncwp4qxawlope7hzfmd2ran35w54o7y

Last two approvers

PluskitOfficial & MatrixStorage

Rule to calculate the allocation request amount

200% of weekly dc amount requested

DataCap allocation requested

600TiB

Total DataCap granted for client so far

150TiB

Datacap to be granted to reach the total amount requested by the client (5PiB)

4.85PiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
3014 4 150TiB 39.25 37.34TiB
PluskitOfficial commented 2 years ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacecjrgrov3quep26b6vlvkgf4waecc2mymspuuvhmjh65vir2iwsfq

Address

f1pediuk4kncwp4qxawlope7hzfmd2ran35w54o7y

Datacap Allocated

600.00TiB

Signer Address

f1tgnlhtcmhwipfm7thsftxhn5k52velyjlazpvka

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacecjrgrov3quep26b6vlvkgf4waecc2mymspuuvhmjh65vir2iwsfq

large-datacap-requests[bot] commented 2 years ago

DataCap Allocation requested

Request number 4

Multisig Notary address

f01858410

Client address

f1pediuk4kncwp4qxawlope7hzfmd2ran35w54o7y

DataCap allocation requested

1.17PiB

large-datacap-requests[bot] commented 2 years ago

Stats & Info for DataCap Allocation

Multisig Notary address

f01858410

Client address

f1pediuk4kncwp4qxawlope7hzfmd2ran35w54o7y

Last two approvers

PluskitOfficial & swatchliu

Rule to calculate the allocation request amount

400% of weekly dc amount requested

DataCap allocation requested

1.17PiB

Total DataCap granted for client so far

750TiB

Datacap to be granted to reach the total amount requested by the client (5PiB)

4.26PiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
19212 6 600TiB 21.76 141.18TiB
MatrixStorage commented 2 years ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacea7ibo5ytrlqckeltsaacmxadf7pwsatbah53inlov5qkl4ppvqb4

Address

f1pediuk4kncwp4qxawlope7hzfmd2ran35w54o7y

Datacap Allocated

1.17PiB

Signer Address

f1tbxqwjxfyv7swsdin4einirlsfquv3vnmlapley

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacea7ibo5ytrlqckeltsaacmxadf7pwsatbah53inlov5qkl4ppvqb4

MRJAVAZHAO commented 2 years ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaceckos4n42akofgfbfwhwvelnborcrnu5qvsxwcsrfr6u2hzzcdluu

Address

f1pediuk4kncwp4qxawlope7hzfmd2ran35w54o7y

Datacap Allocated

1.17PiB

Signer Address

f14gme3f52prtyzk6pblogrdd6b6ivp4swc6qmesi

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceckos4n42akofgfbfwhwvelnborcrnu5qvsxwcsrfr6u2hzzcdluu

MetaWaveInfo commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacec3g6wumzjpnybozmflqfjxqvzjs7fjg3ufr4zbgbn43rmwhiposk

Address

f1pediuk4kncwp4qxawlope7hzfmd2ran35w54o7y

Datacap Allocated

1.17PiB

Signer Address

f1ktlkcxnmzxcdaoqfsunrg3vocfbmgv4n3mrn74a

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacec3g6wumzjpnybozmflqfjxqvzjs7fjg3ufr4zbgbn43rmwhiposk

PluskitOfficial commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaceczmqwztumjn5j5h3wplukxfawocbc3he4ztn4sww4xpjm7ce7wr2

Address

f1pediuk4kncwp4qxawlope7hzfmd2ran35w54o7y

Datacap Allocated

1.17PiB

Signer Address

f1tgnlhtcmhwipfm7thsftxhn5k52velyjlazpvka

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceczmqwztumjn5j5h3wplukxfawocbc3he4ztn4sww4xpjm7ce7wr2

filplus-checker commented 1 year ago

DataCap and CID Checker Report[^1]

If this is the first time a provider takes verified deal, it will be marked as new.

For most of the datacap application, below restrictions should apply.

⚠️ 31.14% of total deal sealed by f01852664 are duplicate data.

⚠️ 31.12% of total deal sealed by f01852325 are duplicate data.

⚠️ 34.83% of total deal sealed by f01852023 are duplicate data.

⚠️ 36.02% of total deal sealed by f01852677 are duplicate data.

⚠️ 31.71% of total deal sealed by f01851482 are duplicate data.

⚠️ 88.68% of total deal sealed by f01919535 are duplicate data.

Provider Location Total Deals Sealed Percentage Unique Data Duplicate Deals
f01852664 Singapore, Singapore, SG 129.75 TiB 21.31% 89.34 TiB 31.14%
f01852325 Hong Kong, Central and Western, HK 125.41 TiB 20.60% 86.38 TiB 31.12%
f01852023 Busan, Busan, KR 121.47 TiB 19.95% 79.16 TiB 34.83%
f01852677 Morrisville, North Carolina, US 112.19 TiB 18.43% 71.78 TiB 36.02%
f01851482 Busan, Busan, KR 91.34 TiB 15.00% 62.38 TiB 31.71%
f01919535 Wuhan, Hubei, CN 28.72 TiB 4.72% 3.25 TiB 88.68%

Provider Distribution

Deal Data Replication

The below table shows how each many unique data are replicated across storage providers.

⚠️ 82.28% of deals are for data replicated across less than 4 storage providers.

Unique Data Size Total Deals Made Number of Providers Deal Percentage
74.38 TiB 136.63 TiB 1 22.44%
44.03 TiB 144.91 TiB 2 23.80%
47.22 TiB 219.44 TiB 3 36.04%
20.72 TiB 102.59 TiB 4 16.85%
1.06 TiB 5.31 TiB 5 0.87%

Replication Distribution

Deal Data Shared with other Clients

The below table shows how many unique data are shared with other clients. Usually different applications owns different data and should not resolve to the same CID.

⚠️ CID sharing has been observed.

Other Client Application Total Deals Affected Unique CIDs Verifier
f3qvn7f5u4z5w5pqx3htckp4jcn5dvgmebq6qkqcz
xdxkicwokf75tt5hbwtrjwldz2sjiyq752ajcn3nd
5tgq		 Beijing Haishi Hengtong Technology Co., LTD 127.47 TiB 1,606 LDN v3 multisig
f14nyld75bnvr2y3ca4ew7vxmwp4tuwytqwggthcy Kimoc 41.56 TiB 487 LDN v3 multisig
f3w4wlayytfmsay6gu5phhij5r4yyx7t4xxrlosgo
tlmqg5eih3co5atsra4h2pe4qd2d6c76bvhj6nwim
7lgq		 GMWBR INC 26.63 TiB 336 LDN v3 multisig
f3uils5cdx3ezyzszjjfnulugknbdsanmqtisd7x7
xkfcljdnshp4jspnrgxpldt5b4aafuz4q4rkebpjy
keha		 Qingyun Education Fund 26.38 TiB 598 LDN v3 multisig
f3q7ablez3jqkcjukwbzaql7lmbx4ldouu66nexpd
cfvu6kgho3v6gricckt77cgr46tdre2l4zmvha7bs
u7qq		 MatrixStorage 13.34 TiB 188 LDN # 72
f14uxcyaoab3qhn42kaquqysga6f6zfry3x4nk3ca China Tianying INC. 8.63 TiB 131 LDN v3 multisig
f3xffjctbyy7zigopfa3za5ha3pvv4z3xfghlw7kw
vyeuabkg4lzsgfwhnghwkvmmvi6yso6k52hq3ca6c
kveq		 Chengdu Digital Media Industry Base Co., Ltd. 8.34 TiB 183 LDN v3 multisig
f1piaz4nodpwdemrfqak5jlfg5ois2onzmjv6fkki Chengdu Yundianshang Technology Co., LTD 7.84 TiB 183 LDN v3 multisig
f1elncewt3sh356aop52uvcappblxmf6asbmhxlya Beijing Wanjie Data Technology Co., LTD 3.50 TiB 21 LDN v3 multisig
f1cuboogcwais57dljrpeltoy6ja2itb7wvwmrl3q Penglaiju 1.16 TiB 26 LDN v3 multisig

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 5

Multisig Notary address

f02049625

Client address

f1pediuk4kncwp4qxawlope7hzfmd2ran35w54o7y

DataCap allocation requested

1.92PiB

Id

98a7cb3e-e73f-429d-8673-78693d0a04b4

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f01858410

Client address

f1pediuk4kncwp4qxawlope7hzfmd2ran35w54o7y

Rule to calculate the allocation request amount

800% of weekly dc amount requested

DataCap allocation requested

1.92PiB

Total DataCap granted for client so far

1.0896474123001098e+37YiB

Datacap to be granted to reach the total amount requested by the client (5PiB)

-1.31B

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
85133 20 1.17PiB 25.76 280.19TiB
github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 14 days, so for now it is being closed. Please feel free to contact the Fil+ Gov team to re-open the application if it is still being processed. Thank you!