filecoin-project / filecoin-plus-large-datasets

Hub for client applications for DataCap at a large scale
110 stars 62 forks source link

[DataCap Allocation] - Data storage of construction industry #999

Closed yvetteoor closed 9 months ago

yvetteoor commented 1 year ago

Large Dataset Notary Application

To apply for DataCap to onboard your dataset to Filecoin, please fill out the following.

Core Information

Please respond to the questions below by replacing the text saying "Please answer here". Include as much detail as you can in your answer.

Project details

Share a brief history of your project and organization.

Yunnan Yuke Architectural Engineering Design Co., Ltd., formerly known as Kunming LANGTU Architectural Design Consulting Co., Ltd., has been engaged in the architectural design consulting industry since 2013. The business scope includes various architectural engineering design, outdoor garden landscape design, urban and rural planning design and indoor and outdoor decoration engineering design. Also,the company can undertake engineering construction feasibility study, engineering technology planning consulting and services.

After 9 years of development and precipitation, the company has grown steadily, with complete professional support and strong technical force. It has a professional and enthusiastic designer team, rich design experience and the ability to implement large-scale comprehensive projects. 

Except conventional construction projects, our company also has rich experience in earthquake reconstruction, and participated in the reconstruction project of Zhaotong earthquake in Yunnan (the sample submitted this time includes the desgin data of Zhaotong). In September 2022, a magnitude 6.8 earthquake occurred in Luding, Sichuan, our company is arranging with the participation of the reconstruction project of Luding Earthquake,too.

The design projects are widely distributed in Western China, Myanmar, Laos and other places.with the accumulation and increase of business volume, many projects of the company have retained a large number of materials, drafts, historical modified versions, finished product drawings and other data that need to be backed up and stored for a long time. These increasing cold storage requirements, whether online or offline storage, face high storage costs and data damage risks. Our colleagues are facing the same problems as us. 

With the introduction of friends, we learned that the filecoin distributed storage project can solve these problems. Therefore, we hope to have the opportunity to join the filecoin distributed storage network to reduce storage costs and ensure the stability of long-term data storage.

What is the primary source of funding for this project?

The data storage fund of the project is provided by our company.

What other projects/ecosystem stakeholders is this project associated with?

None.

Use-case details

Describe the data being stored onto Filecoin

The data content includes design materials, drafts and finished product drawings; project effect drawing; consulting scheme; planning scheme, video, etc. The total amount of data exceeds 1PiB. To ensure data security to the maximum extent,
these data will be copied as the 4 or 5 same duplicate and distributed to different SPs 

Where was the data in this dataset sourced from?

The data mainly comes from Yunnan Yuke Architectural Engineering Design Co., Ltd.,and some of them come from the data entrusted by six companies with the same storage needs.

Can you share a sample of the data? A link to a file, an image, a table, etc., are good ways to do this.

Zhaotong Earthquake Reconstruction Project
https://drive.google.com/drive/folders/1n8Q5n0Mfkj9IkfPNruPI4PV0uEPqikQP?usp=sharing
reconnaissance
https://drive.google.com/drive/folders/1dR5eaB2-ITLOE91uvBtQLkG56ALCgp3T?usp=sharing
Planning scheme and video
https://drive.google.com/drive/folders/1iGT_pQ3bpHZGIFnCMfA_uL8gu-gIVv6T?usp=sharing
Effect Diagram
https://drive.google.com/drive/folders/1zQAstcH0cTUWlHi9uPA_Z8ZguS4yE3Uy?usp=sharing
Design draft
https://drive.google.com/drive/folders/1hb6stqULtJlQK-wc-9tKzH-tM6HA9SGe?usp=sharing
Consulting scheme
https://drive.google.com/drive/folders/1OIcSGRlh8EzUnF0sVo0mKzKVg6VdIAZp?usp=sharing

Confirm that this is a public dataset that can be retrieved by anyone on the Network (i.e., no specific permissions or access rights are required to view the data).

Of course. 

What is the expected retrieval frequency for this data?

Because it is cold stored data, the frequency of retrieval is about once a year

For how long do you plan to keep this dataset stored on Filecoin?

Permanently.

DataCap allocation plan

In which geographies (countries, regions) do you plan on making storage deals?

Considering the safe and fast data transmission, it is preferred to store it in Asia and Greater China.

How will you be distributing your data to storage providers? Is there an offline data transfer process?

Use offline transmission (including hard disk copy) to transmit data.

How do you plan on choosing the storage providers with whom you will be making deals? This should include a plan to ensure the data is retrievable in the future both by you and others.

In order to reduce storage costs, we prefer to cooperate with low-cost sp for storage. On this basis, we will examine whether SP has real data storage experience, long-term storage operation and maintenance ability and data retrieval.

According to our predetermined amount of stored data, the number of SPs to be selected is about 8-10.

At present, we have started to connect with some SPs, and also pay attention to some data trading platforms, such as BDE.

How will you be distributing deals across storage providers?

All data is about 1p. Considering security, all data wiil becopied as the 4 or 5 same duplicate and will be reasonably allocated according to the number of SPS finally selected.

Do you have the resources/funding to start making deals as soon as you receive DataCap? What support from the community would help you onboard onto Filecoin?

Yes, the funds for storage are ready.
large-datacap-requests[bot] commented 1 year ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

large-datacap-requests[bot] commented 1 year ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

large-datacap-requests[bot] commented 1 year ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

large-datacap-requests[bot] commented 1 year ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

large-datacap-requests[bot] commented 1 year ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

large-datacap-requests[bot] commented 1 year ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

large-datacap-requests[bot] commented 1 year ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

yvetteoor commented 1 year ago

Is there any update here? I've been waiting for over two weeks.

yvetteoor commented 1 year ago

What else can we do to advance the application? Anyone who can assist? @raghavrmadya @galen-mcandrew @Kevin-FF-USA @Sunnyiscoming

raghavrmadya commented 1 year ago

Data samples are .jpg files and we cannot determine the need for 3.5 PiBs. Please provide as much justification as possible to support your case as well as SP IDs

yvetteoor commented 1 year ago

Hi, @raghavrmadya ,Thanks for your reply.

  1. I have provided more source files of design drafts and effect drawings, which are all made by software such as CAD, Photoshop, PKPM, 3Dmax and Lumion. The volume of source files is relatively large, and the sample of previous effect drawings is only the final display way. In addition, I also provided more project videos. Each construction project we participated in would have a design video in the early stage, a publicity video in the later stage and a customized video according to customer needs, with many use cases. source file https://drive.google.com/drive/folders/1Sa6JkPELDzJ5VI7-77gEhVyEJG0i7EHG?usp=sharing video https://drive.google.com/drive/folders/1SLDxhmtH6ZxOq4S9PUrEMY4BmMTAnNv9?usp=sharing
  2. As for SP, we are still screening and searching for reputable SP through filrep.io and plus.fil.org. If confirmed, we will timely update it here.
yvetteoor commented 1 year ago

Hi, Raghav, I have submitted the sample. If there is anything else to add, please let me know. @raghavrmadya

yvetteoor commented 1 year ago

What else we can do to speed up the review process?

Sunnyiscoming commented 1 year ago

What's the relationship between you and the organization? Can you provide more detailed information about other storage providers participated in this program, such as you can list SPs you have contacted with at present? Could you send an email to filplus-app-review@fil.org with your official domain in order to confirm your identity? Email name should includes the issue id #999.

yvetteoor commented 1 year ago

What's the relationship between you and the organization? Can you provide more detailed information about other storage providers participated in this program, such as you can list SPs you have contacted with at present? Could you send an email to filplus-app-review@fil.org with your official domain in order to confirm your identity? Email name should includes the issue id https://github.com/filecoin-project/filecoin-plus-large-datasets/issues/999.

@Sunnyiscoming Thank you for your reply,The email has been sent, please check.

yuke

Here is SPs that we have contacted with at present ,and some more are under negotiation. We'll update here in time. Japan: f01451690 China: f0443184 f0442671 f01832877 f01880558

simonkim0515 commented 1 year ago

Datacap Request Trigger

Total DataCap requested

3.5PiB

Expected weekly DataCap usage rate

80TiB

Client address

f1it5bu33xdc2wjt3xhygckvcy3v6iz5f4ojgodii

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Multisig Notary address

f02049625

Client address

f1it5bu33xdc2wjt3xhygckvcy3v6iz5f4ojgodii

DataCap allocation requested

40TiB

Id

684cc63a-4ef1-4d83-aae0-4dcf7d54b03c

Sunnyiscoming commented 1 year ago

Hey @yvetteoor , if you still want to apply for datacap, you can ask notaries to do client due diligence in slack channel. https://app.slack.com/client/TEHTVS1L6/C036JKD8NVA/thread/C03BG1MNQ4T-1673888660.823499

yvetteoor commented 1 year ago

@Sunnyiscoming Thank you for the tip. I'll ask.

cryptowhizzard commented 1 year ago

Hi @yvetteoor

It seems you are not fully aware of the rules of fil+. I screened the miners above and none qualify.

lotus net connect f01451690 f01451690 -> {12D3KooWQKo3kw7dqNuAw8brSWiheGvcQ6LpUxGcbbo4w52e7okg: []} ERROR: failed to parse multiaddr "f01451690": must begin with /

root@proposals:~# lotus net connect f0443184 f0443184 -> {12D3KooWHSazuftYZhZcy7rBGGSbvtGA7gv1pf3TUsB4waW6iQPo: []} ERROR: failed to parse multiaddr "f0443184": must begin with /

root@proposals:~# lotus net connect f0442671 f0442671 -> {12D3KooWRLta9x6sKXHrLri54SzENYKhyEZRzSttNAwBn7uAt8rS: []} ERROR: failed to parse multiaddr "f0442671": must begin with /

root@proposals:~# lotus net connect f01832877 f01832877 -> {12D3KooWN91RquGvfHF7X16KW9svmSwp6DdP3fPQPDbJQULheqRL: []} ERROR: failed to parse multiaddr "f01832877": must begin with /

root@proposals:~# lotus net connect f01880558 f01880558 -> {12D3KooWG1nJRPMsqsLHSTZmxZiJtTuWJd4iX4vp5USiyzj4i2VB: []} ERROR: failed to parse multiaddr "f01880558": must begin with /

cryptowhizzard commented 1 year ago

Can you please fill out this form to provide us with the necessary information to make a educated decision on your LDN request if we would like to support it.

mjroddy commented 1 year ago

I don't see how this dataset is aligned to filecoin

"The dataset should be public, open, and mission aligned with Filecoin and Filecoin Plus. This also means that the data should be accessible to anyone in the network, without requiring any special permissions or access requirement.

yvetteoor commented 1 year ago

Can you please fill out this form to provide us with the necessary information to make a educated decision on your LDN request if we would like to support it.

Hello, @cryptowhizzard About SPs . During the application submission, some of the SPs we contacted have withdrawn from Filecoin project, we are still looking for partners, we will comply with fil+ requirements to select SPs. Regarding the organization information, we have sent a kyc email (https://github.com/filecoin-project/filecoin-plus-large-datasets/issues/999#issuecomment-1330309325) to fil+ team via our corporate domain email. You can check it. Here I post the kyc result of qcc for your confirmation.

yk
NewHuoPool commented 1 year ago

Are there any confirmed storage providers that you have contacted now?

yvetteoor commented 1 year ago

Hello @NewHuoPool ,Thanks for asking. We have reached out to some SPs and currently, we can confirm the following: f0443184 HK f02031264 SGP f01943910 US f01945296 JP

NewHuoPool commented 1 year ago

OK, I'm willing to support you temporarily this time, and I will pay attention to the next cid report.

NewHuoPool commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacebl53dspefhbhed62tgsnqzqccvbudxjhyekqaqges6csfoyrwaqs

Address

f1it5bu33xdc2wjt3xhygckvcy3v6iz5f4ojgodii

Datacap Allocated

40.00TiB

Signer Address

f16karfxq7lxdy7izqrzrk75jf3not34k6sg6zvcy

Id

684cc63a-4ef1-4d83-aae0-4dcf7d54b03c

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebl53dspefhbhed62tgsnqzqccvbudxjhyekqaqges6csfoyrwaqs

newwebgroup commented 1 year ago

Except for the first f0443184 which did not respond (and did not report any error), all the other nodes were reachable after Ask. Willing to support Client going forward in the first round and would like to see more compliance

image
newwebgroup commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacecag7axymt5fk5njlqwerbd5cmptvbejamzkmh7kouz6xcu5liuis

Address

f1it5bu33xdc2wjt3xhygckvcy3v6iz5f4ojgodii

Datacap Allocated

40.00TiB

Signer Address

f1e77zuityhvvw6u2t6tb5qlnsegy2s67qs4lbbbq

Id

684cc63a-4ef1-4d83-aae0-4dcf7d54b03c

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacecag7axymt5fk5njlqwerbd5cmptvbejamzkmh7kouz6xcu5liuis

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 2

Multisig Notary address

f02049625

Client address

f1it5bu33xdc2wjt3xhygckvcy3v6iz5f4ojgodii

DataCap allocation requested

80TiB

Id

b4e1656f-9958-498c-b589-e730b81a4250

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f02049625

Client address

f1it5bu33xdc2wjt3xhygckvcy3v6iz5f4ojgodii

Last two approvers

newwebgroup & not found

Rule to calculate the allocation request amount

100% of weekly dc amount requested

DataCap allocation requested

80TiB

Total DataCap granted for client so far

40TiB

Datacap to be granted to reach the total amount requested by the client (3.5PiB)

3.46PiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
null null 40TiB null 8.62TiB
filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

⚠️ 1 storage providers sealed more than 70% of total datacap - f0442671: 100.00%

⚠️ All storage providers are located in the same region.

Deal Data Replication

⚠️ 100.00% of deals are for data replicated across less than 3 storage providers.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the full report.

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

⚠️ 1 storage providers sealed more than 70% of total datacap - f0442671: 100.00%

⚠️ All storage providers are located in the same region.

Deal Data Replication

⚠️ 100.00% of deals are for data replicated across less than 3 storage providers.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the full report.

METAVERSEDATAMINING commented 1 year ago

Please explain the abnormal information.

f0442671 has sealed 100.00% of total datacap. All storage providers are located in the same region. 100.00% of deals are for data replicated across less than 3 storage providers.

yvetteoor commented 1 year ago

@METAVERSEDATAMINING thanks for asking. The initial allocation was small, according to the distribution plan disclosed in the application, the allocation of a single sp exceeded 500T, and the current allocation is within the reasonable range,Then other SPs began to be encapsulated and got the right allocation, next cid reports will gradually prove this.

METAVERSEDATAMINING commented 1 year ago

Well, I'll keep an eye on the distribution and encapsulation of this application.

METAVERSEDATAMINING commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacedfwnh53i5nt7rbwfvmspqxbkvirhjyzecjul66cfwqyebnkrtjgk

Address

f1it5bu33xdc2wjt3xhygckvcy3v6iz5f4ojgodii

Datacap Allocated

80.00TiB

Signer Address

f17idrnfnxl2mbgcgr57a6z2c6lj2qx56gvm3336i

Id

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedfwnh53i5nt7rbwfvmspqxbkvirhjyzecjul66cfwqyebnkrtjgk

large-datacap-requests[bot] commented 1 year ago

We have found some problems in the information provided in the Approved Comment. We could not find Id** field in the information provided

Please, take a look at the comment and edit the body of the comment providing all the required information.
large-datacap-requests[bot] commented 1 year ago

We have found some problems in the information provided in the Approved Comment. We could not find Id** field in the information provided

Please, take a look at the comment and edit the body of the comment providing all the required information.
OpenGate01 commented 1 year ago

Node retrieval is normal. d8d9e62f192fd7cbfc93c9461c1775a_720 The first round allocation is small, and it's in a node is acceptable. This round I'll support, but I hope you strictly follow the rules in the future.

OpenGate01 commented 1 year ago

@yvetteoor I cannot sign this application as the backend search cannot find it. I need confirmation if it's a problem with my system or for other reasons.Please help check @galen-mcandrew @fabriziogianni7

20230322210114

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

⚠️ 1 storage providers sealed more than 70% of total datacap - f0442671: 100.00%

⚠️ All storage providers are located in the same region.

Deal Data Replication

⚠️ 100.00% of deals are for data replicated across less than 3 storage providers.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the full report.

panges2 commented 1 year ago

@yvetteoor @OpenGate01 it should be there now

OpenGate01 commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaceaoubu7ofa5ycwqq3uvnv3qrmxljdkb5fptsaikdjryocxnamv2pw

Address

f1it5bu33xdc2wjt3xhygckvcy3v6iz5f4ojgodii

Datacap Allocated

80.00TiB

Signer Address

f1im4hmtbfzqnx7ir74kdaiu4ynjhgqh3sdi2snla

Id

b4e1656f-9958-498c-b589-e730b81a4250

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceaoubu7ofa5ycwqq3uvnv3qrmxljdkb5fptsaikdjryocxnamv2pw

yvetteoor commented 1 year ago

Hi @simonkim0515 @panges2 Datacap allocation is used up. request trigger.mang thanks.

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 3

Multisig Notary address

f02049625

Client address

f1it5bu33xdc2wjt3xhygckvcy3v6iz5f4ojgodii

DataCap allocation requested

160TiB

Id

b4a724a6-56a2-41b2-a972-6afcff9cf237

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

⚠️ 100.00% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the full report.

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

⚠️ 100.00% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the full report.

yvetteoor commented 1 year ago

Comparing to the previous CID report, there is a noticeable improvement. At the same time, we are constantly monitoring the storage progress of SPs, adjusting allocation strategies as needed. In the new round of storage, there will be new SPs joining.

Tom-OriginStorage commented 1 year ago

A total of 120T is packaged, but the number of nodes is only 3, which does not comply with the rules and there is no CID sharing. Please explain the plan for the next round of packaging @yvetteoor