search

filecoin-project / filecoin-plus-large-datasets

Hub for client applications for DataCap at a large scale
110 stars 62 forks source link

[DataCap Application] Fujian ZhanXun Geographic Information Co., Ltd #1419

Closed mutablepunk closed 1 year ago

mutablepunk commented 1 year ago

Large Dataset Notary Application

To apply for DataCap to onboard your dataset to Filecoin, please fill out the following.

Core Information

Please respond to the questions below by replacing the text saying "Please answer here". Include as much detail as you can in your answer.

Project details

Share a brief history of your project and organization.

Fujian ZhanXun Geographic Information Co., Ltd , founded in May 2014. The company provides services such as thematic map production and H5 application system development, and is committed to the investment development and application promotion of geographic information applications. The company has successively provided services such as information collection, application development and platform release for provincial-level units such as the Provincial Investigation Bureau、Provincial Safety Administration、Provincial Publicity Department and Provincial Tourism Bureau. At the same time, it also provides information development services for mass merchants and various enterprises. So far, it has served nearly 50 scenic spots and more than 200 enterprises in the province, and accumulated a large number of application data such as tourist attractions, special food, cultural exhibition halls, etc.

What is the primary source of funding for this project?

Company revenue

What other projects/ecosystem stakeholders is this project associated with?

None

Use-case details

Describe the data being stored onto Filecoin

This data includes geographic imagery data and panoramas. We has served nearly 50 scenic spots and more than 200 enterprises in the province, and accumulated a large number of application data such as tourist attractions, special food, cultural exhibition halls, etc.

Where was the data in this dataset sourced from?

From the geographic data in our project

Can you share a sample of the data? A link to a file, an image, a table, etc., are good ways to do this. 

https://drive.google.com/drive/folders/1RjLa99z8qSFRvLxHMEdIgpiKINbyYhjg?usp=sharing

Confirm that this is a public dataset that can be retrieved by anyone on the Network (i.e., no specific permissions or access rights are required to view the data).

Yes, Confirm.

What is the expected retrieval frequency for this data?

1-2 times a year

For how long do you plan to keep this dataset stored on Filecoin?

Three years or more

DataCap allocation plan

In which geographies (countries, regions) do you plan on making storage deals?

Greater China，Asia.

How will you be distributing your data to storage providers? Is there an offline data transfer process?

offline.

How do you plan on choosing the storage providers with whom you will be making deals? This should include a plan to ensure the data is retrievable in the future both by you and others.

We will refer to their storage efficiency and overall capacity, then choose reliable and experienced storage providers

How will you be distributing deals across storage providers?

According to the rules of the fil-plus community, we will evenly distribute them to high-quality storage providers.

Do you have the resources/funding to start making deals as soon as you receive DataCap? What support from the community would help you onboard onto Filecoin?

Yes, we have sufficient funds.
large-datacap-requests[bot] commented 1 year ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

large-datacap-requests[bot] commented 1 year ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

large-datacap-requests[bot] commented 1 year ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

simonkim0515 commented 1 year ago

Datacap Request Trigger

Total DataCap requested

2PiB

Expected weekly DataCap usage rate

100TiB

Client address

f1jqrcpugschakigqqvivbk3mfr6hx6jkf42kyqya

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Multisig Notary address

f01858410

Client address

f1jqrcpugschakigqqvivbk3mfr6hx6jkf42kyqya

DataCap allocation requested

50TiB

Id

5d356a10-1e4a-4c92-b776-d3468688e436

xiaoyuaiheshui commented 1 year ago

1.can you send email to [filplus-app-review@fil.org]； 2.Have you establish cooperation with storage providers? If yes, please list them. If not, please explain how to find them.

mutablepunk commented 1 year ago
  1. Yes, I already sent email.
  2. Yes, we have. Here are their nodes: f0216849, f01949260 and f01907578
Screen Shot 2022-12-20 at 14 22 19
xiaoyuaiheshui commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacedqsgjr33ph4huw3kr4gyfyfffuwassjqc2l25drjw2rd2habydna

Address

f1jqrcpugschakigqqvivbk3mfr6hx6jkf42kyqya

Datacap Allocated

50.00TiB

Signer Address

f122qmy25wdtt5mxd77kndiq7z5x2n3iwiuz2wdsa

Id

5d356a10-1e4a-4c92-b776-d3468688e436

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedqsgjr33ph4huw3kr4gyfyfffuwassjqc2l25drjw2rd2habydna

kernelogic commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacechoo4a2cd2ifhporpcnxbhxya2bhegmkczsqlcwk56w2evrr6psu

Address

f1jqrcpugschakigqqvivbk3mfr6hx6jkf42kyqya

Datacap Allocated

50.00TiB

Signer Address

f1yjhnsoga2ccnepb7t3p3ov5fzom3syhsuinxexa

Id

5d356a10-1e4a-4c92-b776-d3468688e436

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacechoo4a2cd2ifhporpcnxbhxya2bhegmkczsqlcwk56w2evrr6psu

kernelogic commented 1 year ago

First allocation, looking forward to next milestone

Sunnyiscoming commented 1 year ago

image The email address for sending KYB emails is not an authorized email address. Could you send an email to filplus-app-review@fil.org with your official domain in order to confirm your identity? Email name should includes the issue id #1419.

mutablepunk commented 1 year ago

Sorry, the email address used to send the email before was wrong, the new email has been sent, please see the screenshot.

WechatIMG30

Also the email address previously used on the company's official website has been discontinued and has now been updated to a new email address

Screenshot 2023-01-28 at 12 12 45
cryptowhizzard commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report[^1]

No active deals found for this client.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 2

Multisig Notary address

f01858410

Client address

f1jqrcpugschakigqqvivbk3mfr6hx6jkf42kyqya

DataCap allocation requested

100TiB

Id

0be0d103-a186-4baf-8d11-ed25f82b4a78

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f01858410

Client address

f1jqrcpugschakigqqvivbk3mfr6hx6jkf42kyqya

Last two approvers

kernelogic & jggapp

Rule to calculate the allocation request amount

100% of weekly dc amount requested

DataCap allocation requested

100TiB

Total DataCap granted for client so far

50TiB

Datacap to be granted to reach the total amount requested by the client (2 PiB)

1.95PiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
906 3 50TiB 33.33 11.84TiB
filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report[^1]

Storage Provider Distribution

The below table shows the distribution of storage providers that have stored data for this client.

If this is the first time a provider takes verified deal, it will be marked as new.

For most of the datacap application, below restrictions should apply.

Since this is the 3rd allocation, the following restrictions have been relaxed:

✔️ Storage provider distribution looks healthy.

Provider Location Total Deals Sealed Percentage Unique Data Duplicate Deals
f01949260 Shanghai, Shanghai, CN
Fuzhou		 9.44 TiB 33.33% 9.44 TiB 0.00%
f01949267 Shanghai, Shanghai, CN
Fuzhou		 9.44 TiB 33.33% 9.44 TiB 0.00%
f02016677 Hong Kong, Central and Western, HK
Kaopu Cloud HK Limited		 9.44 TiB 33.33% 9.44 TiB 0.00%

Provider Distribution

Deal Data Replication

The below table shows how each many unique data are replicated across storage providers.

Since this is the 3rd allocation, the following restrictions have been relaxed:

✔️ Data replication looks healthy.

Unique Data Size Total Deals Made Number of Providers Deal Percentage
9.44 TiB 28.31 TiB 3 100.00%

Replication Distribution

Deal Data Shared with other Clients

The below table shows how many unique data are shared with other clients. Usually different applications owns different data and should not resolve to the same CID.

However, this could be possible if all below clients use same software to prepare for the exact same dataset or they belong to a series of LDN applications for the same dataset.

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

herrehesse commented 1 year ago

@mutablepunk First allocation does not seem regionally spread out, even though your application states Asia only. I want a clear plan of distribution and a list of SP's and business names before moving forward with the next trench.

cryptowhizzard commented 1 year ago

Trying to retrieve the data what's in here. Will report back shortly.

mutablepunk commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report[^1]

Storage Provider Distribution

The below table shows the distribution of storage providers that have stored data for this client.

If this is the first time a provider takes verified deal, it will be marked as new.

For most of the datacap application, below restrictions should apply.

Since this is the 3rd allocation, the following restrictions have been relaxed:

✔️ Storage provider distribution looks healthy.

Provider Location Total Deals Sealed Percentage Unique Data Duplicate Deals
f01949260 Shanghai, Shanghai, CN
Fuzhou		 9.44 TiB 23.48% 9.44 TiB 0.00%
f01949267 Shanghai, Shanghai, CN
Fuzhou		 9.44 TiB 23.48% 9.44 TiB 0.00%
f02016677 Hong Kong, Central and Western, HK
Kaopu Cloud HK Limited		 9.44 TiB 23.48% 9.44 TiB 0.00%
f060754new Xiamen, Fujian, CN
Kaopu Cloud HK Limited		 6.97 TiB 17.34% 6.97 TiB 0.00%
f054420 Hong Kong, Central and Western, HK
SkyExchange Internet Access		 4.91 TiB 12.21% 4.91 TiB 0.00%

Provider Distribution

Deal Data Replication

The below table shows how each many unique data are replicated across storage providers.

Since this is the 3rd allocation, the following restrictions have been relaxed:

✔️ Data replication looks healthy.

Unique Data Size Total Deals Made Number of Providers Deal Percentage
11.88 TiB 11.88 TiB 1 29.55%
9.44 TiB 28.31 TiB 3 70.45%

Replication Distribution

Deal Data Shared with other Clients

The below table shows how many unique data are shared with other clients. Usually different applications owns different data and should not resolve to the same CID.

However, this could be possible if all below clients use same software to prepare for the exact same dataset or they belong to a series of LDN applications for the same dataset.

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

mutablepunk commented 1 year ago

@herrehesse @cryptowhizzard The encapsulation speed of some SP is relatively slow, and now they are all on the chain. Can you support me?

cryptowhizzard commented 1 year ago

Hi @mutablepunk

Thanks for your message. The SP's that stored your data don't see to support retrieval. I will attempt a second try over the weekend to see if i can retrieve your data. Retrievability is a key component of the FIL+ rules to enable me to check what the data is that is stored.

Thanks for your understanding.

mutablepunk commented 1 year ago

@cryptowhizzard There is one SP that does not support retrieval, and other SPs support retrieval. We have informed SP by email that we need to open the online retrieval mode. Can you sign for me before that? I will solve this problem in the second round.

C00kies77 commented 1 year ago

@mutablepunk We recommend not signing anything before retrievals on all SP's are working properly. As soon as you can show us proper results we will take another look at your application

Thanks for your understanding.

mutablepunk commented 1 year ago

@cryptowhizzard @C00kies77 The SP f054420 cannot be retrieved. SP is also trying to solve the problem. I will not send a new order to this SP until the problem is solved. Can you help me enter the second round first. Until it solves the retrieval problem, I will issue it again.

mutablepunk commented 1 year ago

F054420 email replies can already support retrieval. Can you help me enter the second round? @cryptowhizzard

mutablepunk commented 1 year ago

图片

cryptowhizzard commented 1 year ago

Hi,

I checked on f054420 but they are listed in our system as abusive SP connected to abuse LDN's. I have it in my system under #1214 , the owner of that ticket deleted it.

Tbh, i would like you to replace this SP for another one. I will test retrievals on the others.

cryptowhizzard commented 1 year ago

Feb 14 10:55:03 proposals dealscanner-f01997810-f060754: ERROR: failed to find offer satisfying maxPrice: 0 FIL. Try increasing maxPrice Feb 14 10:55:03 proposals dealscanner-f01997810-f060754: Feb 14 10:55:03 proposals dealscanner-f01997810-f054420: ERROR: failed to find offer satisfying maxPrice: 0 FIL. Try increasing maxPrice

Btw ... the issue is still not fixed. The price above is 0.1 . Retrieval should be free according to the FIL+ rules!

mutablepunk commented 1 year ago

Feb 14 10:55:03 proposals dealscanner-f01997810-f060754: ERROR: failed to find offer satisfying maxPrice: 0 FIL. Try increasing maxPrice Feb 14 10:55:03 proposals dealscanner-f01997810-f060754: Feb 14 10:55:03 proposals dealscanner-f01997810-f054420: ERROR: failed to find offer satisfying maxPrice: 0 FIL. Try increasing maxPrice

Btw ... the issue is still not fixed. The price above is 0.1 . Retrieval should be free according to the FIL+ rules!

@cryptowhizzard I have helped SP solve this problem.

mutablepunk commented 1 year ago

Hi,

I checked on f054420 but they are listed in our system as abusive SP connected to abuse LDN's. I have it in my system under #1214 , the owner of that ticket deleted it.

Tbh, i would like you to replace this SP for another one. I will test retrievals on the others.

We had in-depth exchanges before reaching cooperation with the SP. If we can't store the data according to the Fil+rule, we will stop distributing the data at any time. There is still a retrieval problem in the early stage. SP has fixed this problem yesterday. If there are other problems in the future, we will terminate the cooperation with this SP at any time

mutablepunk commented 1 year ago

@herrehesse @cryptowhizzard Hello, what can I do for you? Continue to wait?

herrehesse commented 1 year ago

I would like to see a list of SP's and their business names and regions. And why are you only storing in "Asia", if you can reach HK, SG and JP from your location you can also distribute to EU and USA.

mutablepunk commented 1 year ago

@herrehesse At present, the partners are mainly in Asia. We are also looking for suitable SP in the United States and Europe

herrehesse commented 1 year ago

@mutablepunk Love to assist you in that process, let me know if you need any assistance. Love to support if you can distribute your dataset better.

mutablepunk commented 1 year ago

@herrehesse Thanks, glad to have your help! I will choose more regions to distribute my data at a later stage! Can you help me pass this round?

herrehesse commented 1 year ago

Nope, first an SP list with more regions, then we can (if the retrieval is OK) help you sign this next round!

mutablepunk commented 1 year ago

f01949260 Fuzhou f01949267 Fuzhou f02016677 HK f060754 HK f01907578 PuTian f0123931 Fuzhou f01807707 ShenZhen Frankfurt confirming @herrehesse Here is our SP list for the second round There is also a partner of Frankfurt who is communicating. If appropriate, he will start Boost!

herrehesse commented 1 year ago

Awesome! What are their business names? Locations are noted.

mutablepunk commented 1 year ago

@herrehesse f01949260 Fuzhoushuidiwangluo Technology Co. Ltd f01949267 Fuzhoushuidiwangluo Technology Co. Ltd f02016677 Personal f060754 Shenzhenshuidixinxijishu Technology Co. Ltd f01907578 Personal f0123931 Fuzhou Blue Coast Network Technology Co. LTD f01807707 ShenzhenXingyuecunchu Technology Co. Ltd

herrehesse commented 1 year ago

@mutablepunk Thank you for being transparent. Will put the data inside our database and discuss with @cryptowhizzard about next steps. If all is in order love to sign your application.

cryptowhizzard commented 1 year ago

Hi @mutablepunk

Thanks for this. Based on above information you are almost good to go. I only have a problem with SP f01807707. It is in a subnet where much abuse has been going on.

Can we agree that you will not use that SP and onboard with the others? If so, i am ok to sign.

mutablepunk commented 1 year ago

@cryptowhizzard Thank you for your reminding and suggestion. I agree with your suggestion and terminate the cooperation with the SP

cryptowhizzard commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacectqeicwbzvr6uuu6ryta2yf4pfageqs6ml4sonb2t36ytx35erxo

Address

f1jqrcpugschakigqqvivbk3mfr6hx6jkf42kyqya

Datacap Allocated

100.00TiB

Signer Address

f1krmypm4uoxxf3g7okrwtrahlmpcph3y7rbqqgfa

Id

0be0d103-a186-4baf-8d11-ed25f82b4a78

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacectqeicwbzvr6uuu6ryta2yf4pfageqs6ml4sonb2t36ytx35erxo

YuanHeHK commented 1 year ago

The previous information is disclosed completely, and I tried to retrieve it without problems, I will support it。 mmexport1676953549316 mmexport1676953551489

YuanHeHK commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaceau4pr5dwxz775hz77xvmzaa4m5o3zozzkvpl76sqryciqd4n3lpa

Address

f1jqrcpugschakigqqvivbk3mfr6hx6jkf42kyqya

Datacap Allocated

100.00TiB

Signer Address

f1fg6jkxsr3twfnyhdlatmq36xca6sshptscds7xa

Id

0be0d103-a186-4baf-8d11-ed25f82b4a78

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceau4pr5dwxz775hz77xvmzaa4m5o3zozzkvpl76sqryciqd4n3lpa

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 3

Multisig Notary address

f01858410

Client address

f1jqrcpugschakigqqvivbk3mfr6hx6jkf42kyqya

DataCap allocation requested

200TiB

Id

aeb81f52-1795-40fa-b00a-70f32553b007

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f01858410

Client address

f1jqrcpugschakigqqvivbk3mfr6hx6jkf42kyqya

Last two approvers

fireflyHZ & cryptowhizzard

Rule to calculate the allocation request amount

200% of weekly dc amount requested

DataCap allocation requested

200TiB

Total DataCap granted for client so far

50TiB

Datacap to be granted to reach the total amount requested by the client (2 PiB)

1.95PiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
1521 5 100TiB 20.74 2.52TiB
filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

⚠️ 100.00% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the full report.