filecoin-project / filecoin-plus-large-datasets

Hub for client applications for DataCap at a large scale
110 stars 62 forks source link

[DataCap Application] Brazil data for Drust-Agee #1050

Closed Agee-Drust closed 9 months ago

Agee-Drust commented 1 year ago

Large Dataset Notary Application

To apply for DataCap to onboard your dataset to Filecoin, please fill out the following.

Core Information

Please respond to the questions below by replacing the text saying "Please answer here". Include as much detail as you can in your answer.

Project details

Share a brief history of your project and organization.

I'm a member in Filecoin, but I have ambitious with Filecoin and Filecoin slingshot about ensuring meaningful data is stored and retrieved from the Filecoin Network.

What is the primary source of funding for this project?

My own source.

What other projects/ecosystem stakeholders is this project associated with?

No

Use-case details

Describe the data being stored onto Filecoin

Earth Observation Data Cubes for Brazil - Sentinel 2A/2B Earth Observation Data Cubes for Brazil - CBERS 4
Notifications for new EO Data Cubes Sentinel-2 scenes
Notifications for new EO Data Cubes CBERS scenes

Where was the data in this dataset sourced from?

The data is from AWS Open dataset.

Can you share a sample of the data? A link to a file, an image, a table, etc., are good ways to do this.

https://registry.opendata.aws/brazil-data-cubes/

Confirm that this is a public dataset that can be retrieved by anyone on the Network (i.e., no specific permissions or access rights are required to view the data).

I confirm. This is a open dataset.

What is the expected retrieval frequency for this data?

Follow LDN Application rules. 1-2 times half a year.

For how long do you plan to keep this dataset stored on Filecoin?

More than 1 year.

DataCap allocation plan

In which geographies (countries, regions) do you plan on making storage deals?

All regions. 

How will you be distributing your data to storage providers? Is there an offline data transfer process?

After CAR files are ready, I will upload it to the server for SP, or I will prepare disks. Then I will propose offline deals after they get files.

How do you plan on choosing the storage providers with whom you will be making deals? This should include a plan to ensure the data is retrievable in the future both by you and others.

I plan to make deals with SPs that support data retrieval and have stable sealing rate. Also, I welcome every one who are interested with this deal.

How will you be distributing deals across storage providers?

Try to make every SP under 25% of deals.

Do you have the resources/funding to start making deals as soon as you receive DataCap? What support from the community would help you onboard onto Filecoin?

Enough.
github-actions[bot] commented 10 months ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

Bennyyangpu commented 10 months ago

checker:manualTrigger

filplus-checker-app[bot] commented 10 months ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

⚠️ 68.70% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard.

Bennyyangpu commented 10 months ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacedzkqfw67sjfg3ycuylgzr4qlh6x4ds63j6ww7ozdrxhglq2dxrey

Address

f1xskwjewhafb6db23zrhh2uok4kxitcr47acwhgi

Datacap Allocated

400.00TiB

Signer Address

f174fg3bqbln3zjnkxtyf6s54txqkr7yqkj6cig7y

Id

783e1e74-3405-4ba8-8f9f-3f1118d3762d

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedzkqfw67sjfg3ycuylgzr4qlh6x4ds63j6ww7ozdrxhglq2dxrey

Sunnyiscoming commented 10 months ago

Hello, @Agee-Drust per the https://github.com/filecoin-project/notary-governance/issues/922 for Open, Public Dataset applicants, please complete the following Fil+ registration form to identify yourself as the applicant and also please add the contact information of the SP entities you are working with to store copies of the data.

This information will be reviewed by Fil+ Governance team to confirm validity and then the application will be allowed to move forward for additional notary review.

Agee-Drust commented 9 months ago

@Sunnyiscoming OK. I finished it.

herrehesse commented 9 months ago

also please add the contact information of the SP entities you are working with to store copies of the data.

herrehesse commented 9 months ago

checker:manualTrigger

filplus-checker-app[bot] commented 9 months ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

⚠️ 47.94% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard.

github-actions[bot] commented 9 months ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

Agee-Drust commented 9 months ago

also please add the contact information of the SP entities you are working with to store copies of the data.

Yes, I have added it to the form. Thanks for the comments.

herrehesse commented 9 months ago

checker:manualTrigger

filplus-checker-app[bot] commented 9 months ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

⚠️ 47.94% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard.

herrehesse commented 9 months ago

Screenshot 2023-11-29 at 09 53 06

@Agee-Drust all data has been stored in the same region, against FIL+ rules. A lot of the datacap has gone to known abusive miners/entities.

Screenshot 2023-11-29 at 09 54 07

Suggest closure of this LDN @Filplus-govteam

Agee-Drust commented 9 months ago

@herrehesse Can you show me the rules about how to define abusive sps?

ghost commented 9 months ago

SP info provided: f01084913 | Bill | HongKong | no | bill f02368314 | Barry | Deyang, Sichuan, CN | no | Barry f02208475 | Jack Liu | Guangzhou, Guangdong,CN | no | Jack f02247136 | Vincent | | Beijing | no | Vincent f01879772 | Ronald | HongKong | no | Ronald

SPs taking deals: 01879772 | Hong Kong, Central and Western, HKBIH-Global Internet Harbor | 121.19 TiB | 4.62% | 121.19 TiB | 0.00% f02247136 | Beijing, Beijing, CNChina Mobile Communicaitons Corporation | 157.59 TiB | 6.01% | 157.59 TiB | 0.00% f02205811 | Beijing, Beijing, CNChina Mobile Communicaitons Corporation | 81.97 TiB | 3.13% | 81.97 TiB | 0.00% f02063246 | Hangzhou, Zhejiang, CNCHINA UNICOM China169 Backbone | 51.38 TiB | 1.96% | 51.38 TiB | 0.00% f02033556 | Hangzhou, Zhejiang, CNCHINA UNICOM China169 Backbone | 25.38 TiB | 0.97% | 25.38 TiB | 0.00% f01084913 | Hong Kong, Central and Western, HKChina Unicom Global | 224.84 TiB | 8.58% | 224.84 TiB | 0.00% f01083913 | Hong Kong, Central and Western, HKChina Unicom Global | 161.16 TiB | 6.15% | 161.16 TiB | 0.00% f02245242 | Wuhan, Hubei, CNCHINANET Hubei province network | 248.25 TiB | 9.47% | 248.25 TiB | 0.00% f02184300 | Wuhan, Hubei, CNCHINANET Hubei province network | 127.97 TiB | 4.88% | 127.97 TiB | 0.00% f02829771 | Chengdu, Sichuan, CNCHINANET SiChuan Telecom Internet Data Center | 155.59 TiB | 5.93% | 155.59 TiB | 0.00% f02211576 | Shenzhen, Guangdong, CNCHINANET-BACKBONE | 268.22 TiB | 10.23% | 268.22 TiB | 0.00% f02208475 | Shenzhen, Guangdong, CNCHINANET-BACKBONE | 167.31 TiB | 6.38% | 167.31 TiB | 0.00% f02816097 | Shenzhen, Guangdong, CNCHINANET-BACKBONE | 104.59 TiB | 3.99% | 104.59 TiB | 0.00% f01563172new | Shenzhen, Guangdong, CNCHINANET-BACKBONE | 80.00 TiB | 3.05% | 80.00 TiB | 0.00% f01931631 | Shenzhen, Guangdong, CNCHINANET-BACKBONE | 79.81 TiB | 3.04% | 79.81 TiB | 0.00% f0844916 | Shenzhen, Guangdong, CNCHINANET-BACKBONE | 39.94 TiB | 1.52% | 39.94 TiB | 0.00% f02130191 | Shenzhen, Guangdong, CNCHINANET-BACKBONE | 32.63 TiB | 1.24% | 32.63 TiB | 0.00% f02037983 | Shenzhen, Guangdong, CNCHINANET-BACKBONE | 13.63 TiB | 0.52% | 13.63 TiB | 0.00% f0114153 | Changsha, Hunan, CNCT-HuNan-Changsha-IDC | 65.97 TiB | 2.52% | 65.97 TiB | 0.00% f0157769 | Changsha, Hunan, CNCT-HuNan-Changsha-IDC | 64.25 TiB | 2.45% | 64.25 TiB | 0.00% f02196792 | Hong Kong, Central and Western, HKHK Cable TV Ltd | 36.38 TiB | 1.39% | 36.38 TiB | 0.00% f01975299 | Hong Kong, Central and Western, HKHONG KONG BRIDGE INFO-TECH LIMITED | 60.63 TiB | 2.31% | 60.63 TiB | 0.00% f02123612 | Seattle, Washington, USMisaka Network, Inc. | 36.81 TiB | 1.40% | 36.81 TiB | 0.00% f02368314 | Deyang, Sichuan, CNSichuan Chuanxn IDC | 198.44 TiB | 7.57% | 198.44 TiB | 0.00% f02019788 | Hong Kong, Central and Western, HKTowngas Telecommunications Fixed Network Ltd | 18.00 TiB | 0.69% | 18.00 TiB | 0.00%

@Agee-Drust not all SPs match - closing until clear SP entity and location information is provided

Agee-Drust commented 8 months ago

@Filplus-govteam Hello, the 5 sps I provided in the form are all in actual cooperation with us. It is all SPs match in your comment. And for distributed storage, we look for more sp to store our copy. I think it is under the rules. Can you please reopen my application?

large-datacap-requests[bot] commented 7 months ago

Thanks for your request! :exclamation: We have found some problems in the information provided. We could not find Website \/ Social Media field in the information provided We could not find Total amount of DataCap being requested (between 500 TiB and 5 PiB) field in the information provided We could not find Weekly allocation of DataCap requested (usually between 1-100TiB) field in the information provided We could not find On-chain address for first allocation field in the information provided We could not find Data Type of Application field in the information provided

Please, take a look at the request and edit the body of the issue providing all the required information.
large-datacap-requests[bot] commented 5 months ago

RootKeyHolders have approved multisig account. You can now request first datacap release