nicelove666 / Allocator-Pathway-IPFSTT

4 stars 2 forks source link

[DataCap Application] <MODRONS' LAW> - <MODRONS'LAW-EUD-1> #72

Closed Dengminer closed 2 hours ago

Dengminer commented 1 month ago

Version

1

DataCap Applicant

MODRONS' LAW

Project ID

MODRONS'LAW-EUD-1

Data Owner Name

European Union

Data Owner Country/Region

Luxembourg

Data Owner Industry

Government

Website

https://data.europa.eu/

Social Media Handle

X: https://x.com/EU_opendata

Social Media Type

Twitter

What is your role related to the dataset

Data Preparer

Total amount of DataCap being requested

5 PiB

Expected size of single dataset (one copy)

1 PiB

Number of replicas to store

5

Weekly allocation of DataCap requested

800 TiB

On-chain address for first allocation

f1cx4a3awirb4nkxw5isxxa5rt2xafjflbnx3cmzq

Data Type of Application

Public, Open Dataset (Research/Non-Profit)

Custom multisig

Identifier

No response

Share a brief history of your project and organization

At Modrons' Law, we believe that rules and laws are the fundamental principles governing the world, and that the mechanical paradigm is the key to the advancement of civilization. Humanity is progressively deepening its understanding of these principles, categorizing them into various scientific disciplines. As civilization evolves by emulating these principles—whether socially or mathematically—our comprehension of the world's rules is both validated and enhanced. Our mission is to drive this emulation forward in the field of computer science, fostering innovation and progress.

Is this project associated with other projects/ecosystem stakeholders?

No

If answered yes, what are the other projects/ecosystem stakeholders

No response

Describe the data being stored onto Filecoin

The European Data organization, led by the EU, offers a broad range of datasets across various fields, such as environment, economy, healthcare, and demographics. These datasets support research, policy-making, and innovation, aiming to increase transparency and stimulate growth through data-driven approaches. The data is designed for easy use across different sectors and regions within the EU.

Where was the data currently stored in this dataset sourced from

My Own Storage Infra

If you answered "Other" in the previous question, enter the details here

No response

If you are a data preparer. What is your location (Country/Region)

None

If you are a data preparer, how will the data be prepared? Please include tooling used and technical details?

No response

If you are not preparing the data, who will prepare the data? (Provide name and business)

No response

Has this dataset been stored on the Filecoin network before? If so, please explain and make the case why you would like to store this dataset again to the network. Provide details on preparation and/or SP distribution.

No response

Please share a sample of the data

https://data.europa.eu/data/datasets/b0203a76-ab90-4564-9789-9b7c21fb9c65~~1

Confirm that this is a public dataset that can be retrieved by anyone on the Network

If you chose not to confirm, what was the reason

No response

What is the expected retrieval frequency for this data

Yearly

For how long do you plan to keep this dataset stored on Filecoin

More than 3 years

In which geographies do you plan on making storage deals

Asia other than Greater China

How will you be distributing your data to storage providers

Cloud storage (i.e. S3), HTTP or FTP server, IPFS, Shipping hard drives

How did you find your storage providers

Slack

If you answered "Others" in the previous question, what is the tool or platform you used

No response

Please list the provider IDs and location of the storage providers you will be working with.

f03215853, US
f03218576, US
f03220176, HongKong
f03157910, Guangdong, China
f03224283, Japan
f03224551, Japan
f03230392, Vietnam
f03230423, Malaysia
f03232064, Malaysia
f03232134, Malaysia

How do you plan to make deals to your storage providers

Boost client, Lotus client

If you answered "Others/custom tool" in the previous question, enter the details here

No response

Can you confirm that you will follow the Fil+ guideline

Yes

datacap-bot[bot] commented 1 month ago

Application is waiting for allocator review

nicelove666 commented 1 month ago

Thank you for your application, please fill in https://github.com/nicelove666/IPFSTT-Client-Due-Diligence-Form/issues/new/choose.

Our main focus is:

You need to provide comprehensive SP information, including SP, geographical location, and company name. Why do you want to put the data on Filecoin? Do you have relevant data processing experience? Can you satisfy spark retrieval and store unseal files?

Dengminer commented 1 month ago

Hi, I have filled the diligence form as in https://github.com/nicelove666/IPFSTT-Client-Due-Diligence-Form/issues/9.

The SP information is provided in the application form. We want to put the data on Filecoin because it's cheap, reliable, and will work with web3 applications. And yes, we have experience in onboarding data to Filecoin network and we confirm that we will only work with SP's who support SPARK retrieval and who always store hot copies of the data.

Please let me know if you have any further questions.

nicelove666 commented 1 month ago

Could you tell us how do you prepare the data? Thank you!

Dengminer commented 1 month ago

Sure, so basically we have a set of homemade tools to combine or split the original data files to appropriate sizes, usually around 34GB, as preferred by SP's. And then we use our scripts to convert the data into .car files and deliver to SP's. Please let us know if you have additional questions.

datacap-bot[bot] commented 1 month ago

Datacap Request Trigger

Total DataCap requested

5 PiB

Expected weekly DataCap usage rate

800 TiB

DataCap Amount - First Tranche

256TiB

Client address

f1cx4a3awirb4nkxw5isxxa5rt2xafjflbnx3cmzq

datacap-bot[bot] commented 1 month ago

DataCap Allocation requested

Multisig Notary address

Client address

f1cx4a3awirb4nkxw5isxxa5rt2xafjflbnx3cmzq

DataCap allocation requested

256TiB

Id

5cd5d728-5134-4c7d-b85e-c20a1fe87dc8

datacap-bot[bot] commented 1 month ago

Application is ready to sign

datacap-bot[bot] commented 1 month ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacebuoxgwby4jk3n3yxzqizejqk5ebuddgev7a3kcexgzlzxbux7g3k

Address

f1cx4a3awirb4nkxw5isxxa5rt2xafjflbnx3cmzq

Datacap Allocated

256TiB

Signer Address

f16hmuu3w247dkkhsrbbcbeqbugmpjbxpkrpcdatq

Id

5cd5d728-5134-4c7d-b85e-c20a1fe87dc8

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebuoxgwby4jk3n3yxzqizejqk5ebuddgev7a3kcexgzlzxbux7g3k

datacap-bot[bot] commented 1 month ago

Application is Granted

datacap-bot[bot] commented 1 month ago

Client used 75% of the allocated DataCap. Consider allocating next tranche.

Dengminer commented 1 month ago

checker:manualTrigger

datacap-bot[bot] commented 1 month ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

⚠️ 33.33% of Storage Providers have retrieval success rate equal to zero.

⚠️ 100.00% of Storage Providers have retrieval success rate less than 75%.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report.

nicelove666 commented 1 month ago

We cannot sign for you until f03220176 supports spark

nicelove666 commented 1 month ago

checker:manualTrigger

datacap-bot[bot] commented 1 month ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

⚠️ 66.67% of Storage Providers have retrieval success rate less than 75%.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report.

nicelove666 commented 1 month ago

There are only 3 SPs in the first round. Will the number of SPs increase in the next round?

Dengminer commented 1 month ago

Definitely, we will onboard more SP's in the following round.

datacap-bot[bot] commented 1 month ago

Application is in Refill

datacap-bot[bot] commented 1 month ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaced6qx3uy5rvzpnrxzymhtbk5ve4eia6p73larn5cpwgl42fs2xe7o

Address

f1cx4a3awirb4nkxw5isxxa5rt2xafjflbnx3cmzq

Datacap Allocated

512TiB

Signer Address

f16hmuu3w247dkkhsrbbcbeqbugmpjbxpkrpcdatq

Id

196fe1ee-8d93-4cba-9962-f7ee4f9deaef

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaced6qx3uy5rvzpnrxzymhtbk5ve4eia6p73larn5cpwgl42fs2xe7o

datacap-bot[bot] commented 1 month ago

Application is Granted

nicelove666 commented 1 month ago

checker:manualTrigger

datacap-bot[bot] commented 1 month ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

⚠️ 66.67% of Storage Providers have retrieval success rate less than 75%.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report.

datacap-bot[bot] commented 1 month ago

Client used 75% of the allocated DataCap. Consider allocating next tranche.

nicelove666 commented 1 month ago

checker:manualTrigger

datacap-bot[bot] commented 1 month ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

⚠️ 33.33% of Storage Providers have retrieval success rate less than 75%.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report.

nicelove666 commented 1 month ago

The bot data has not been updated and cannot sign for you.

Dengminer commented 1 month ago

Now we have onboarded 4 SP's and the retrieval rate has significantly been improved, as can be seen here: https://ehume.com/#/clientDetail?clientAddress=f1cx4a3awirb4nkxw5isxxa5rt2xafjflbnx3cmzq&dataCapConsumeStr=710.71%20TB&dataCapReleasedStr=1.75%20PB&dataCapTotalStr=1.75%20PB&dataCapNotConsumeStr=1.05%20PB

Please support for the next round.

nicelove666 commented 1 month ago

checker:manualTrigger

datacap-bot[bot] commented 1 month ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

⚠️ 33.33% of Storage Providers have retrieval success rate less than 75%.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report.

Dengminer commented 1 month ago

We are working with another 4 SP's listed below. Please help approve, thanks!

f03230392 - Vietnam f03230423 - Malaysia f03232064 - Malaysia f03232134 - Malaysia

datacap-bot[bot] commented 1 month ago

Issue has been modified. Changes below:

(OLD vs NEW)

Please list the provider IDs and location of the storage providers you will be working with: f03215853, US f03218576, US f03220176, HongKong f03157910, Guangdong, China f03224283, Japan f03224551, Japan f03230392 - Vietnam f03230423 - Malaysia f03232064 - Malaysia f03232134 - Malaysia vs f03215853, US f03218576, US f03220176, HongKong f03157910, Guangdong, China f03224283, Japan f03224551, Japan State: ChangesRequested vs Granted

datacap-bot[bot] commented 1 month ago

Issue has been modified. Changes below:

(OLD vs NEW)

Please list the provider IDs and location of the storage providers you will be working with: f03215853, US f03218576, US f03220176, HongKong f03157910, Guangdong, China f03224283, Japan f03224551, Japan f03230392, Vietnam f03230423, Malaysia f03232064, Malaysia f03232134, Malaysia vs f03215853, US f03218576, US f03220176, HongKong f03157910, Guangdong, China f03224283, Japan f03224551, Japan f03230392 - Vietnam f03230423 - Malaysia f03232064 - Malaysia f03232134 - Malaysia

nicelove666 commented 1 month ago

checker:manualTrigger

datacap-bot[bot] commented 1 month ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

⚠️ 25.00% of Storage Providers have retrieval success rate less than 75%.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report.

datacap-bot[bot] commented 1 month ago

Issue information change request has been approved.

datacap-bot[bot] commented 1 month ago

Application is in Refill

datacap-bot[bot] commented 1 month ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacebygce535s42kpzmcsj5xqnsibtjdr2gf45dnucmorchigcz3nylm

Address

f1cx4a3awirb4nkxw5isxxa5rt2xafjflbnx3cmzq

Datacap Allocated

1PiB

Signer Address

f16hmuu3w247dkkhsrbbcbeqbugmpjbxpkrpcdatq

Id

2acdebf3-5d7d-44e1-8463-672d6c18cdbd

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebygce535s42kpzmcsj5xqnsibtjdr2gf45dnucmorchigcz3nylm

datacap-bot[bot] commented 1 month ago

Application is Granted

nicelove666 commented 1 month ago

The bot has been updated.Collaboration SP is the same as Disclosure SP.

datacap-bot[bot] commented 1 month ago

Client used 75% of the allocated DataCap. Consider allocating next tranche.

Dengminer commented 4 weeks ago

checker:manualTrigger

Dengminer commented 4 weeks ago

We have used up all the DC. We work very closely with our collaborating SPs to make sure they are publicly retrievable and, ideally support SPARK well. So far, all the retrieval data checks good from our end, please let us know if there are any issues and we will communicate with the SP's to work them out.

We have a small batch of data needs to be onboarded with current SPs. Please support with another small batch. Thank you very much.

Dengminer commented 3 weeks ago

checker:manualTrigger

datacap-bot[bot] commented 3 weeks ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

⚠️ 20.00% of Storage Providers have retrieval success rate equal to zero.

⚠️ 40.00% of Storage Providers have retrieval success rate less than 75%.

Deal Data Replication

⚠️ 78.45% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report.

nicelove666 commented 3 weeks ago

Fix the retrieval rate of f03224283, then we sign

nicelove666 commented 3 weeks ago

checker:manualTrigger

datacap-bot[bot] commented 3 weeks ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

⚠️ 10.00% of Storage Providers have retrieval success rate equal to zero.

⚠️ 30.00% of Storage Providers have retrieval success rate less than 75%.

Deal Data Replication

⚠️ 66.50% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report.

nicelove666 commented 3 weeks ago

According to the guidance of the governance team, this round will mainly focus on assisting existing clients. Please provide a detailed description of your technical solutions, market capabilities, and what new data you can bring to Filecoin

nicelove666 commented 3 weeks ago

checker:manualTrigger