search

joshua-ne / FIL_DC_Allocator_1022

This repo serves as a bookkeeping tool for DC allocation from Allocator_1022
0 stars 0 forks source link

MOVED_FROM_ISSUE_10 - [DataCap Application] DataFortress #12

Open AlfredZKY opened 5 months ago

AlfredZKY commented 5 months ago

Version

1

DataCap Applicant

DataFortress

Project ID

DataFortress-Health-WHO-GHO24

Data Owner Name

World Health Organization

Data Owner Country/Region

Geneva, Switzerland

Data Owner Industry

Life Science / Healthcare

Website

https://www.who.int/

Social Media Handle

Twitter: https://x.com/WHO

Social Media Type

Twitter

What is your role related to the dataset

Dataset Owner

Total amount of DataCap being requested

3PiB

Expected size of single dataset (one copy)

500 TiB

Number of replicas to store

4

Weekly allocation of DataCap requested

600 TiB

On-chain address for first allocation

f1c43foqnlfesp5hjfa4bupo42c5u2ij6ji54hn5i

Data Type of Application

Public, Open Dataset (Research/Non-Profit)

Custom multisig

Identifier

No response

Share a brief history of your project and organization

Data Fortress is a leading organization dedicated to providing exceptional data services for AI firms and data science research. We specialize in data acquisition, storage, and analysis, ensuring our clients access accurate, secure, and comprehensive data solutions. Our mission is to empower AI and research projects with the highest quality data, driving innovation and excellence in the field of data science.

Is this project associated with other projects/ecosystem stakeholders?

No

If answered yes, what are the other projects/ecosystem stakeholders

No response

Describe the data being stored onto Filecoin

Providing better health information around the world is a core goal of the World Health Organization, which publicly releases their global health data through the Global Health Observatory (GHO). The GHO is a portal for understanding and analyzing the health situation and major issues.
The various datasets are organized by themes such as mortality, health systems, communicable and noncommunicable diseases, medicines and vaccines, health risks, etc. WHO health statistics are the best source of global health information and are also used by the Centers for Disease Control and Prevention in the United States.

Where was the data currently stored in this dataset sourced from

AWS Cloud

If you answered "Other" in the previous question, enter the details here

No response

If you are a data preparer. What is your location (Country/Region)

None

If you are a data preparer, how will the data be prepared? Please include tooling used and technical details?

No response

If you are not preparing the data, who will prepare the data? (Provide name and business)

No response

Has this dataset been stored on the Filecoin network before? If so, please explain and make the case why you would like to store this dataset again to the network. Provide details on preparation and/or SP distribution.

No response

Please share a sample of the data

http://www.who.int/gho/maternal_health/reproductive_health/en/

Confirm that this is a public dataset that can be retrieved by anyone on the Network

If you chose not to confirm, what was the reason

No response

What is the expected retrieval frequency for this data

Monthly

For how long do you plan to keep this dataset stored on Filecoin

1.5 to 2 years

In which geographies do you plan on making storage deals

Asia other than Greater China

How will you be distributing your data to storage providers

Cloud storage (i.e. S3), IPFS, Shipping hard drives

How did you find your storage providers

Slack, Filmine

If you answered "Others" in the previous question, what is the tool or platform you used

No response

Please list the provider IDs and location of the storage providers you will be working with.

f01422327 - Japan
f02252023 - Japan
f02252024 - Japan
f01989013 - Malaysia
f01989014 - Malaysia
f01989015 - Malaysia
f02105010 - Malaysia

Other nodes might be added later.

How do you plan to make deals to your storage providers

Boost client, Lotus client

If you answered "Others/custom tool" in the previous question, enter the details here

No response

Can you confirm that you will follow the Fil+ guideline

Yes

datacap-bot[bot] commented 5 months ago

Application is waiting for allocator review

datacap-bot[bot] commented 5 months ago

Datacap Request Trigger

Total DataCap requested

2 PiB

Expected weekly DataCap usage rate

600 TiB

DataCap Amount - First Tranche

100TiB

Client address

f1c43foqnlfesp5hjfa4bupo42c5u2ij6ji54hn5i

datacap-bot[bot] commented 5 months ago

DataCap Allocation requested

Multisig Notary address

Client address

f1c43foqnlfesp5hjfa4bupo42c5u2ij6ji54hn5i

DataCap allocation requested

100TiB

Id

48009d57-708f-43d6-a371-52bff6aa20a3

datacap-bot[bot] commented 5 months ago

Application is ready to sign

AlfredZKY commented 5 months ago

Hi, here is our updated application to replace to original application: https://github.com/joshua-ne/FIL_DC_Allocator_1022/issues/10

datacap-bot[bot] commented 5 months ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaceafxpfaa7gubmpvzc43m5davffirnm26cjh2s7nccvcvbrgfyn6hq

Address

f1c43foqnlfesp5hjfa4bupo42c5u2ij6ji54hn5i

Datacap Allocated

100TiB

Signer Address

f1sfffys4o2w64rdpd3alpmvpvj4ik6x2iyjsjmry

Id

48009d57-708f-43d6-a371-52bff6aa20a3

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceafxpfaa7gubmpvzc43m5davffirnm26cjh2s7nccvcvbrgfyn6hq

datacap-bot[bot] commented 5 months ago

Application is Granted

joshua-ne commented 5 months ago

Hi @AlfredZKY I notice you are already using some of the datacap. Would you mind upload your CID report when you are ready? Thanks!

datacap-bot[bot] commented 5 months ago

Issue has been modified. Changes below:

(OLD vs NEW)

State: ChangesRequested vs Granted

AlfredZKY commented 5 months ago

Here is the csv file as requested. Please let me know if you have any questions. output.csv

AlfredZKY commented 5 months ago

My quota is running low and I would like to apply for the next allocation.

joshua-ne commented 5 months ago

I just ran a test program to check the retrievability of your SP's (report will be made public later). I can see that two of your listed SP's have a very low success rate: f01422327, and f02252024. Please check!

Meanwhile, since the majority of you SP's are good, I will for now proceed to the next round of allocation for 500TiB. But do remember to keep good retrievability. Thanks!

AlfredZKY commented 5 months ago

Hi, thank you very much for your on-time allocation. It helps us a lot.

In regard to the two SP's with low retrieve rates, f01422327 and f02252024, I have contacted the node runners. They told me that for some reason, they haven't finished the sealing yet. The data will be ready for retrieval once they finish sealing.

datacap-bot[bot] commented 5 months ago

Issue information change request has been approved.

datacap-bot[bot] commented 5 months ago

Issue has been modified. Changes below:

(OLD vs NEW)

Total Requested Amount: 2PiB vs 2 PiB State: ChangesRequested vs Granted

datacap-bot[bot] commented 5 months ago

Issue information change request has been approved.

datacap-bot[bot] commented 5 months ago

Application is in Refill

datacap-bot[bot] commented 5 months ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacecxrdhd556wmmhizxde4qv26xhmpnjnzllkv5rdbakmufca2cg3p4

Address

f1c43foqnlfesp5hjfa4bupo42c5u2ij6ji54hn5i

Datacap Allocated

512TiB

Signer Address

f1sfffys4o2w64rdpd3alpmvpvj4ik6x2iyjsjmry

Id

a81b26ed-4cc9-4c56-9eae-e9ac73938a68

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacecxrdhd556wmmhizxde4qv26xhmpnjnzllkv5rdbakmufca2cg3p4

datacap-bot[bot] commented 5 months ago

Application is Granted

datacap-bot[bot] commented 5 months ago

Application is in Refill

joshua-ne commented 5 months ago

trigger:run_retrieval_test https://github.com/user-attachments/files/15864784/output.csv

myfil512 commented 5 months ago

miner_id, dc_usage_percentage, retrieval_rate, retrieval_success_counts, retrieval_fail_counts f01422327, 5%, 70%, 14, 6 f01989013, 21%, 100%, 20, 0 f01989014, 21%, 100%, 20, 0 f01989015, 21%, 85%, 17, 3 f02105010, 21%, 75%, 15, 5 f02252023, 5%, 100%, 20, 0 f02252024, 5%, 55%, 11, 9

joshua-ne commented 4 months ago

trigger:run_retrieval_test https://github.com/user-attachments/files/15864784/output.csv

myfil512 commented 4 months ago
miner_id dc_usage_percentage retrieval_rate retrieval_success_counts retrieval_fail_counts
f01422327 5% 70% 14 6
f01989013 21% 100% 20 0
f01989014 21% 100% 20 0
f01989015 21% 85% 17 3
f02105010 21% 75% 15 5
f02252023 5% 100% 20 0
f02252024 5% 85% 17 3
joshua-ne commented 4 months ago

trigger:run_retrieval_test https://github.com/user-attachments/files/15864784/output.csv

myfil512 commented 4 months ago
miner_id dc_usage_percentage retrieval_rate retrieval_success_counts retrieval_fail_counts
f01422327 5% 70% 14 6
f01989013 21% 100% 20 0
f01989014 21% 100% 20 0
f01989015 21% 85% 17 3
f02105010 21% 75% 15 5
f02252023 5% 100% 20 0
f02252024 5% 90% 18 2
joshua-ne commented 4 months ago

@AlfredZKY I can see that retrieval is getting better, especially for f02252024. Congratulations!

Remember to upload your latest deal/data_cid lists as before whenever you are ready.

datacap-bot[bot] commented 4 months ago

Issue has been modified. Changes below:

(OLD vs NEW)

State: ChangesRequested vs ReadyToSign

datacap-bot[bot] commented 4 months ago

Issue information change request has been approved.

datacap-bot[bot] commented 4 months ago

Application is in Refill

AlfredZKY commented 4 months ago

Here is the csv file as requested. Please let me know if you have any questions. output.csv

mitchellsoo commented 4 months ago

checker:manualTrigger

datacap-bot[bot] commented 4 months ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report.

joshua-ne commented 4 months ago

trigger:run_retrieval_test https://github.com/user-attachments/files/15896744/output.csv

myfil512 commented 4 months ago
miner_id dc_usage_percentage retrieval_rate retrieval_success_counts retrieval_fail_counts
f01422327 9% 80% 16 4
f01989013 17% 65% 13 7
f01989014 17% 50% 10 10
f01989015 17% 95% 19 1
f02105010 17% 100% 20 0
f02252023 9% 40% 8 12
f02252024 9% 95% 19 1
joshua-ne commented 4 months ago

trigger:run_retrieval_test https://github.com/user-attachments/files/15896744/output.csv

myfil512 commented 4 months ago
miner_id dc_usage_percentage retrieval_rate retrieval_success_counts retrieval_fail_counts
f01422327 9% 80% 16 4
f01989013 17% 100% 20 0
f01989014 17% 100% 20 0
f01989015 17% 100% 20 0
f02105010 17% 100% 20 0
f02252023 9% 40% 8 12
f02252024 9% 95% 19 1
joshua-ne commented 4 months ago

@AlfredZKY Retrieval looks good overall so far, but you may want to contact the maintainer of f02252023 to see if there is a problem with sealing or retrieving. And let me know if you need more DataCap soon.

joshua-ne commented 4 months ago

trigger:run_retrieval_test https://github.com/user-attachments/files/15896744/output.csv

myfil512 commented 4 months ago
miner_id dc_usage_percentage retrieval_rate retrieval_success_counts retrieval_fail_counts
f01422327 9% 80% 16 4
f01989013 17% 100% 20 0
f01989014 17% 100% 20 0
f01989015 17% 100% 20 0
f02105010 17% 100% 20 0
f02252023 9% 40% 8 12
f02252024 9% 95% 19 1
AlfredZKY commented 4 months ago

Hi, we are running critically low on Datacap, could you allocate another round for us? We have contacted the maintainer of f02252023, who replied that they are currently going through some technical issues and expect to recover for successful retrieval very soon.

joshua-ne commented 4 months ago

Okay, that sounds reasonable. I will continue to support another round.

datacap-bot[bot] commented 4 months ago

Issue has been modified. Changes below:

(OLD vs NEW)

Total Requested Amount: 3PiB vs 2PiB State: ChangesRequested vs ReadyToSign

datacap-bot[bot] commented 4 months ago

Issue information change request has been approved.

datacap-bot[bot] commented 4 months ago

Application is in Refill

datacap-bot[bot] commented 4 months ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacec3hkxcsdqk2uxxtj2hjfc36u3u4g6pco5byenr5bdnghjeqcfiim

Address

f1c43foqnlfesp5hjfa4bupo42c5u2ij6ji54hn5i

Datacap Allocated

1PiB

Signer Address

f1sfffys4o2w64rdpd3alpmvpvj4ik6x2iyjsjmry

Id

3a9ff18f-74a6-4700-81c1-de5330403b1d

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacec3hkxcsdqk2uxxtj2hjfc36u3u4g6pco5byenr5bdnghjeqcfiim

datacap-bot[bot] commented 4 months ago

Application is Granted

joshua-ne commented 4 months ago

Note： I have temporarily updated the "Total amount of DataCap being requested" from 2PiB to 3PiB to bypass the signing bug mentioned in https://docs.google.com/document/d/1_gIJK5vC9_LeINE6djIsRdV5jSxmhDtlrzDZPcNvSf8/edit#heading=h.su4ryyzgqbrm

joshua-ne commented 4 months ago

@AlfredZKY Hi, the new round has been allocated and please do remember to upload the csv file containing deal info when you are ready.

datacap-bot[bot] commented 4 months ago

Application is in Refill