search

filecoin-project / filecoin-plus-large-datasets

Hub for client applications for DataCap at a large scale
110 stars 62 forks source link

[DataCap Application] <Resolution Rapid Refresh (HRRR) Model> #2311

Open himomo007 opened 5 months ago

himomo007 commented 5 months ago

Data Owner Name

Resolution Rapid Refresh (HRRR) Model

What is your role related to the dataset

Data Preparer

Data Owner Country/Region

United States

Data Owner Industry

Environment

Website

https://github.com/awslabs/open-data-docs/tree/main/docs/noaa/noaa-hrrr

Social Media

Website: https://www.noaa.gov/ (Website)

Total amount of DataCap being requested

15PiB

Expected size of single dataset (one copy)

3PiB

Number of replicas to store

5

Weekly allocation of DataCap requested

1PiB

On-chain address for first allocation

f1gavvrcgtksbr3p5g7clqx2xyji7usvdagd75may

Data Type of Application

Public, Open Dataset (Research/Non-Profit)

Custom multisig

Identifier

No response

Share a brief history of your project and organization

The HRRR is a NOAA real-time 3-km resolution, hourly updated, cloud-resolving, convection-allowing atmospheric model, initialized by 3km grids with 3km radar assimilation. Radar data is assimilated in the HRRR every 15 min over a 1-h period adding further detail to that provided by the hourly data assimilation from the 13km radar-enhanced.
HRRR implementations at NCEP
HRRRv1 - 30 Sept 2014
HRRRv2 - 23 Aug 2016
HRRRv3 - 12 July 2018
HRRRv4 - 2 Dec 2020

Is this project associated with other projects/ecosystem stakeholders?

No

If answered yes, what are the other projects/ecosystem stakeholders

No response

Describe the data being stored onto Filecoin

The High-Resolution Rapid Refresh (HRRR) is sourced by Global System Laboratory from National Oceanic && Atmospheric Administration. It is a NOAA real-time 3-km resolution, hourly updated, cloud-resolving, convection-allowing atmospheric model, initialized by 3km grids with 3km radar assimilation. Radar data is assimilated in the HRRR every 15 min over a 1-h period adding further detail to that provided by the hourly data assimilation from the 13km radar-enhanced Rapid Refresh.
HRRR dataset stored on AWS is the archive since 2014 with a total size of 2PiB and 38526457 Object.

Where was the data currently stored in this dataset sourced from

AWS Cloud

If you answered "Other" in the previous question, enter the details here

No response

If you are a data preparer. What is your location (Country/Region)

United States

If you are a data preparer, how will the data be prepared? Please include tooling used and technical details?

Mainly official tools such as boost or lotus

If you are not preparing the data, who will prepare the data? (Provide name and business)

No response

Has this dataset been stored on the Filecoin network before? If so, please explain and make the case why you would like to store this dataset again to the network. Provide details on preparation and/or SP distribution.

It was stored before, but there are some applications that have been closed. As a public dataset of significance to the development of human history, it deserves to be stored over and over again.

Please share a sample of the data

https://rapidrefresh.noaa.gov/hrrr/

Confirm that this is a public dataset that can be retrieved by anyone on the Network

If you chose not to confirm, what was the reason

No response

What is the expected retrieval frequency for this data

Yearly

For how long do you plan to keep this dataset stored on Filecoin

1.5 to 2 years

In which geographies do you plan on making storage deals

Greater China, Asia other than Greater China, North America, Europe, Australia (continent)

How will you be distributing your data to storage providers

HTTP or FTP server, Shipping hard drives

How do you plan to choose storage providers

Slack, Partners

If you answered "Others" in the previous question, what is the tool or platform you plan to use

No response

If you already have a list of storage providers to work with, fill out their names and provider IDs below

f02363308
f02820572  
f02829747 
f02807399
f02916995 
f02078669
f02363306   
f02917080
f02934915 
f02934916
f02934869   
f02934879

How do you plan to make deals to your storage providers

Boost client, Lotus client, Singularity

If you answered "Others/custom tool" in the previous question, enter the details here

No response

Can you confirm that you will follow the Fil+ guideline

Yes

large-datacap-requests[bot] commented 5 months ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

data-programs commented 5 months ago
KYC

This user’s identity has been verified through filplus.storage

Sunnyiscoming commented 5 months ago

Hello. Please provide ID, City, Country, Organization of each SP here.

Sunnyiscoming commented 5 months ago

"governance review needed" label appearance is normal.

Sunnyiscoming commented 5 months ago

SP List provided: [{"providerID": "f02888899","City": "Hubei", "Country": "China", "SPOrg","SXX"}, {"providerID": "f02900008","City": "XYZ", "Country": "Sigapore", "SPOrg","Bitengine"}, {"providerID": "f02820572","City": "XYZ", "Country": "USA", "SPOrg","Ding"}, {"providerID": "f02916995","City": "XYZ", "Country": "HK", "SPOrg","HK_Fil"}, {"providerID": "f02917080","City": "XYZ", "Country": "HK", "SPOrg","Datastone"}, {"providerID": "f02829747","City": "XYZ", "Country": "USA", "SPOrg","Josef"}, {"providerID": "f02934915","City": "XYZ", "Country": "HK", "SPOrg","Xing"},]

Sunnyiscoming commented 5 months ago

Datacap Request Trigger

Total DataCap requested

15PiB

Expected weekly DataCap usage rate

1PiB

Client address

f1gavvrcgtksbr3p5g7clqx2xyji7usvdagd75may

large-datacap-requests[bot] commented 5 months ago

DataCap Allocation requested

Multisig Notary address

f02049625

Client address

f1gavvrcgtksbr3p5g7clqx2xyji7usvdagd75may

DataCap allocation requested

512TiB

Id

aef6e802-6bc8-4048-9544-c1bc519991bd

woshidama323 commented 5 months ago

LGTM Will support this first round Keep tracing

woshidama323 commented 5 months ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacebxmaiopae6w72vruchoqtypj66s6cxvct5p6f4auwotmo7gj4o3u

Address

f1gavvrcgtksbr3p5g7clqx2xyji7usvdagd75may

Datacap Allocated

512.00TiB

Signer Address

f12tk3adljauwnd3hjbigpfxb7b7gdlj63p6afwtq

Id

aef6e802-6bc8-4048-9544-c1bc519991bd

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebxmaiopae6w72vruchoqtypj66s6cxvct5p6f4auwotmo7gj4o3u

liyunzhi-666 commented 5 months ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaceajgd2q26a6vlt43g6zjepnrtjgjhuriytonhrrwoicn7a5rjtd5y

Address

f1gavvrcgtksbr3p5g7clqx2xyji7usvdagd75may

Datacap Allocated

512.00TiB

Signer Address

f1pszcrsciyixyuxxukkvtazcokexbn54amf7gvoq

Id

aef6e802-6bc8-4048-9544-c1bc519991bd

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceajgd2q26a6vlt43g6zjepnrtjgjhuriytonhrrwoicn7a5rjtd5y

github-actions[bot] commented 4 months ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

filplus-checker-app[bot] commented 4 months ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard.

himomo007 commented 4 months ago

checker:manualTrigger

filplus-checker-app[bot] commented 4 months ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard.

himomo007 commented 4 months ago

Hi @fabriziogianni7 @huseyincansoylu @jbesraa @philippe，Can I get some help，thanks！～

large-datacap-requests[bot] commented 4 months ago

DataCap Allocation requested

Request number 2

Multisig Notary address

f02049625

Client address

f1gavvrcgtksbr3p5g7clqx2xyji7usvdagd75may

DataCap allocation requested

512TiB

Id

7d70c559-e4b2-41d9-b6a7-76fa9ea51bf7

1ane-1 commented 4 months ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacedf2tncl527hw7o7dkqsq5g3o5mncciv5msqpmuideaefrpqqekw2

Address

f1gavvrcgtksbr3p5g7clqx2xyji7usvdagd75may

Datacap Allocated

512.00TiB

Signer Address

f1mdk7s2vntzm6hu35yuo6vjubtrpfnb2awhgvrri

Id

7d70c559-e4b2-41d9-b6a7-76fa9ea51bf7

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedf2tncl527hw7o7dkqsq5g3o5mncciv5msqpmuideaefrpqqekw2

stcloudlisa commented 4 months ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacebvl4yth57lxxexq3fs3u6if7dq2i4e7wvzyb5l7vghri7sbfisiu

Address

f1gavvrcgtksbr3p5g7clqx2xyji7usvdagd75may

Datacap Allocated

512.00TiB

Signer Address

f1jvvltduw35u6inn5tr4nfualyd42bh3vjtylgci

Id

7d70c559-e4b2-41d9-b6a7-76fa9ea51bf7

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebvl4yth57lxxexq3fs3u6if7dq2i4e7wvzyb5l7vghri7sbfisiu

large-datacap-requests[bot] commented 4 months ago

DataCap Allocation requested

Request number 2

Multisig Notary address

f02049625

Client address

f1gavvrcgtksbr3p5g7clqx2xyji7usvdagd75may

DataCap allocation requested

512TiB

Id

44a16086-9412-4297-b6fd-63ef8b2b2ba6

Bitengine-reeta commented 4 months ago

checker:manualTrigger

filplus-checker-app[bot] commented 4 months ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard.

Bitengine-reeta commented 4 months ago

LGTM. will support this round

Bitengine-reeta commented 4 months ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacebvvsoeep3xjqvddg6nml67uws3ctpzd2yuxul5mx4amonndbddfq

Address

f1gavvrcgtksbr3p5g7clqx2xyji7usvdagd75may

Datacap Allocated

512.00TiB

Signer Address

f1jyvhxp4kmwreo22ke4itspraznpudw3uqaink5i

Id

44a16086-9412-4297-b6fd-63ef8b2b2ba6

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebvvsoeep3xjqvddg6nml67uws3ctpzd2yuxul5mx4amonndbddfq

Bitrise0111 commented 4 months ago

report shows healthy, will support for this round.

Bitrise0111 commented 4 months ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaceafdfnfmcnlisovicopzm2nr2tr63ciriptjud3y2nvtwyilcro44

Address

f1gavvrcgtksbr3p5g7clqx2xyji7usvdagd75may

Datacap Allocated

512.00TiB

Signer Address

f1nknj7ayq4o43czrtdoauggtwl43fbqatmqis3yy

Id

44a16086-9412-4297-b6fd-63ef8b2b2ba6

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceafdfnfmcnlisovicopzm2nr2tr63ciriptjud3y2nvtwyilcro44

large-datacap-requests[bot] commented 3 months ago

DataCap Allocation requested

Request number 4

Multisig Notary address

f02049625

Client address

f1gavvrcgtksbr3p5g7clqx2xyji7usvdagd75may

DataCap allocation requested

2PiB

Id

1e3d01d4-6b87-4cef-98db-6d02bb6686ea

himomo007 commented 2 months ago

checker:manualTrigger

filplus-checker-app[bot] commented 2 months ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard.