search

filecoin-project / filecoin-plus-large-datasets

Hub for client applications for DataCap at a large scale
110 stars 62 forks source link

[DataCap Application] <CAO> <NOAA GFS-HRRR-GEFS> #2227

Open caoxinhe0108 opened 9 months ago

caoxinhe0108 commented 9 months ago

Data Owner Name

National Oceanic and Atmospheric Administration

What is your role related to the dataset

Dataset Owner

Data Owner Country/Region

Afghanistan

Data Owner Industry

Life Science / Healthcare

Website

https://registry.opendata.aws/noaa-gefs/ https://registry.opendata.aws/noaa-hrrr-pds https://registry.opendata.aws/noaa-gefs

Social Media

Website: https://www.noaa.gov/
Twitter: https://twitter.com/NOAA

Total amount of DataCap being requested

10PiB

Expected size of single dataset (one copy)

32g

Number of replicas to store

5

Weekly allocation of DataCap requested

1PiB

On-chain address for first allocation

f1ynv3f5ne5k7xxr7z5q7coro7rqpls3ilhuz3ndy

Data Type of Application

Slingshot

Custom multisig

Identifier

No response

Share a brief history of your project and organization

NOAA is an agency that enriches life through science. Their reach goes from the surface of the sun to the depths of the ocean floor as we work to keep the public informed of the changing environment around them.

The HRRR is a NOAA real-time 3-km resolution, hourly updated, cloud-resolving, convection-allowing atmospheric model, initialized by 3km grids with 3km radar assimilation. Radar data is assimilated in the HRRR every 15 min over a 1-h period adding further detail to that provided by the hourly data assimilation from the 13km radar-enhanced Rapid Refresh.

The Global Ensemble Forecast System (GEFS), previously known as the GFS Global ENSemble (GENS), is a weather forecast model made up of 21 separate forecasts, or ensemble members. The National Centers for Environmental Prediction (NCEP) started the GEFS to address the nature of uncertainty in weather observations, which is used to initialize weather forecast models. The GEFS attempts to quantify the amount of uncertainty in a forecast by generating an ensemble of multiple forecasts, each minutely different, or perturbed, from the original observations. With global coverage, GEFS is produced four times a day with weather forecasts going out to 16 days.

Is this project associated with other projects/ecosystem stakeholders?

Yes

If answered yes, what are the other projects/ecosystem stakeholders

No response

Describe the data being stored onto Filecoin

NOAA is an agency that enriches life through science. Their reach goes from the surface of the sun to the depths of the ocean floor as we work to keep the public informed of the changing environment around them.

The HRRR is a NOAA real-time 3-km resolution, hourly updated, cloud-resolving, convection-allowing atmospheric model, initialized by 3km grids with 3km radar assimilation. Radar data is assimilated in the HRRR every 15 min over a 1-h period adding further detail to that provided by the hourly data assimilation from the 13km radar-enhanced Rapid Refresh.

The Global Ensemble Forecast System (GEFS), previously known as the GFS Global ENSemble (GENS), is a weather forecast model made up of 21 separate forecasts, or ensemble members. The National Centers for Environmental Prediction (NCEP) started the GEFS to address the nature of uncertainty in weather observations, which is used to initialize weather forecast models. The GEFS attempts to quantify the amount of uncertainty in a forecast by generating an ensemble of multiple forecasts, each minutely different, or perturbed, from the original observations. With global coverage, GEFS is produced four times a day with weather forecasts going out to 16 days.

Where was the data currently stored in this dataset sourced from

AWS Cloud

If you answered "Other" in the previous question, enter the details here

No response

If you are a data preparer. What is your location (Country/Region)

None

If you are a data preparer, how will the data be prepared? Please include tooling used and technical details?

No response

If you are not preparing the data, who will prepare the data? (Provide name and business)

No response

Has this dataset been stored on the Filecoin network before? If so, please explain and make the case why you would like to store this dataset again to the network. Provide details on preparation and/or SP distribution.

No response

Please share a sample of the data

350-resule 
2022-06-27 08:00:43   34 Bytes sst.20220626/rtgssthr_grb_0.5.grib2.idx
2022-06-28 08:00:22    2.2 MiB sst.20220627/rtgssthr_grb_0.083.grib2
2022-06-28 08:00:21   34 Bytes sst.20220627/rtgssthr_grb_0.083.grib2.idx
2022-06-28 08:00:21    3.7 MiB sst.20220627/rtgssthr_grb_0.083_awips.grib2
2022-06-28 08:00:21   34 Bytes sst.20220627/rtgssthr_grb_0.083_awips.grib2.idx
2022-06-28 08:00:21  122.8 KiB sst.20220627/rtgssthr_grb_0.5.grib2
2022-06-28 08:00:21   34 Bytes sst.20220627/rtgssthr_grb_0.5.grib2.idx

Total Objects: 64802983
   Total Size: 1.7 PiB
357-resule 
2020-08-25 17:22:47  398.4 MiB hrrr_v2.20160823/conus/hrrr.t09z.wrfprsf13.grib2
2020-08-25 17:22:52  396.0 MiB hrrr_v2.20160823/conus/hrrr.t09z.wrfprsf14.grib2
2020-08-25 17:22:52  394.6 MiB hrrr_v2.20160823/conus/hrrr.t09z.wrfprsf15.grib2
2020-08-25 17:22:52  390.2 MiB hrrr_v2.20160823/conus/hrrr.t09z.wrfprsf16.grib2
2020-08-25 17:23:08  387.1 MiB hrrr_v2.20160823/conus/hrrr.t09z.wrfprsf17.grib2
2020-08-25 17:23:05  384.8 MiB hrrr_v2.20160823/conus/hrrr.t09z.wrfprsf18.grib2
2021-09-28 03:48:22   31.6 KiB index.html

Total Objects: 43282174
   Total Size: 2.1 PiB
347-resule 
2023-02-15 13:35:33   13.5 MiB gefs.20230215/00/wave/gridded/gefs.wave.t00z.p30.global.0p25.f258.grib2
2023-02-15 13:35:31    1.2 KiB gefs.20230215/00/wave/gridded/gefs.wave.t00z.p30.global.0p25.f258.grib2.idx
2023-02-15 13:38:42   13.5 MiB gefs.20230215/00/wave/gridded/gefs.wave.t00z.p30.global.0p25.f264.grib2
2023-02-15 13:37:38    1.2 KiB gefs.20230215/00/wave/gridded/gefs.wave.t00z.p30.global.0p25.f264.grib2.idx
2023-02-15 13:40:59   13.5 MiB gefs.20230215/00/wave/gridded/gefs.wave.t00z.p30.global.0p25.f270.grib2
2023-02-15 13:40:57    1.2 KiB gefs.20230215/00/wave/gridded/gefs.wave.t00z.p30.global.0p25.f270.grib2.idx
2021-09-28 03:47:57   31.6 KiB index.html

Total Objects: 116639180
   Total Size: 2.2 PiB

Confirm that this is a public dataset that can be retrieved by anyone on the Network

If you chose not to confirm, what was the reason

No response

What is the expected retrieval frequency for this data

Yearly

For how long do you plan to keep this dataset stored on Filecoin

1.5 to 2 years

In which geographies do you plan on making storage deals

Greater China, Asia other than Greater China, Africa, North America, South America

How will you be distributing your data to storage providers

Cloud storage (i.e. S3)

How do you plan to choose storage providers

Slack, Big Data Exchange

If you answered "Others" in the previous question, what is the tool or platform you plan to use

No response

If you already have a list of storage providers to work with, fill out their names and provider IDs below

f02050599 Charlie Thailand
f02063409 wangkunpeg Singapore
f02041085 TopValue Japan
f02096851 EdgeIPC Indonesia
f02047843 John Japan

How do you plan to make deals to your storage providers

Boost client, Lotus client

If you answered "Others/custom tool" in the previous question, enter the details here

No response

Can you confirm that you will follow the Fil+ guideline

Yes

large-datacap-requests[bot] commented 9 months ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

Sunnyiscoming commented 9 months ago
caoxinhe0108 commented 9 months ago
image
Sunnyiscoming commented 8 months ago

Have you completed KYC? Contact email information and sp organization name has not been all completed in the form. Please resubmit the form with complete information. The Miner IDs and entity name and locaiton should be posted here.

caoxinhe0108 commented 8 months ago

yes f02050599 Charlie Thailand f02063409 wangkunpeg Singapore f02041085 TopValue Japan f02096851 EdgeIPC Indonesia f02047843 John Japan

github-actions[bot] commented 8 months ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

caoxinhe0108 commented 8 months ago

@Sunnyiscoming can you help me approve it

github-actions[bot] commented 7 months ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

caoxinhe0108 commented 7 months ago

@Sunnyiscoming Do you have any other questions?

ghost commented 7 months ago

@caoxinhe0108 no SP entities listed on your registration form. https://github.com/filecoin-project/filecoin-plus-large-datasets/issues/2227#issuecomment-1772201357

Please update to confirm SP distribution

github-actions[bot] commented 7 months ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

Sunnyiscoming commented 7 months ago

SP List provided: [{"providerID": "f02050599", "City": "XYZ", "Country": "Thailand", "SPOrg","Charlie"}, {"providerID": "f02063409", "City": "XYZ", "Country": "Singapore","SPOrg","wangkunpeg"}, {"providerID": "f02041085", "City": "XYZ", "Country": "Japan","SPOrg","TopValue"}, {"providerID": "f02096851", "City": "XYZ", "Country": "Indonesia","SPOrg","EdgeIPC"}, {"providerID": "f02047843", "City": "XYZ", "Country": "Japan","SPOrg","XYZ"},]

large-datacap-requests[bot] commented 7 months ago

DataCap Allocation requested

Request number 5

Multisig Notary address

f02049625

Client address

f1ynv3f5ne5k7xxr7z5q7coro7rqpls3ilhuz3ndy

DataCap allocation requested

2PiB

Id

a621f3bf-015c-4b97-913c-b6f16272ae9d

SuperChaiChai commented 7 months ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacecxpbc42ey4zrewls3uakuezceb4ucvev44gc3s5gcoh2wocqnimm

Address

f1ynv3f5ne5k7xxr7z5q7coro7rqpls3ilhuz3ndy

Datacap Allocated

2.00PiB

Signer Address

f12mckci3omexgzoeosjvstcfxfe4vqw7owdia3da

Id

a621f3bf-015c-4b97-913c-b6f16272ae9d

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacecxpbc42ey4zrewls3uakuezceb4ucvev44gc3s5gcoh2wocqnimm

luobin544 commented 7 months ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacebt5yrchtdulnmdhbcgqs6frxq4rwlqqdicyamv5ml6qqsdvzpxgc

Address

f1ynv3f5ne5k7xxr7z5q7coro7rqpls3ilhuz3ndy

Datacap Allocated

2.00PiB

Signer Address

f1tbd632f6w62glfaf7wjpimacbnjiz26poyoes2q

Id

a621f3bf-015c-4b97-913c-b6f16272ae9d

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebt5yrchtdulnmdhbcgqs6frxq4rwlqqdicyamv5ml6qqsdvzpxgc

large-datacap-requests[bot] commented 7 months ago

DataCap Allocation requested

Request number 5

Multisig Notary address

f02049625

Client address

f1ynv3f5ne5k7xxr7z5q7coro7rqpls3ilhuz3ndy

DataCap allocation requested

2PiB

Id

aaa0c954-15fb-44f4-9321-bbe5d711c72a

stcloudlisa commented 7 months ago

checker:manualTrigger

filplus-checker-app[bot] commented 7 months ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard.

stcloudlisa commented 7 months ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacea5amo2bzg5khwpcuyz7eser7n6bcm776qekygo5jabrk3wilvzwm

Address

f1ynv3f5ne5k7xxr7z5q7coro7rqpls3ilhuz3ndy

Datacap Allocated

2.00PiB

Signer Address

f1jvvltduw35u6inn5tr4nfualyd42bh3vjtylgci

Id

aaa0c954-15fb-44f4-9321-bbe5d711c72a

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacea5amo2bzg5khwpcuyz7eser7n6bcm776qekygo5jabrk3wilvzwm

Tom-OriginStorage commented 6 months ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaceaviwttox2cej3zri3obbmfjcmwzy2ullv3rbe3wd7mpumixeen2u

Address

f1ynv3f5ne5k7xxr7z5q7coro7rqpls3ilhuz3ndy

Datacap Allocated

2.00PiB

Signer Address

f1q6bpjlqia6iemqbrdaxr2uehrhpvoju3qh4lpga

Id

aaa0c954-15fb-44f4-9321-bbe5d711c72a

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceaviwttox2cej3zri3obbmfjcmwzy2ullv3rbe3wd7mpumixeen2u

caoxinhe0108 commented 6 months ago

checker:manualTrigger

filplus-checker-app[bot] commented 6 months ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard.

large-datacap-requests[bot] commented 6 months ago

DataCap Allocation requested

Request number 7

Multisig Notary address

f02049625

Client address

f1ynv3f5ne5k7xxr7z5q7coro7rqpls3ilhuz3ndy

DataCap allocation requested

2PiB

Id

255d352e-7251-4e93-b3ce-06d9ae9415d8

1ane-1 commented 6 months ago

checker:manualTrigger

filplus-checker-app[bot] commented 6 months ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard.

1ane-1 commented 6 months ago

The report looks good.

1ane-1 commented 6 months ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacedzdeo6eqty7oiccvdcfrvj2dashbqiyo3osovxqyjdircyog2u3g

Address

f1ynv3f5ne5k7xxr7z5q7coro7rqpls3ilhuz3ndy

Datacap Allocated

2.00PiB

Signer Address

f1mdk7s2vntzm6hu35yuo6vjubtrpfnb2awhgvrri

Id

255d352e-7251-4e93-b3ce-06d9ae9415d8

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedzdeo6eqty7oiccvdcfrvj2dashbqiyo3osovxqyjdircyog2u3g

nj-steve commented 6 months ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacec22gqpdgyul3b5yid2wzajsyhtd4ywmvmaysbiwyhupuiivbmum4

Address

f1ynv3f5ne5k7xxr7z5q7coro7rqpls3ilhuz3ndy

Datacap Allocated

2.00PiB

Signer Address

f1xx6555qijma7igpnjspyvdunc4vfxkawnpqy5ii

Id

255d352e-7251-4e93-b3ce-06d9ae9415d8

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacec22gqpdgyul3b5yid2wzajsyhtd4ywmvmaysbiwyhupuiivbmum4

nj-steve commented 6 months ago

hope to support this round

github-actions[bot] commented 6 months ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

caoxinhe0108 commented 5 months ago

It's in progress

caoxinhe0108 commented 5 months ago

checker:manualTrigger

filplus-checker-app[bot] commented 5 months ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard.

zcfil commented 5 months ago

checker:manualTrigger

filplus-checker-app[bot] commented 5 months ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard.

a1991car commented 5 months ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacebf5vvk7gf46ei4b3enawdjngodtr5wzxubnsjth7j3aueatpyoku

Address

f1ynv3f5ne5k7xxr7z5q7coro7rqpls3ilhuz3ndy

Datacap Allocated

2.00PiB

Signer Address

f1qnumecdypgrbaebtkdfjnwt5ndacadcuas3deiq

Id

255d352e-7251-4e93-b3ce-06d9ae9415d8

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebf5vvk7gf46ei4b3enawdjngodtr5wzxubnsjth7j3aueatpyoku

luobin544 commented 5 months ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacecwxjvlhrvikp3w4vcajnu2obuw6w45mz2phhbvp42avsjvmmppvm

Address

f1ynv3f5ne5k7xxr7z5q7coro7rqpls3ilhuz3ndy

Datacap Allocated

2.00PiB

Signer Address

f1tbd632f6w62glfaf7wjpimacbnjiz26poyoes2q

Id

255d352e-7251-4e93-b3ce-06d9ae9415d8

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacecwxjvlhrvikp3w4vcajnu2obuw6w45mz2phhbvp42avsjvmmppvm

github-actions[bot] commented 5 months ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

caoxinhe0108 commented 5 months ago

It's in progress

github-actions[bot] commented 5 months ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

caoxinhe0108 commented 5 months ago

It's in progress

github-actions[bot] commented 4 months ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

caoxinhe0108 commented 4 months ago

It's in progress

github-actions[bot] commented 4 months ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

caoxinhe0108 commented 4 months ago

It's in progress

large-datacap-requests[bot] commented 3 months ago

The issue reached the total datacap requested. This should be closed