filecoin-project / filecoin-plus-large-datasets

Hub for client applications for DataCap at a large scale
110 stars 62 forks source link

[DataCap Application] Coupled Model Intercomparison Project 6 (CMIP6) [2/3] #1891

Closed VincentShii closed 1 year ago

VincentShii commented 1 year ago

Data Owner Name

ESGF and Pangeo

Data Owner Country/Region

United States

Data Owner Industry

Life Science / Healthcare

Website

https://www.wcrp-climate.org/wgcm-cmip/wgcm-cmip6

Social Media

https://pangeo-data.github.io/pangeo-cmip6-cloud/

Total amount of DataCap being requested

5PiB

Weekly allocation of DataCap requested

1PiB

On-chain address for first allocation

f1dsl4vadldbyw32syqpzoqm2t2rax336g6o3jcci

Data Type of Application

Public, Open Dataset (Research/Non-Profit)

Custom multisig

Identifier

No response

Share a brief history of your project and organization

Organization: Yuga was founded in 2019 with core members mainly from Microsoft, AWS and other major leading organizations. Till now we have about 20 employees in Shanghai office. We've been dedicated to building and optimizing industry-leading order encapsulation components for three years, enabling rapid processing and encapsulation of real data, and continuously empowering storage providers and data content providers.

Project: CMIP began in 1995 under the auspices of the Working Group on Coupled Modelling (WGCM). The first set of common experiments involved comparing the model response to an idealized forcing – a constant rate of increase which was accomplished using a CO2 increase of 1% per year compounded.

Is this project associated with other projects/ecosystem stakeholders?

Yes

If answered yes, what are the other projects/ecosystem stakeholders

No response

Describe the data being stored onto Filecoin

Zarr formatted data
The sixth phase of global coupled ocean-atmosphere general circulation model ensemble.

Where was the data currently stored in this dataset sourced from

AWS Cloud

If you answered "Other" in the previous question, enter the details here

No response

How do you plan to prepare the dataset

lotus

If you answered "other/custom tool" in the previous question, enter the details here

No response

Please share a sample of the data

https://registry.opendata.aws/cmip6/
s3://cmip6-pds/

Confirm that this is a public dataset that can be retrieved by anyone on the Network

If you chose not to confirm, what was the reason

No response

What is the expected retrieval frequency for this data

Yearly

For how long do you plan to keep this dataset stored on Filecoin

1.5 to 2 years

In which geographies do you plan on making storage deals

Asia other than Greater China, North America, South America, Europe

How will you be distributing your data to storage providers

Shipping hard drives, Lotus built-in data transfer

How do you plan to choose storage providers

Slack, Filmine, Big data exchange, Partners

If you answered "Others" in the previous question, what is the tool or platform you plan to use

No response

If you already have a list of storage providers to work with, fill out their names and provider IDs below

No response

How do you plan to make deals to your storage providers

Lotus client

If you answered "Others/custom tool" in the previous question, enter the details here

No response

Can you confirm that you will follow the Fil+ guideline

Yes

large-datacap-requests[bot] commented 1 year ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

large-datacap-requests[bot] commented 1 year ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

Sunnyiscoming commented 1 year ago

Datacap Request Trigger

Total DataCap requested

5PiB

Expected weekly DataCap usage rate

1PiB

Client address

f1dsl4vadldbyw32syqpzoqm2t2rax336g6o3jcci

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Multisig Notary address

f02049625

Client address

f1dsl4vadldbyw32syqpzoqm2t2rax336g6o3jcci

DataCap allocation requested

256TiB

Id

34d25739-22ad-437d-9f92-2fd7aff76809

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Multisig Notary address

f02049625

Client address

f1dsl4vadldbyw32syqpzoqm2t2rax336g6o3jcci

DataCap allocation requested

256TiB

Id

65d6f20c-51c9-4aa2-941c-5eafdd726425

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report[^1]

No application info found for this issue on https://filplus.d.interplanetary.one/clients.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report[^1]

No application info found for this issue on https://filplus.d.interplanetary.one/clients.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

Casey-PG commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaceajwdajxm2niwbph5my3moqmnqu2kpwucswhyj5nm4msm3bvmm35i

Address

f1dsl4vadldbyw32syqpzoqm2t2rax336g6o3jcci

Datacap Allocated

256.00TiB

Signer Address

f1d4yb3wags3mtddzesxoo63jv7dmlec3bq4yteni

Id

65d6f20c-51c9-4aa2-941c-5eafdd726425

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceajwdajxm2niwbph5my3moqmnqu2kpwucswhyj5nm4msm3bvmm35i

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 2

Multisig Notary address

f02049625

Client address

f1dsl4vadldbyw32syqpzoqm2t2rax336g6o3jcci

DataCap allocation requested

512TiB

Id

58bfdf77-81ac-42cf-bbd2-71303a10a8d6

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f02049625

Client address

f1dsl4vadldbyw32syqpzoqm2t2rax336g6o3jcci

Rule to calculate the allocation request amount

10% of total dc amount requested

DataCap allocation requested

512TiB

Total DataCap granted for client so far

256TiB

Datacap to be granted to reach the total amount requested by the client (5PiB)

4.75PiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
null null 256TiB null 67.40TiB
filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

⚠️ 98.62% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the full report.

BobbyChoii commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacebesl3kzyqzyomjbsnwynm5ztoxrfkmbh3a2jbxkdpju6wraht74q

Address

f1dsl4vadldbyw32syqpzoqm2t2rax336g6o3jcci

Datacap Allocated

512.00TiB

Signer Address

f1irqs2gmctiv3jcdfwuch7oxvf4ixh3k4b2wc24i

Id

58bfdf77-81ac-42cf-bbd2-71303a10a8d6

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebesl3kzyqzyomjbsnwynm5ztoxrfkmbh3a2jbxkdpju6wraht74q

BobbyChoii commented 1 year ago

There're lots of inputs from many notaries and reasonable responses from the applicant. I'm willing to give my sign and help to onboard this application.

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 3

Multisig Notary address

f02049625

Client address

f1dsl4vadldbyw32syqpzoqm2t2rax336g6o3jcci

DataCap allocation requested

1PiB

Id

fc136556-183c-4d64-b462-e703e4e51b35

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f02049625

Client address

f1dsl4vadldbyw32syqpzoqm2t2rax336g6o3jcci

Rule to calculate the allocation request amount

20% of total dc amount requested

DataCap allocation requested

1PiB

Total DataCap granted for client so far

465661.3YiB

Datacap to be granted to reach the total amount requested by the client (5PiB)

-5.62B

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
null null 512TiB null 101.28TiB
Casey-PG commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacecuvah2c5t74jlomntbyuvtcuiqvf5mvkzyunh4t2oqeoiz2zsm5q

Address

f1dsl4vadldbyw32syqpzoqm2t2rax336g6o3jcci

Datacap Allocated

1.00PiB

Signer Address

f1d4yb3wags3mtddzesxoo63jv7dmlec3bq4yteni

Id

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacecuvah2c5t74jlomntbyuvtcuiqvf5mvkzyunh4t2oqeoiz2zsm5q

MEIYAN666 commented 1 year ago

Public data, complaint allocation plan. Willing to support.

MEIYAN666 commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacea6cm3ahxoyrmuk422f76sltmg4wt64mypsmovpb3ypoh4jfameei

Address

f1dsl4vadldbyw32syqpzoqm2t2rax336g6o3jcci

Datacap Allocated

1.00PiB

Signer Address

f1bwugfihrmn3iyunzyxst5nttql3dge4khwmurtq

Id

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacea6cm3ahxoyrmuk422f76sltmg4wt64mypsmovpb3ypoh4jfameei

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 4

Multisig Notary address

f02049625

Client address

f1dsl4vadldbyw32syqpzoqm2t2rax336g6o3jcci

DataCap allocation requested

2PiB

Id

7160916c-2d9f-477c-805b-8d51f248d21f

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f02049625

Client address

f1dsl4vadldbyw32syqpzoqm2t2rax336g6o3jcci

Rule to calculate the allocation request amount

40% of total dc amount requested

DataCap allocation requested

2PiB

Total DataCap granted for client so far

931322574615478927360.0YiB

Datacap to be granted to reach the total amount requested by the client (5PiB)

-1.12B

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
null null 1PiB null 268.12TiB
Casey-PG commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacebod6j7jn7x6bb57puowvzfrsmz4v6svzi4lpigrobrmdfpxcbmwg

Address

f1dsl4vadldbyw32syqpzoqm2t2rax336g6o3jcci

Datacap Allocated

2.00PiB

Signer Address

f1d4yb3wags3mtddzesxoo63jv7dmlec3bq4yteni

Id

7160916c-2d9f-477c-805b-8d51f248d21f

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebod6j7jn7x6bb57puowvzfrsmz4v6svzi4lpigrobrmdfpxcbmwg

Bennyyangpu commented 1 year ago

I did Q&A with this client, dataset is authentic and allocation plan is reasonable. Willing to sign for this round!

Bennyyangpu commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacedyajcd2y3mv2oddnajg2rw4rvdvcsc75nshpxdogojiuhyfsolmu

Address

f1dsl4vadldbyw32syqpzoqm2t2rax336g6o3jcci

Datacap Allocated

2.00PiB

Signer Address

f174fg3bqbln3zjnkxtyf6s54txqkr7yqkj6cig7y

Id

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedyajcd2y3mv2oddnajg2rw4rvdvcsc75nshpxdogojiuhyfsolmu

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 14 days, so for now it is being closed. Please feel free to re-open if this is relevant, or start a new application for DataCap anytime. Thank you!

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 14 days, so for now it is being closed. Please feel free to contact the Fil+ Gov team to re-open the application if it is still being processed. Thank you!

Sunnyiscoming commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

⚠️ 56.35% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

Sunnyiscoming commented 1 year ago

It seems that the retrieval rate of some sp shows 0, can you explain?

VincentShii commented 1 year ago

@Sunnyiscoming I have commanded SPs to support retrieval. Some SPs are updating their code.

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

VincentShii commented 1 year ago

keep open

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 5

Multisig Notary address

f02049625

Client address

f1dsl4vadldbyw32syqpzoqm2t2rax336g6o3jcci

DataCap allocation requested

1.25PiB

Id

aa2cd075-f1b1-4ba7-b49b-478c650ea3d9

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f02049625

Client address

f1dsl4vadldbyw32syqpzoqm2t2rax336g6o3jcci

Rule to calculate the allocation request amount

400% weekly > 2PiB, requesting 2PiB

DataCap allocation requested

1.25PiB

Total DataCap granted for client so far

1.862645149230957e+37YiB

Datacap to be granted to reach the total amount requested by the client (5PiB)

1.862645149230957e+37YiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
69018 33 2PiB 9.57 1.09PiB
Suyanj commented 1 year ago

checker:manualTrigger

Suyanj commented 1 year ago

image

Suyanj commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzaceal6dk5y3fllhbnosgn5bjuvc7dly4dnjm62vdr64yttpnh2cp5zg

Address

f1dsl4vadldbyw32syqpzoqm2t2rax336g6o3jcci

Datacap Allocated

1.25PiB

Signer Address

f1ihv7gz3vn3xqvikpt4rwryecgisl7745lodx3yi

Id

aa2cd075-f1b1-4ba7-b49b-478c650ea3d9

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceal6dk5y3fllhbnosgn5bjuvc7dly4dnjm62vdr64yttpnh2cp5zg

Casey-PG commented 1 year ago

Retrieval success rates are improving quickly, keep up the good work!

Casey-PG commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaceaaifnpw3unieh7lhduugerxospbyqgojaazmujjvqj3rmlazcbng

Address

f1dsl4vadldbyw32syqpzoqm2t2rax336g6o3jcci

Datacap Allocated

1.25PiB

Signer Address

f1d4yb3wags3mtddzesxoo63jv7dmlec3bq4yteni

Id

aa2cd075-f1b1-4ba7-b49b-478c650ea3d9

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceaaifnpw3unieh7lhduugerxospbyqgojaazmujjvqj3rmlazcbng

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

VincentShii commented 1 year ago

keep open

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

VincentShii commented 1 year ago

keep open

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 14 days, so for now it is being closed. Please feel free to contact the Fil+ Gov team to re-open the application if it is still being processed. Thank you!

-- Commented by Stale Bot.