[DataCap Application] NCAR CESM2 ARISE

Jaida0 commented 1 year ago

Data Owner Name

NCAR

What is your role related to the dataset

Data Preparer

Data Owner Country/Region

United States

Data Owner Industry

Environment

Website

https://ncar.ucar.edu/

Social Media

https://www.facebook.com/NCAR.UCAR/
https://www.youtube.com/ncarucar
https://www.instagram.com/ncar_ucar/

Total amount of DataCap being requested

6PiB

Expected size of single dataset (one copy)

597.2TiB

Number of replicas to store

10

Weekly allocation of DataCap requested

800TiB

On-chain address for first allocation

f16grdbr73dccc66dxhtrr2shqoarpupowpteo2yi

Data Type of Application

Public, Open Dataset (Research/Non-Profit)

Custom multisig

[ ] Use Custom Multisig

Identifier

No response

Share a brief history of your project and organization

NCAR was established by the National Science Foundation in 1960 to provide the university community with world-class facilities and services that were beyond the reach of any individual institution. NCAR provides the atmospheric and related Earth system science community with state-of-the-art resources, including supercomputers, research aircraft, sophisticated computer models, and extensive data sets.

Is this project associated with other projects/ecosystem stakeholders?

No

If answered yes, what are the other projects/ecosystem stakeholders

No response

Describe the data being stored onto Filecoin

Project data files from ARISE-SAI Experiments with CESM2

Where was the data currently stored in this dataset sourced from

AWS Cloud

If you answered "Other" in the previous question, enter the details here

No response

If you are a data preparer. What is your location (City and Country)

Sichuan, China

If you are a data preparer, how will the data be prepared? Please include tooling used and technical details?

We use lotus to prepare data.

If you are not preparing the data, who will prepare the data? (Provide name and business)

No response

Has this dataset been stored on the Filecoin network before? If so, please explain and make the case why you would like to store this dataset again to the network. Provide details on preparation and/or SP distribution.

I don't know.
I will strictly follow that each sp will be sent exceed 30% of the total datacap.

Please share a sample of the data

s3://ncar-cesm2-arise/

Confirm that this is a public dataset that can be retrieved by anyone on the Network

[X] I confirm

If you chose not to confirm, what was the reason

No response

What is the expected retrieval frequency for this data

Yearly

For how long do you plan to keep this dataset stored on Filecoin

1.5 to 2 years

In which geographies do you plan on making storage deals

Greater China, Asia other than Greater China, North America

How will you be distributing your data to storage providers

HTTP or FTP server, Shipping hard drives, Lotus built-in data transfer

How do you plan to choose storage providers

Slack, Big Data Exchange, Partners

If you answered "Others" in the previous question, what is the tool or platform you plan to use

No response

If you already have a list of storage providers to work with, fill out their names and provider IDs below

f01211025 HK
f01086553 CN
f01125783 HK
f01347695 KR
f0131464 US

How do you plan to make deals to your storage providers

Boost client, Lotus client, Singularity

If you answered "Others/custom tool" in the previous question, enter the details here

No response

Can you confirm that you will follow the Fil+ guideline

Yes

large-datacap-requests[bot] commented 1 year ago

Thanks for your request!

Heads up, you’re requesting more than the typical weekly onboarding rate of DataCap!

large-datacap-requests[bot] commented 1 year ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

Sunnyiscoming commented 1 year ago

Please introduce your organization and project.
Have you prepared enough token for sector pledge?
It has been stored before. Why are you choosing to store it again? https://github.com/filecoin-project/filecoin-plus-large-datasets/issues/1805
Best practice for storing large datasets includes ideally, storing it in 3 or more regions, with 4 or more storage provider operators or owners.You should list Miner ID, Business Entity, Location of sps you will cooperate with.
Could you send an email to filplus-app-review@fil.org with your official domain in order to confirm your identity? Email name should includes the issue id #2147.
Per the https://github.com/filecoin-project/notary-governance/issues/922 for Open, Public Dataset applicants, please complete the following Fil+ registration form to identify yourself as the applicant and also please add the contact information of the SP entities you are working with to store copies of the data.

This information will be reviewed by Fil+ Governance team to confirm validity and then the application will be triggered for notary review. Let us know if you have any questions.

Jaida0 commented 1 year ago

@Sunnyiscoming I'm an individual dp. This application is for the data I have downloaded. Before cooperation, I will ensure it with SPs. I didn't know this in advance. After you mentioned, I saw that he didn't begin storing. I have listed sps in the application and Fil+ registration form Email has been sent and the form has been filled. Please check it. Thank you!

herrehesse commented 1 year ago

@Jaida0 Hello friend, welcome to FIL+.

I can see you are asking for a huge amount of datacap as a first-try, can you explain to me why you do not start with a smaller amount?

Thank you.

Jaida0 commented 1 year ago

@herrehesse Hello friend, the amount is just based on the size of dataset. Before applied for datacap, I learned about IPFS, then I began to know about Filecoin. I think Fil+ can help data be stored safely and distributedly, that's why I'm very interested in joining in the development process of Fil+. And I am confident that I can do well with this application and storage.

herrehesse commented 1 year ago

We're thrilled to hear about your interest, and we're excited to have you on board. While we're enthusiastic about your confidence, I'd strongly recommend starting with a smaller allocation. This approach will help you build a reputation and gain experience within the Filecoin and datacap community.

Before considering an allocation as substantial as 6 PiB of datacap, it's important for us to establish a sense of trust within the community. Could you perhaps begin with a 10TiB allocation? This would provide an opportunity for you to showcase your skills, share your minerID list, and provide insight into the regions and businesses you're involved with.

Not supportive of a 6PiB grant on an account with no reputation.

ghost commented 1 year ago

SP entities confirmed via registration form: f01211025 HK f01086553 CN f01125783 HK f01347695 KR f0131464 US

Sunnyiscoming commented 1 year ago

Datacap Request Trigger

Total DataCap requested

6PiB

Expected weekly DataCap usage rate

800TiB

Client address

f16grdbr73dccc66dxhtrr2shqoarpupowpteo2yi

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Multisig Notary address

f02049625

Client address

f16grdbr73dccc66dxhtrr2shqoarpupowpteo2yi

DataCap allocation requested

307.19TiB

Id

e6432638-122c-4208-80f5-cf313ef593e7

cryptowhizzard commented 1 year ago

f01211025 is a CC miner, no IP adres set on chain, not reachable. f01086553 is a CC miner, no IP adres set on chain, not reachable. f01125783 is a CC miner, no IP adres set on chain, not reachable. f01347695 is a CC miner, no IP adres set on chain, not reachable. f0131464 is a CC miner, no IP adres set on chain, not reachable.

None of these miners are useable for FIL+ and none of these comply with the rules & guidelines.

ghost commented 1 year ago

If true - please remove these miner IDs @Jaida0 and find other SPs

Jaida0 commented 1 year ago

@cryptowhizzard First, I haven't been granted datacap. These SPs didn't start sealing yet, so that they are still CC miners. Second, there's no rules that clients can not cooperate with CC miners. Third, these SPs didn't start sealing yet, so there's no need to public their ip. I will inform them when they begin sealing.

You can look forward to our report after they begin storing data.

@filplus-govteam I will find more SPs if I found any SPs are not good ideas to work with. Thank you.

cryptowhizzard commented 1 year ago

Hi @Jaida0

If your miners are not reachable:

We cannot verify their location.
We cannot verify they are able to serve retrievals and fit in the FIL+ program.

Please contact your SP's and fix.

Jaida0 commented 1 year ago

@cryptowhizzard OK. I have contacted with SPs and they will check their settings. To make the process go better, I went ahead and reached out to some other SPs and would add them into our plan. f02123612 | Bruce | US f01975299 | Emily | HK f01939387 | Alice | CN

Welcome notaries to support my application!

cryptowhizzard commented 1 year ago

Hi,

f02123612 is running on Misaka Network, Inc in Denver. This is a cloud VPN provider. Where is this miner located? It is not in the US. Secondly, it is not providing retrieval.

f01975299 is seen in abuse with application #1558 and not proving retrieval. f01939387 is seen in abuse with application #1558 and not proving retrieval.

Please clarify?

Jaida0 commented 1 year ago

They have told me that they locate in US. And they agree to support retrieval when they begin sealing.

We are willing to follow the rules of the community. If community think we should change SPs even if they support retrieval, we will look for new SP.

Jaida0 commented 1 year ago

Can notaries help sign my application? We will be serious and follow the rules.

mikezli commented 1 year ago

Thank you for contacting me, the first round, temporary support, I will pay attention to the SP situation of follow-up cooperation!

mikezli commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacea6zf3rnmiqtmjktwytgzjszqcxd65obq2fsvgd6wlg65odw4do74

Address

f16grdbr73dccc66dxhtrr2shqoarpupowpteo2yi

Datacap Allocated

307.19TiB

Signer Address

f1dnb3uz7sylxk6emti3ififcvu3nlufnnsjui6ea

Id

e6432638-122c-4208-80f5-cf313ef593e7

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacea6zf3rnmiqtmjktwytgzjszqcxd65obq2fsvgd6wlg65odw4do74

kevzak commented 1 year ago

@Jaida0 you need 4 SP entities and different locations. Also you will be responsible to ensure your SPs provide retrievals on this dataset.

Please provide a clear list of miner IDs you are partnering with before notary signature here

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

Jaida0 commented 1 year ago

Yes, we continue updating the list. f02114994 | Sichuan ; f01084913 | HongKong ; f01841134 | Guizhou ; f02368321 | Singapore Hope notaries can sign for us. Then there'll be more SPs want to cooperate with us!

@Jaida0 you need 4 SP entities and different locations. Also you will be responsible to ensure your SPs provide retrievals on this dataset.

Please provide a clear list of miner IDs you are partnering with before notary signature here

cryptowhizzard commented 1 year ago

I would like to protest against this change.

All these SP's are tagged in my system. None of them provide graphsync retrievals, secondly I have received garbage data from some over http retrieving files.

@Jaida0 please provide us with a sample CID ( .car ) file of the data you will be storing so we can check your data.

Jaida0 commented 1 year ago

@cryptowhizzard Can you share your system to us and explain why you flag these SPs? What's your standard about SPs? Is there any conflict of interest between you and these SPs?

None of them provide graphsync retrievals

Is there any rules that say SPs don't support graphsync retrieval are breaking the rules? The SPs we chose has been checked to support http retrieval. Is http retrieval not a standard of community?

cryptowhizzard commented 1 year ago

f02114994 | Sichuan ; f01084913 | HongKong ; f01841134 | Guizhou ; f02368321 | Singapore

f02368321 https://github.com/filecoin-project/filecoin-plus-large-datasets/issues/2057

Your reputational SP's did get their datacap removed.

I suggest you look at others.

Jaida0 commented 1 year ago

I do not see any problem from that application on these SPs. @galen-mcandrew @kevzak @raghavrmadya Can you help provide some standard or advice about choosing SPs? I don't know what's wrong with the sp I'm looking for.

@cryptowhizzard Also, can you help answer these questions? I really puzzled. Thank you.

@cryptowhizzard Can you share your system to us and explain why you flag these SPs? What's your standard about SPs? Is there any conflict of interest between you and these SPs?

None of them provide graphsync retrievals

Is there any rules that say SPs don't support graphsync retrieval are breaking the rules? The SPs we chose has been checked to support http retrieval. Is http retrieval not a standard of community?

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

Jaida0 commented 1 year ago

Yes

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

Jaida0 commented 1 year ago

Yes should be open.

MEIYAN666 commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaceczhell5a6y67vcxnxw6enoqngfcpysbmz5asdyhcetvpvgd2zgqi

Address

f16grdbr73dccc66dxhtrr2shqoarpupowpteo2yi

Datacap Allocated

307.19TiB

Signer Address

f1bwugfihrmn3iyunzyxst5nttql3dge4khwmurtq

Id

e6432638-122c-4208-80f5-cf313ef593e7

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceczhell5a6y67vcxnxw6enoqngfcpysbmz5asdyhcetvpvgd2zgqi

herrehesse commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report[^1]

No active deals found for this client.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

Jaida0 commented 1 year ago

Yes! We're in process.

Sunnyiscoming commented 12 months ago

Sps in the table do not participated in. Sps outside the form participated in. @Jaida0 Can you explain about that?

ghost commented 11 months ago

Closing as SPs in upfront list do not match those taking deals, 922 violation

Jaida0 commented 11 months ago

@Sunnyiscoming @Filplus-govteam Hello, I have updated the list of sps and resumitted the Fil+ registration form. Although we made plans in advance, the actual cooperation process is likely to face changes. I'll keep it updated. Can you open my application?

ghost commented 11 months ago

SP 1 f02208475 redstone Guangzhou, Guangdong,CN no Jack SP 2 f02211576 flyship Dongguan,Guangdong no York SP 3 f081976 wantou Xian, Shanxi no Earl SP 4 f02834511 webgroup HongKong no Don SP 5 f01844043 abel Ashburn, US no abel

@Jaida0 thanks for the list. We're finding it's easier for you to collect SP entity and location proof from your SPs and send over to filplus.govteam@gmail.com and we'll confirm distribution

Jaida0 commented 11 months ago

@Filplus-govteam Are you joking? I have submitted Fil+ registration form twice and provided list of SPs a million times. Now you let me send all of information to another email. Are you just looking for an excuse to stop applicants? I don't see you asking for the same thing on other's application. @cryptowhizzard @xinaxu Can they be treated the same way? Can @jbenet be treated the same way?

ghost commented 11 months ago

@Jaida0 each process utilized has to iterate to evolve with users. What we've learned in this process, is that sending a list of names and emails does not actually verify anything. We emailed SP addresses and asked for SPs to send me proof of entity and location. We did not receive anything. This is not about other users, this is about your data.

From a quick search, SP IDs are under VPN vender services, so there is no way to validate actual locations.

And now, yes we changed our process to make it easier for you to provide information. You can now control collecting information from SPs and send to the email above.

filecoin-project / filecoin-plus-large-datasets