[DataCap Application] NREL National Solar Radiation Database

FroghubMan commented 1 year ago

Data Owner Name

NREL

What is your role related to the dataset

Data Preparer

Data Owner Country/Region

United States

Data Owner Industry

Environment

Website

https://nsrdb.nrel.gov/

Social Media

https://registry.opendata.aws/nrel-pds-nsrdb/
https://nsrdb.nrel.gov/

Total amount of DataCap being requested

8PiB

Expected size of single dataset (one copy)

800

Number of replicas to store

10

Weekly allocation of DataCap requested

1PiB

On-chain address for first allocation

f16eoaknzqxdw3rgqcv7kx5xs65my2rub6yoliswa

Data Type of Application

Public, Open Dataset (Research/Non-Profit)

Custom multisig

[ ] Use Custom Multisig

Identifier

No response

Share a brief history of your project and organization

FrogHub files is a platform that will provide users with free storage, allowing users to enjoy free storage at the same time. Facilitating the Filecoin network to store more and more valuable public data sets.

About the FrogHub data set

FrogHub files is a user-oriented cloud storage platform. Any user with data storage needs can use the platform tools free of charge.

The Filecoin Network provides reliable, secure, and affordable decentralized storage, and FrogHub files wants to make it easy for users to store their data in the filecoin network.

Attracting traditional cloud and object-based storage users to the Filecoin network and benefiting from it is a challenge. FrogHub, as developers in the Felicoin ecosystem, needs to face this challenge together. We've been building useful tools to make it easier for users to store data on the Filecoin network.

Froghub files integrates a set of tools, including storage bucket, custom gateway, data encryption service KMS, static web deployment (under development), open API, to provide storage services compatible with cloud storage and object storage, providing better user experience and attracting more users.
FrogHub has always defined itself as a tool developer and infrastructure builder in the Filecoin ecosystem. In 2019, we started to focus on technical solutions and development based on the IPFS protocol and Filecoin network. We have been working hard to become a qualified builder in the filecoin ecosystem.

Our team is a very pure development team, more than 90% of which are developers, more than half of whom have more than 5 years of development experience in communication, Internet, blockchain and other industries. We hope that we can gain users' recognition by exporting useful tools and platforms.

In order to contribute to the filecoin community, we have developed the open source sector repair tool Filecoin-Sealer-Recover and the nft free authoring platform NFT-Creator.
In addition, we plan to provide a sector browser for the community in 2023 and build the liquidity pledge platform STFIL on FVM.

See the links below for details.
- github: https://github.com/orgs/froghub-io
- twitter: https://twitter.com/froghub_io

Is this project associated with other projects/ecosystem stakeholders?

No

If answered yes, what are the other projects/ecosystem stakeholders

No response

Describe the data being stored onto Filecoin

"Released to the public as part of the Department of Energy's Open Energy Data Initiative, the National Solar Radiation Database (NSRDB) is a serially complete collection of hourly and half-hourly values of the three most common measurements of solar radiation – global horizontal, direct normal, and diffuse horizontal irradiance — and meteorological data. These data have been collected at a sufficient number of locations and temporal and spatial scales to accurately represent regional solar radiation climates."

Source: https://registry.opendata.aws/nrel-pds-nsrdb/

Where was the data currently stored in this dataset sourced from

AWS Cloud

If you answered "Other" in the previous question, enter the details here

No response

If you are a data preparer. What is your location (City and Country)

No response

If you are a data preparer, how will the data be prepared? Please include tooling used and technical details?

No response

If you are not preparing the data, who will prepare the data? (Provide name and business)

No response

Has this dataset been stored on the Filecoin network before? If so, please explain and make the case why you would like to store this dataset again to the network. Provide details on preparation and/or SP distribution.

I saw that although some people have applied for the storage of this data, they have not implemented it. And the related application is closed, so I want to store it.

Please share a sample of the data

s3://nrel-pds-nsrdb/ (420.8 TiB)
s3://nrel-pds-nsrdb/v3/(47.5 TiB)
s3://nrel-pds-nsrdb/v3/tmy/(4.0 TiB)
s3://nrel-pds-nsrdb/v3/tdy/(4.0 TiB)
s3://nrel-pds-nsrdb/v3/tgy/(4.0 TiB)
s3://nrel-pds-nsrdb/v3/puerto_rico/(114.6 GiB)
s3://nrel-pds-nsrdb/conus/(48.2 TiB)
s3://nrel-pds-nsrdb/full_disc/(81.8 TiB)
s3://nrel-pds-nsrdb/meteosat/(16.1 TiB)
s3://nrel-pds-nsrdb/himawari/(189.1 TiB)
s3://nrel-pds-nsrdb/india/(515.3 GiB)

Confirm that this is a public dataset that can be retrieved by anyone on the Network

[X] I confirm

If you chose not to confirm, what was the reason

No response

What is the expected retrieval frequency for this data

Yearly

For how long do you plan to keep this dataset stored on Filecoin

2 to 3 years

In which geographies do you plan on making storage deals

Greater China, Asia other than Greater China, North America, Europe

How will you be distributing your data to storage providers

Cloud storage (i.e. S3), Shipping hard drives

How do you plan to choose storage providers

Slack, Big Data Exchange, Partners

If you answered "Others" in the previous question, what is the tool or platform you plan to use

No response

If you already have a list of storage providers to work with, fill out their names and provider IDs below

No response

How do you plan to make deals to your storage providers

No response

If you answered "Others/custom tool" in the previous question, enter the details here

No response

Can you confirm that you will follow the Fil+ guideline

Yes

large-datacap-requests[bot] commented 1 year ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

Sunnyiscoming commented 1 year ago

Has this dataset been stored on the Filecoin network before? If so, please explain and make the case why you would like to store this dataset again to the network. Provide details on preparation and/or SP distribution. I saw that although some people have applied for the storage of this data, they have not implemented it. And the related application is closed, so I want to store it.

https://github.com/filecoin-project/filecoin-plus-large-datasets/issues/1995 What's the relationship between two applications? https://github.com/filecoin-project/filecoin-plus-large-datasets/issues/1104 This dataset has been stored before.

Have you prepared enough token for sector pledge?
Are you a data preparer? What is your previous experience as a data-preparer?
If you are not preparing the data, who will prepare the data? (Name and Business)
Best practice for storing large datasets includes ideally, storing it in 3 or more regions, with 4 or more storage provider operators or owners.You should list Miner ID, Business Entity, Location of sps you will cooperate with.
Could you send an email to filplus-app-review@fil.org with your official domain in order to confirm your identity? Email name should includes the issue id #2117.

FroghubMan commented 1 year ago

We have nothing to do with #1995. (I just found out that the names are somewhat similar.)
1104 stored data. I just discovered this too, so can I continue to store it?
We are data preparer, and have experience and confidence to complete. Colleagues comply with FIL+ rules.
I have sent a confirmation email.

FroghubMan commented 1 year ago

@Sunnyiscoming I found some valuable public datasets from closed applications that had been passed but not actually implemented. The SP I'm working with has strong storage needs, so I want to get started quickly on storing this data.

kevzak commented 1 year ago

Hi @FroghubMan - the one thing missing here is information about SP entities you are working with. Can you map that out publicly here?

FroghubMan commented 1 year ago

Hi @kevzak ，Honestly, it's hard to get some SPs to expose their entities for some reason (I could totally fake it, but it's obviously not necessary.). Hope you understand. But I can fully guarantee the authenticity and effectiveness of these nodes, and they will store data fully satisfy the FIL+ rules.

FroghubMan commented 1 year ago

MinerID	City	Continent
f01811024	HongKong	CN
f0827006	Tokyo	Japan
f02227726	HongKong	CN
f02195153	Tokyo	Japan
f02098006	Krabi	Thailand
f02223012	Hebei	CN
f02235154	HongKong	CN
f02085722	Jiangmen	CN

ghost commented 1 year ago

Left a comment on 2119

FroghubMan commented 1 year ago

Hi @Filplus-govteam @Sunnyiscoming ,I have submitted the information in #2119, please review it for me. Thank you very much.

Sunnyiscoming commented 1 year ago

Datacap Request Trigger

Total DataCap requested

8PiB

Expected weekly DataCap usage rate

1PiB

Client address

f16eoaknzqxdw3rgqcv7kx5xs65my2rub6yoliswa

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Multisig Notary address

f02049625

Client address

f16eoaknzqxdw3rgqcv7kx5xs65my2rub6yoliswa

DataCap allocation requested

409.59TiB

Id

d0b265f9-d692-48b7-b3ff-4ba776f6dbf5

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 14 days, so for now it is being closed. Please feel free to contact the Fil+ Gov team to re-open the application if it is still being processed. Thank you!

FroghubMan commented 1 year ago

We are preparing the data.

FroghubMan commented 1 year ago

Hi @Sunnyiscoming ,We have been preparing data, only to find that it was closed. We have now completed the prep function, can you reopen this application for me?

1ane-1 commented 1 year ago

How many copies will you store? And show me your identity email address. Thanks.

FroghubMan commented 1 year ago

Screenshot_20230831_100606_Gmail

Chuangshi1 commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacebvlehaycc44xzoeur6nmwh775fb5yhybp6vw42laryo4dphs5xmy

Address

f16eoaknzqxdw3rgqcv7kx5xs65my2rub6yoliswa

Datacap Allocated

409.59TiB

Signer Address

f1mdk7s2vntzm6hu35yuo6vjubtrpfnb2awhgvrri

Id

d0b265f9-d692-48b7-b3ff-4ba776f6dbf5

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebvlehaycc44xzoeur6nmwh775fb5yhybp6vw42laryo4dphs5xmy

AlanGreaterheat commented 1 year ago

Client reached out to me via slack as the first round of a public dataset and is willing to support it, looking forward to seeing the first round of sealing reports.

AlanGreaterheat commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacedm4nlkeuzv6ch3ysi7wurt7oq6r2xotdsjxthwsvk7ehty27ywlm

Address

f16eoaknzqxdw3rgqcv7kx5xs65my2rub6yoliswa

Datacap Allocated

409.59TiB

Signer Address

f1pnmzlxj7cfeo2v6oj5nco46hkg2l46wj7o4xxui

Id

d0b265f9-d692-48b7-b3ff-4ba776f6dbf5

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedm4nlkeuzv6ch3ysi7wurt7oq6r2xotdsjxthwsvk7ehty27ywlm

FroghubMan commented 1 year ago

thank you!

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 2

Multisig Notary address

f02049625

Client address

f16eoaknzqxdw3rgqcv7kx5xs65my2rub6yoliswa

DataCap allocation requested

512TiB

Id

6530207c-de9b-4b67-8cbf-02682adbbf30

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f02049625

Client address

f16eoaknzqxdw3rgqcv7kx5xs65my2rub6yoliswa

Rule to calculate the allocation request amount

100% weekly > 0.5PiB, requesting 0.5PiB

DataCap allocation requested

512TiB

Total DataCap granted for client so far

409.59TiB

Datacap to be granted to reach the total amount requested by the client (8PiB)

7.60PiB

Stats

Number of deals	Number of storage providers	Previous DC Allocated	Top provider	Remaining DC
0	0	409.59TiB	NaN	409.59TiB

FroghubMan commented 1 year ago

We're preparing dataset.

spaceT9 commented 1 year ago

checker:manualTrigger

FroghubMan commented 1 year ago

We're preparing dataset.

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

FroghubMan commented 1 year ago

We're preparing data

FroghubMan commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

⚠️ All retrieval success ratios are below 1%.

Overall Graphsync retrieval success rate:
Overall HTTP retrieval success rate:
Overall Bitswap retrieval success rate:

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

⚠️ 80.18% of deals are for data replicated across less than 3 storage providers.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

FroghubMan commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

⚠️ All retrieval success ratios are below 1%.

Overall Graphsync retrieval success rate:
Overall HTTP retrieval success rate:
Overall Bitswap retrieval success rate:

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

FroghubMan commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

⚠️ All retrieval success ratios are below 1%.

Overall Graphsync retrieval success rate:
Overall HTTP retrieval success rate:
Overall Bitswap retrieval success rate:

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

FroghubMan commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

⚠️ All retrieval success ratios are below 1%.

Overall Graphsync retrieval success rate:
Overall HTTP retrieval success rate:
Overall Bitswap retrieval success rate:

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

FroghubMan commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

⚠️ All retrieval success ratios are below 1%.

Overall Graphsync retrieval success rate:
Overall HTTP retrieval success rate:
Overall Bitswap retrieval success rate:

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

FroghubMan commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

⚠️ All retrieval success ratios are below 1%.

Overall Graphsync retrieval success rate:
Overall HTTP retrieval success rate:
Overall Bitswap retrieval success rate:

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

FroghubMan commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

⚠️ All retrieval success ratios are below 1%.

Overall Graphsync retrieval success rate:
Overall HTTP retrieval success rate:
Overall Bitswap retrieval success rate:

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

FroghubMan commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

⚠️ All retrieval success ratios are below 1%.

Overall Graphsync retrieval success rate:
Overall HTTP retrieval success rate:
Overall Bitswap retrieval success rate:

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

FroghubMan commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

⚠️ All retrieval success ratios are below 1%.

Overall Graphsync retrieval success rate:
Overall HTTP retrieval success rate:
Overall Bitswap retrieval success rate:

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

FroghubMan commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard.

mikezli commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzaceafgmzoqlhp3nkp2qxfd2vp4dsoybxkmbj5zhf2nu7dlp67v2sbyg

Address

f16eoaknzqxdw3rgqcv7kx5xs65my2rub6yoliswa

Datacap Allocated

512.00TiB

Signer Address

f1dnb3uz7sylxk6emti3ififcvu3nlufnnsjui6ea

Id

6530207c-de9b-4b67-8cbf-02682adbbf30

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceafgmzoqlhp3nkp2qxfd2vp4dsoybxkmbj5zhf2nu7dlp67v2sbyg

filecoin-project / filecoin-plus-large-datasets