search

filecoin-project / filecoin-plus-large-datasets

Hub for client applications for DataCap at a large scale
110 stars 62 forks source link

[DataCap Application] NEXRAD Dataset #1062

Closed Megan008 closed 1 year ago

Megan008 commented 1 year ago

Large Dataset Notary Application

To apply for DataCap to onboard your dataset to Filecoin, please fill out the following.

Core Information

Please respond to the questions below by replacing the text saying "Please answer here". Include as much detail as you can in your answer.

Project details

Share a brief history of your project and organization.

I have participated in some projects and hackathon. I have experience on it.

What is the primary source of funding for this project?

Personal income.

What other projects/ecosystem stakeholders is this project associated with?

No.

Use-case details

Describe the data being stored onto Filecoin

Real-time and archival data from the Next Generation Weather Radar (NEXRAD) network.

Where was the data in this dataset sourced from?

https://registry.opendata.aws/noaa-nexrad/

Can you share a sample of the data? A link to a file, an image, a table, etc., are good ways to do this. 

s3://noaa-nexrad-level2/1992/08/24/KMLB/KMLB19920824_224809.gz
s3://noaa-nexrad-level2/2021/01/01/KBBX/KBBX20210101_135953_V06_MDM

Confirm that this is a public dataset that can be retrieved by anyone on the Network (i.e., no specific permissions or access rights are required to view the data).


Yes, it is AWS open dataset.

What is the expected retrieval frequency for this data?

Not very often.

For how long do you plan to keep this dataset stored on Filecoin?

2 years. Then I'll extend the time if it is successful.

DataCap allocation plan

In which geographies (countries, regions) do you plan on making storage deals?

North america; Korea; China.

How will you be distributing your data to storage providers? Is there an offline data transfer process?

75% data will be distributed by offline data transfer. Other data will use online transfer for distributing with storage providers who close to me.

How do you plan on choosing the storage providers with whom you will be making deals? This should include a plan to ensure the data is retrievable in the future both by you and others.

I would let 1 sp who used to cooperate with me for this deal. Now I'm chatting with other sps.

How will you be distributing deals across storage providers?

I have communicated with 4 sp. In first time, I will divide 1/4 data to each sp. If I find out more sp, I will decrease the percentage of deals to them --- for decentralized storage.

Do you have the resources/funding to start making deals as soon as you receive DataCap? What support from the community would help you onboard onto Filecoin?

Yes.
large-datacap-requests[bot] commented 1 year ago

Thanks for your request! :exclamation: We have found some problems in the information provided. We could not find any Web site or social media info in the information provided We could not find any Region in the information provided

Please, take a look at the request and edit the body of the issue providing all the required information.
large-datacap-requests[bot] commented 1 year ago

Thanks for your request!

Heads up, you’re requesting more than the typical weekly onboarding rate of DataCap!
large-datacap-requests[bot] commented 1 year ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

raghavrmadya commented 1 year ago

Please share the SPs you plan to work with

raghavrmadya commented 1 year ago

Please share the SPs you plan to work with

Megan008 commented 1 year ago

Thanks @raghavrmadya , good to have your reply! We previously contacted 7-8 SPs from all over the world who are eager to store data. There are 5 SPs, including f023495 and f0508988, who are willing to participate in the first round of sealing. I am also continuing to contact more SPs on BDE platform in hopes of cooperating.

simonkim0515 commented 1 year ago

Datacap Request Trigger

Total DataCap requested

5PiB

Expected weekly DataCap usage rate

200TiB

Client address

f1xi564ajokw3ocdn2kslwvwhzvvusqyijinvau7i

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Multisig Notary address

f02049625

Client address

f1xi564ajokw3ocdn2kslwvwhzvvusqyijinvau7i

DataCap allocation requested

100TiB

Id

34edeb10-6602-4f7d-af23-33bea47acf0f

cryptowhizzard commented 1 year ago

https://github.com/filecoin-project/filecoin-plus-large-datasets/issues/1013 -> Abuse ( CID sharing )

MEIYAN666 commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

⚠️ CID sharing has been observed. (Top 3)

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the full report.

Megan008 commented 1 year ago

Actually I didn't know that this would happen. The sp I worked with made some mistake in their work and I will ask them to check themselves next time to make sure it doesn't happen next time.

BobbyChoii commented 1 year ago

Thank you for integrating more trustworthy data into the network and please follow fileplus' guidelines. You need to remind SPs not to confuse datasets anymore.

BobbyChoii commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacec7nyhhvkzej6fbynsbl6rcu7jqn4rycghsy245voswll2qc6szua

Address

f1xi564ajokw3ocdn2kslwvwhzvvusqyijinvau7i

Datacap Allocated

100.00TiB

Signer Address

f1irqs2gmctiv3jcdfwuch7oxvf4ixh3k4b2wc24i

Id

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacec7nyhhvkzej6fbynsbl6rcu7jqn4rycghsy245voswll2qc6szua

large-datacap-requests[bot] commented 1 year ago

We have found some problems in the information provided in the Approved Comment. We could not find Id** field in the information provided

Please, take a look at the comment and edit the body of the comment providing all the required information.
large-datacap-requests[bot] commented 1 year ago

Thanks for your request!

Heads up, you’re requesting more than the typical weekly onboarding rate of DataCap!
large-datacap-requests[bot] commented 1 year ago

Thanks for your request!

Heads up, you’re requesting more than the typical weekly onboarding rate of DataCap!
github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

Megan008 commented 1 year ago

OK, I know.

cryptowhizzard commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

⚠️ All retrieval success ratios are below 1%.

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

⚠️ CID sharing has been observed. (Top 3)

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

Megan008 commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

⚠️ CID sharing has been observed. (Top 3)

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

cryptowhizzard commented 1 year ago

Scherm­afbeelding 2023-08-04 om 11 19 08

As per the CID checker , most SP's are involved in Abuse / CID sharing. Those who are not have retrieval disabled.

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 14 days, so for now it is being closed. Please feel free to contact the Fil+ Gov team to re-open the application if it is still being processed. Thank you!

large-datacap-requests[bot] commented 8 months ago

Thanks for your request! :exclamation: We have found some problems in the information provided. We could not find Website \/ Social Media field in the information provided We could not find Total amount of DataCap being requested (between 500 TiB and 5 PiB) field in the information provided We could not find Weekly allocation of DataCap requested (usually between 1-100TiB) field in the information provided We could not find On-chain address for first allocation field in the information provided We could not find Data Type of Application field in the information provided

Please, take a look at the request and edit the body of the issue providing all the required information.