filecoin-project / filecoin-plus-large-datasets

Hub for client applications for DataCap at a large scale
110 stars 62 forks source link

[DataCap Application] <Outercore> - <Web3PH> #2102

Closed corinne-antonia closed 1 year ago

corinne-antonia commented 1 year ago

Data Owner Name

Outercore - Network Growth - Engineering

What is your role related to the dataset

Data onramp entity that provides data onboarding services to multiple clients

Data Owner Country/Region

United States

Data Owner Industry

Web3 / Crypto

Website

https://fw.services

Social Media

https://twitter.com/OutercoreEng
https://twitter.com/Estuary_Tech

Total amount of DataCap being requested

1PiB

Expected size of single dataset (one copy)

166TiB

Number of replicas to store

6

Weekly allocation of DataCap requested

10TiB

On-chain address for first allocation

f1ys5m675da4ao4vwefbewtyafuw3zhgemya6jnky

Data Type of Application

Public, Open Dataset (Research/Non-Profit)

Custom multisig

Identifier

efil

Share a brief history of your project and organization

Root website: https://fw.services
Project website: https://delta.store

We are the Engineering team at Protocol Labs ➝ Outercore ➝ Engineering/Network Growth. Our goal is develop tooling for entire Filecoin Ecosystem to use to onboard data to the Filecoin Network.

This is the fourth application for our tool Delta. The first one is located here. 
https://github.com/filecoin-project/filecoin-plus-large-datasets/issues/1854
https://github.com/filecoin-project/filecoin-plus-large-datasets/issues/160. The second one is here: https://github.com/filecoin-project/filecoin-plus-large-datasets/issues/1602

#### Our mission

We want an internet where resources are owned & shared by everyone. Everyone shares consensus over a distributed ledger. Data storage is verifiably stored for the end-user, and more fault tolerant. There is no central authority of control and no single point of failure. And the security promise only improves as the network continues to grow. Just imagine all of the certifications that can be automatically generated for end users by a new network like this. This is how you achieve storage as a human right for anyone in the world.

We're going to prove that we can get here by building the tools that can support onboarding 10 PiBs a day of data to the Filecoin Network. But we're also not going to forget about retrievability and helping the Filecoin Network actually become usable.

#### Delta

Our solution to archival and cold storage use cases.

Use ∆ Delta to upload all of your useful public data to Filecoin storage providers. Delta is a straight-forward Filecoin storage deal making tool that manages deals, and does not do anything else. It is purely for the function of helping Storage Providers fill capacity either through online or offline methods. It is written in Go and designed to be paired well with bare-metal infrastructure.

Is this project associated with other projects/ecosystem stakeholders?

Yes

If answered yes, what are the other projects/ecosystem stakeholders

Protocol Labs -> Outercore -> Network Growth

Describe the data being stored onto Filecoin

We are working directly with Web3PH (https://web3philippines.org/). Web3PH is the first and primary Web3 tech and non-tech builder community in the Phillapines. They have given us permission to upload their data to Filecoin on their behalf using our tooling, Delta.

- Images, Videos (public)
- Documentations  (public)

Where was the data currently stored in this dataset sourced from

Other

If you answered "Other" in the previous question, enter the details here

S3, Moralis, IPFS gateways

How do you plan to prepare the dataset

others/custom tool

If you answered "other/custom tool" in the previous question, enter the details here

Our new tool Delta

Learn more at https://delta.store

Please share a sample of the data

Digital asset images/videos from here:

https://opensea.io/collection/web3phl-community-pass
https://opensea.io/collection/web3phl-top-members 
https://thirdweb.com/polygon/0x2Bc1672B0ff7761988046b83bfc0dadE004ccd00

Confirm that this is a public dataset that can be retrieved by anyone on the Network

If you chose not to confirm, what was the reason

Delta is not focused on retrieval, its focused on storage onboarding. We are going to help all SPs onboard all of their data.

Estuary is focused on retrieval.

What is the expected retrieval frequency for this data

Yearly

For how long do you plan to keep this dataset stored on Filecoin

More than 3 years

In which geographies do you plan on making storage deals

North America

How will you be distributing your data to storage providers

Others

How do you plan to choose storage providers

Others

If you answered "Others" in the previous question, what is the tool or platform you plan to use

We will be developing our tool Delta

Learn more at https://delta.store

If you already have a list of storage providers to work with, fill out their names and provider IDs below

We have developed our own list at https://data.storage.market but we will work with everyone/anyone. We actively work with storage providers and we have a storage provider on our team.

93 of them that we plan to start with, obviously we'll work with more.

➝ https://data.storage.market/api/providers/f0840770
➝ https://data.storage.market/api/providers/f01624021
➝ https://data.storage.market/api/providers/f01806491
➝ https://data.storage.market/api/providers/f08399
➝ https://data.storage.market/api/providers/f01790264
➝ https://data.storage.market/api/providers/f0875769
➝ https://data.storage.market/api/providers/f01035680
➝ https://data.storage.market/api/providers/f033356
➝ https://data.storage.market/api/providers/f01683871
➝ https://data.storage.market/api/providers/f03488
➝ https://data.storage.market/api/providers/f030379
➝ https://data.storage.market/api/providers/f01466075
➝ https://data.storage.market/api/providers/f02301
➝ https://data.storage.market/api/providers/f010479
➝ https://data.storage.market/api/providers/f010088
➝ https://data.storage.market/api/providers/f0773157
➝ https://data.storage.market/api/providers/f0717969
➝ https://data.storage.market/api/providers/f0461791
➝ https://data.storage.market/api/providers/f01746964
➝ https://data.storage.market/api/providers/f01059489
➝ https://data.storage.market/api/providers/f023467
➝ https://data.storage.market/api/providers/f01392893
➝ https://data.storage.market/api/providers/f01736668
➝ https://data.storage.market/api/providers/f022352
➝ https://data.storage.market/api/providers/f01199430
➝ https://data.storage.market/api/providers/f09848
➝ https://data.storage.market/api/providers/f01222595
➝ https://data.storage.market/api/providers/f01794610
➝ https://data.storage.market/api/providers/f01443744
➝ https://data.storage.market/api/providers/f01199442
➝ https://data.storage.market/api/providers/f02401
➝ https://data.storage.market/api/providers/f01402814
➝ https://data.storage.market/api/providers/f0104671
➝ https://data.storage.market/api/providers/f01207045
➝ https://data.storage.market/api/providers/f0406703
➝ https://data.storage.market/api/providers/f01175097
➝ https://data.storage.market/api/providers/f010446
➝ https://data.storage.market/api/providers/f0724219
➝ https://data.storage.market/api/providers/f022142
➝ https://data.storage.market/api/providers/f058369
➝ https://data.storage.market/api/providers/f01201327
➝ https://data.storage.market/api/providers/f0406322
➝ https://data.storage.market/api/providers/f01278
➝ https://data.storage.market/api/providers/f0187709
➝ https://data.storage.market/api/providers/f01045784
➝ https://data.storage.market/api/providers/f0706693
➝ https://data.storage.market/api/providers/f024184
➝ https://data.storage.market/api/providers/f01385207
➝ https://data.storage.market/api/providers/f01652333
➝ https://data.storage.market/api/providers/f01319368
➝ https://data.storage.market/api/providers/f0214334
➝ https://data.storage.market/api/providers/f082635
➝ https://data.storage.market/api/providers/f0836160
➝ https://data.storage.market/api/providers/f0135078
➝ https://data.storage.market/api/providers/f039940
➝ https://data.storage.market/api/providers/f0408717
➝ https://data.storage.market/api/providers/f01345523
➝ https://data.storage.market/api/providers/f01108096
➝ https://data.storage.market/api/providers/f01611097
➝ https://data.storage.market/api/providers/f01208862
➝ https://data.storage.market/api/providers/f01662356
➝ https://data.storage.market/api/providers/f01423116
➝ https://data.storage.market/api/providers/f017665
➝ https://data.storage.market/api/providers/f01666984
➝ https://data.storage.market/api/providers/f0440429
➝ https://data.storage.market/api/providers/f01028552
➝ https://data.storage.market/api/providers/f0707721
➝ https://data.storage.market/api/providers/f099608
➝ https://data.storage.market/api/providers/f0142637
➝ https://data.storage.market/api/providers/f01127678
➝ https://data.storage.market/api/providers/f0501283
➝ https://data.storage.market/api/providers/f01133080
➝ https://data.storage.market/api/providers/f0127896
➝ https://data.storage.market/api/providers/f010617
➝ https://data.storage.market/api/providers/f097777
➝ https://data.storage.market/api/providers/f01367109
➝ https://data.storage.market/api/providers/f01225882
➝ https://data.storage.market/api/providers/f023971
➝ https://data.storage.market/api/providers/f0466405
➝ https://data.storage.market/api/providers/f01310564
➝ https://data.storage.market/api/providers/f02620
➝ https://data.storage.market/api/providers/f02576
➝ https://data.storage.market/api/providers/f01764587
➝ https://data.storage.market/api/providers/f019551
➝ https://data.storage.market/api/providers/f034258
➝ https://data.storage.market/api/providers/f0240185
➝ https://data.storage.market/api/providers/f01619524
➝ https://data.storage.market/api/providers/f01096124
➝ https://data.storage.market/api/providers/f01163272
➝ https://data.storage.market/api/providers/f01337533
➝ https://data.storage.market/api/providers/f01091851
➝ https://data.storage.market/api/providers/f01784458
➝ https://data.storage.market/api/providers/f08403

How do you plan to make deals to your storage providers

Others/custom tool

If you answered "Others/custom tool" in the previous question, enter the details here

Our new tool Delta, which we plan to release to the whole ecosystem.

Learn more at https://delta.store

Can you confirm that you will follow the Fil+ guideline

Yes

large-datacap-requests[bot] commented 1 year ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

dannyob commented 1 year ago

Hey folks, I don't see this request in my Fil+ registry interface. Is there something the team could do to make it visible? @galen-mcandrew @Kevin-FF-USA @raghavrmadya

Sunnyiscoming commented 1 year ago

@kevzak

kevzak commented 1 year ago

Hey folks, I don't see this request in my Fil+ registry interface. Is there something the team could do to make it visible? @galen-mcandrew @Kevin-FF-USA @raghavrmadya

@dannyob this application was not triggered yet, this is why you cannot see it.

kevzak commented 1 year ago

Data Preparation clear. List of Delta onboarding tool vetted SPs being used clear. @Sunnyiscoming please trigger this application.

large-datacap-requests[bot] commented 1 year ago

Deleting comment

@kevzak hasn't the permissions to post this comment.

Please, contact the assignee of this issue.

Sunnyiscoming commented 1 year ago

Datacap Request Trigger

Total DataCap requested

1PiB

Expected weekly DataCap usage rate

10TiB

Client address

f1ys5m675da4ao4vwefbewtyafuw3zhgemya6jnky

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Multisig Notary address

f01940930

Client address

f1ys5m675da4ao4vwefbewtyafuw3zhgemya6jnky

DataCap allocation requested

5TiB

Id

8300b570-d878-4d5e-aee1-d24f37f643e7

cryptowhizzard commented 1 year ago

@corinne-antonia requested that I endorse both outercore applications. Given their affiliation with PL & Estuary, their reputation is exceptionally strong. Observing their client and data onboarding strategies, it becomes evident that they are an ideal candidate for Filecoin+ support.

They have our complete backing. We eagerly anticipate receiving their CID & Retrieval reports.

cryptowhizzard commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacecso2kv3d3sicwn2xesyqxu22lnairsy3l6gcvbzoj44jfvfatzfu

Address

f1ys5m675da4ao4vwefbewtyafuw3zhgemya6jnky

Datacap Allocated

5.00TiB

Signer Address

f1krmypm4uoxxf3g7okrwtrahlmpcph3y7rbqqgfa

Id

8300b570-d878-4d5e-aee1-d24f37f643e7

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacecso2kv3d3sicwn2xesyqxu22lnairsy3l6gcvbzoj44jfvfatzfu

kernelogic commented 1 year ago

In support for outercore-dev

kernelogic commented 1 year ago
image

Can't sign

steven004 commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report[^1]

No application info found for this issue on https://filplus.d.interplanetary.one/clients.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

steven004 commented 1 year ago

Just notice I am not a signer of f01940930. Sorry I can not propose or approve until I get in.

kevzak commented 1 year ago

try it now @kernelogic can you see?

jamerduhgamer commented 1 year ago

Outercore team is developing a data onboarding application tool called Delta which is currently already sending storage deals to a large number of SPs and is willing to send to any SP that signs up. Willing to support this application.

jamerduhgamer commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacedgpmsbnsiwpxrmjuoincc6gbi7g653crcmaujx6nlihoehvmj3vk

Address

f1ys5m675da4ao4vwefbewtyafuw3zhgemya6jnky

Datacap Allocated

5.00TiB

Signer Address

f1ypuqpi4xn5q7zi5at3rmdltosozifhqmrt66vhq

Id

8300b570-d878-4d5e-aee1-d24f37f643e7

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedgpmsbnsiwpxrmjuoincc6gbi7g653crcmaujx6nlihoehvmj3vk

jamerduhgamer commented 1 year ago

As a heads up for @kernelogic, @steven004 I had to swap notary IDs from my round 4 to round 3. Not sure why though since this LDN was created 2 weeks ago which is well into the round 4 period.

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 14 days, so for now it is being closed. Please feel free to contact the Fil+ Gov team to re-open the application if it is still being processed. Thank you!