search

filecoin-project / filecoin-plus-large-datasets

Hub for client applications for DataCap at a large scale
110 stars 62 forks source link

[DataCap Application] <Outercore> - <Vulcanic Labs> #2131

Closed corinne-antonia closed 11 months ago

corinne-antonia commented 1 year ago

Data Owner Name

Outercore - Network Growth - Engineering

What is your role related to the dataset

Data onramp entity that provides data onboarding services to multiple clients

Data Owner Country/Region

United States

Data Owner Industry

Web3 / Crypto

Website

https://fw.services

Social Media

https://twitter.com/OutercoreEng
https://twitter.com/Estuary_Tech
https://twitter.com/FilecoinTools

Total amount of DataCap being requested

1PiB

Expected size of single dataset (one copy)

166TiB

Number of replicas to store

6

Weekly allocation of DataCap requested

10TiB

On-chain address for first allocation

f1cvqsewsg7w5y3h7xwdsdd6qgsznx6ueacmparla

Data Type of Application

Public, Open Dataset (Research/Non-Profit)

Custom multisig

Identifier

No response

Share a brief history of your project and organization

Root website: https://fw.services
Project website: https://delta.store

We are the Engineering team at Protocol Labs ➝ Outercore ➝ Engineering/Network Growth. Our goal is develop tooling for entire Filecoin Ecosystem to use to onboard data to the Filecoin Network.

This is the fourth application for our tool Delta. The first one is located here. 
https://github.com/filecoin-project/filecoin-plus-large-datasets/issues/1854
https://github.com/filecoin-project/filecoin-plus-large-datasets/issues/160. The second one is here: https://github.com/filecoin-project/filecoin-plus-large-datasets/issues/1602

#### Our mission

We want an internet where resources are owned & shared by everyone. Everyone shares consensus over a distributed ledger. Data storage is verifiably stored for the end-user, and more fault tolerant. There is no central authority of control and no single point of failure. And the security promise only improves as the network continues to grow. Just imagine all of the certifications that can be automatically generated for end users by a new network like this. This is how you achieve storage as a human right for anyone in the world.

We're going to prove that we can get here by building the tools that can support onboarding 10 PiBs a day of data to the Filecoin Network. But we're also not going to forget about retrievability and helping the Filecoin Network actually become usable.

#### Delta

Our solution to archival and cold storage use cases.

Use ∆ Delta to upload all of your useful public data to Filecoin storage providers. Delta is a straight-forward Filecoin storage deal making tool that manages deals, and does not do anything else. It is purely for the function of helping Storage Providers fill capacity either through online or offline methods. It is written in Go and designed to be paired well with bare-metal infrastructure.

Is this project associated with other projects/ecosystem stakeholders?

Yes

If answered yes, what are the other projects/ecosystem stakeholders

Protocol Labs -> Outercore -> Network Growth

Describe the data being stored onto Filecoin

We are working directly with Vulvanic Labs (https://www.vulcaniclabs.com/, https://twitter.com/VulcanicLabs?s=20), a software development firm based in Singapore. They have given us permission to upload their data to Filecoin on their behalf using our tooling, Delta. 

This data includes:
- Real Estate Workers Metadata
- Educational images and videos
- NFT/Images
- Movie films and movie demonstrations
- Employement metadata (encrypted)

Where was the data currently stored in this dataset sourced from

AWS Cloud

If you answered "Other" in the previous question, enter the details here

S3

If you are a data preparer. What is your location (City and Country)

(Dallas) United States

If you are a data preparer, how will the data be prepared? Please include tooling used and technical details?

Our new tool Delta

Learn more at https://delta.store

If you are not preparing the data, who will prepare the data? (Provide name and business)

We are preparing the data

Has this dataset been stored on the Filecoin network before? If so, please explain and make the case why you would like to store this dataset again to the network. Provide details on preparation and/or SP distribution.

No

Please share a sample of the data

[{"id":8925,"uid":"ce8cd91f-b7ff-47bb-b4b0-080823756fc6","account_number":"4080342552","iban":"GB62YWJR86357767614013","bank_name":"SANTANDER UK PLC","routing_number":"250401941","swift_bic":"BSABGB2LXXX"},{"id":1464,"uid":"2a7b8e46-fb0a-461f-8c91-171b79ecabf8","account_number":"6088464791","iban":"GB27VROC23773615210426","bank_name":"ABN AMRO HOARE GOVETT SECURITIES","routing_number":"661979729","swift_bic":"BCYPGB2LBBB"},{"id":9625,"uid":"248fd05e-b9f5-430f-a6eb-f79a38fad1a3","account_number":"8855725555","iban":"GB77QDJM10916066897618","bank_name":"ABN AMRO HOARE GOVETT SECURITIES","routing_number":"017402329","swift_bic":"BCYPGB2LHHB"},{"id":825,"uid":"c91f8ec3-3c7e-460c-bb9c-43c0942a5458","account_number":"5075456453","iban":"GB87LCSG46800801934853","bank_name":"UBS CLEARING AND EXECUTION SERVICES LIMITED","routing_number":"316762553","swift_bic":"AAFMGB21"},{"id":5330,"uid":"a7e7ab23-94b5-489c-9d1f-8bc21e3bf544","account_number":"5146895848","iban":"GB25CBTG22161987383906","bank_name":"OTKRITIE SECURITIES LIMITED","routing_number":"714068804","swift_bic":"ABECGB21"},{"id":1413,"uid":"d61790e6-983d-4de2-aa3b-54952460413d","account_number":"3879406211","iban":"GB20WIXF44604236264292","bank_name":"ALKEN ASSET MANAGEMENT","routing_number":"704351932","swift_bic":"ANTSGB2LXXX"},{"id":7528,"uid":"486df39e-40aa-4e2d-8625-39d4c3269077","account_number":"6035744381","iban":"GB64WGNN63736044100563","bank_name":"ABN AMRO FUND MANAGERS LIMITED","routing_number":"111786271","swift_bic":"AAPEGB21"},{"id":6908,"uid":"c3e0804c-72db-47c1-9639-ca793769f94b","account_number":"5110156883","iban":"GB19QUZX58209113285734","bank_name":"ABN AMRO HOARE GOVETT CORPORATE FINANCE LTD.","routing_number":"276385478","swift_bic":"UBKLGB2LBAK"},{"id":4935,"uid":"1c18b44c-2972-421a-b29d-0af8dc7ec7c6","account_number":"0347754417","iban":"GB52SKWB77254858986903","bank_name":"ABU DHABI ISLAMIC BANK","routing_number":"111740042","swift_bic":"ABBYGB2L"},{"id":8696,"uid":"1f2b0d25-0a75-47f5-8c57-bb2a3741302b","account_number":"9225806338","iban":"GB05YWEL69702551284117","bank_name":"ABU DHABI ISLAMIC BANK","routing_number":"625661947","swift_bic":"BARBGB2LKEN"}]

Confirm that this is a public dataset that can be retrieved by anyone on the Network

If you chose not to confirm, what was the reason

Delta is not focused on retrieval, its focused on storage onboarding. We are going to help all SPs onboard all of their data.
Estuary is focused on retrieval.

What is the expected retrieval frequency for this data

Yearly

For how long do you plan to keep this dataset stored on Filecoin

More than 3 years

In which geographies do you plan on making storage deals

North America

How will you be distributing your data to storage providers

Others

How do you plan to choose storage providers

Others

If you answered "Others" in the previous question, what is the tool or platform you plan to use

We will be developing our tool Delta

Learn more at https://delta.store

If you already have a list of storage providers to work with, fill out their names and provider IDs below

We have developed our own list at https://data.storage.market but we will work with everyone/anyone. We actively work with storage providers and we have a storage provider on our team.

93 of them that we plan to start with, obviously we'll work with more.

➝ https://data.storage.market/api/providers/f0840770
➝ https://data.storage.market/api/providers/f01624021
➝ https://data.storage.market/api/providers/f01806491
➝ https://data.storage.market/api/providers/f08399
➝ https://data.storage.market/api/providers/f01790264
➝ https://data.storage.market/api/providers/f0875769
➝ https://data.storage.market/api/providers/f01035680
➝ https://data.storage.market/api/providers/f033356
➝ https://data.storage.market/api/providers/f01683871
➝ https://data.storage.market/api/providers/f03488
➝ https://data.storage.market/api/providers/f030379
➝ https://data.storage.market/api/providers/f01466075
➝ https://data.storage.market/api/providers/f02301
➝ https://data.storage.market/api/providers/f010479
➝ https://data.storage.market/api/providers/f010088
➝ https://data.storage.market/api/providers/f0773157
➝ https://data.storage.market/api/providers/f0717969
➝ https://data.storage.market/api/providers/f0461791
➝ https://data.storage.market/api/providers/f01746964
➝ https://data.storage.market/api/providers/f01059489
➝ https://data.storage.market/api/providers/f023467
➝ https://data.storage.market/api/providers/f01392893
➝ https://data.storage.market/api/providers/f01736668
➝ https://data.storage.market/api/providers/f022352
➝ https://data.storage.market/api/providers/f01199430
➝ https://data.storage.market/api/providers/f09848
➝ https://data.storage.market/api/providers/f01222595
➝ https://data.storage.market/api/providers/f01794610
➝ https://data.storage.market/api/providers/f01443744
➝ https://data.storage.market/api/providers/f01199442
➝ https://data.storage.market/api/providers/f02401
➝ https://data.storage.market/api/providers/f01402814
➝ https://data.storage.market/api/providers/f0104671
➝ https://data.storage.market/api/providers/f01207045
➝ https://data.storage.market/api/providers/f0406703
➝ https://data.storage.market/api/providers/f01175097
➝ https://data.storage.market/api/providers/f010446
➝ https://data.storage.market/api/providers/f0724219
➝ https://data.storage.market/api/providers/f022142
➝ https://data.storage.market/api/providers/f058369
➝ https://data.storage.market/api/providers/f01201327
➝ https://data.storage.market/api/providers/f0406322
➝ https://data.storage.market/api/providers/f01278
➝ https://data.storage.market/api/providers/f0187709
➝ https://data.storage.market/api/providers/f01045784
➝ https://data.storage.market/api/providers/f0706693
➝ https://data.storage.market/api/providers/f024184
➝ https://data.storage.market/api/providers/f01385207
➝ https://data.storage.market/api/providers/f01652333
➝ https://data.storage.market/api/providers/f01319368
➝ https://data.storage.market/api/providers/f0214334
➝ https://data.storage.market/api/providers/f082635
➝ https://data.storage.market/api/providers/f0836160
➝ https://data.storage.market/api/providers/f0135078
➝ https://data.storage.market/api/providers/f039940
➝ https://data.storage.market/api/providers/f0408717
➝ https://data.storage.market/api/providers/f01345523
➝ https://data.storage.market/api/providers/f01108096
➝ https://data.storage.market/api/providers/f01611097
➝ https://data.storage.market/api/providers/f01208862
➝ https://data.storage.market/api/providers/f01662356
➝ https://data.storage.market/api/providers/f01423116
➝ https://data.storage.market/api/providers/f017665
➝ https://data.storage.market/api/providers/f01666984
➝ https://data.storage.market/api/providers/f0440429
➝ https://data.storage.market/api/providers/f01028552
➝ https://data.storage.market/api/providers/f0707721
➝ https://data.storage.market/api/providers/f099608
➝ https://data.storage.market/api/providers/f0142637
➝ https://data.storage.market/api/providers/f01127678
➝ https://data.storage.market/api/providers/f0501283
➝ https://data.storage.market/api/providers/f01133080
➝ https://data.storage.market/api/providers/f0127896
➝ https://data.storage.market/api/providers/f010617
➝ https://data.storage.market/api/providers/f097777
➝ https://data.storage.market/api/providers/f01367109
➝ https://data.storage.market/api/providers/f01225882
➝ https://data.storage.market/api/providers/f023971
➝ https://data.storage.market/api/providers/f0466405
➝ https://data.storage.market/api/providers/f01310564
➝ https://data.storage.market/api/providers/f02620
➝ https://data.storage.market/api/providers/f02576
➝ https://data.storage.market/api/providers/f01764587
➝ https://data.storage.market/api/providers/f019551
➝ https://data.storage.market/api/providers/f034258
➝ https://data.storage.market/api/providers/f0240185
➝ https://data.storage.market/api/providers/f01619524
➝ https://data.storage.market/api/providers/f01096124
➝ https://data.storage.market/api/providers/f01163272
➝ https://data.storage.market/api/providers/f01337533
➝ https://data.storage.market/api/providers/f01091851
➝ https://data.storage.market/api/providers/f01784458
➝ https://data.storage.market/api/providers/f08403

How do you plan to make deals to your storage providers

No response

If you answered "Others/custom tool" in the previous question, enter the details here

Our new tool Delta, which we plan to release to the whole ecosystem.

Learn more at https://delta.store

Can you confirm that you will follow the Fil+ guideline

Yes

large-datacap-requests[bot] commented 1 year ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

kevzak commented 1 year ago

Datacap Request Trigger

Total DataCap requested

1PiB

Expected weekly DataCap usage rate

10TiB

Client address

f1cvqsewsg7w5y3h7xwdsdd6qgsznx6ueacmparla

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Multisig Notary address

f02049625

Client address

f1cvqsewsg7w5y3h7xwdsdd6qgsznx6ueacmparla

DataCap allocation requested

5TiB

Id

de3d066e-d137-4674-9b71-6cbd499c6b40

kernelogic commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacearytglnh5zj2fck3gcylbu3otrgey3lvmpsfbcty2sx3lyndxxg4

Address

f1cvqsewsg7w5y3h7xwdsdd6qgsznx6ueacmparla

Datacap Allocated

5.00TiB

Signer Address

f1yjhnsoga2ccnepb7t3p3ov5fzom3syhsuinxexa

Id

de3d066e-d137-4674-9b71-6cbd499c6b40

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacearytglnh5zj2fck3gcylbu3otrgey3lvmpsfbcty2sx3lyndxxg4

BobbyChoii commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaceca5webxsv6keybujxwecobekkwcxz3nbd7xvjfcoyeho25qu2kfq

Address

f1cvqsewsg7w5y3h7xwdsdd6qgsznx6ueacmparla

Datacap Allocated

5.00TiB

Signer Address

f1irqs2gmctiv3jcdfwuch7oxvf4ixh3k4b2wc24i

Id

de3d066e-d137-4674-9b71-6cbd499c6b40

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceca5webxsv6keybujxwecobekkwcxz3nbd7xvjfcoyeho25qu2kfq

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 2

Multisig Notary address

f02049625

Client address

f1cvqsewsg7w5y3h7xwdsdd6qgsznx6ueacmparla

DataCap allocation requested

10TiB

Id

0fc9956e-aeb0-4a76-99b0-e4cd5acca26d

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f02049625

Client address

f1cvqsewsg7w5y3h7xwdsdd6qgsznx6ueacmparla

Rule to calculate the allocation request amount

100% of weekly dc amount requested

DataCap allocation requested

10TiB

Total DataCap granted for client so far

5TiB

Datacap to be granted to reach the total amount requested by the client (1PiB)

0.0YiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
0 0 5TiB NaN 5TiB
cryptowhizzard commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzaceaxi567r3z5yp66hcrfv2m2b7pcfzc2wys3cl2mmbob6qaykmhzoy

Address

f1cvqsewsg7w5y3h7xwdsdd6qgsznx6ueacmparla

Datacap Allocated

10.00TiB

Signer Address

f1krmypm4uoxxf3g7okrwtrahlmpcph3y7rbqqgfa

Id

0fc9956e-aeb0-4a76-99b0-e4cd5acca26d

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceaxi567r3z5yp66hcrfv2m2b7pcfzc2wys3cl2mmbob6qaykmhzoy

psh0691 commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacedvcbwafpylyen2xjrfply6ksdhfil4qjocl6jr32bo2vjze5uv3m

Address

f1cvqsewsg7w5y3h7xwdsdd6qgsznx6ueacmparla

Datacap Allocated

10.00TiB

Signer Address

f1qdko4jg25vo35qmyvcrw4ak4fmuu3f5rif2kc7i

Id

0fc9956e-aeb0-4a76-99b0-e4cd5acca26d

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedvcbwafpylyen2xjrfply6ksdhfil4qjocl6jr32bo2vjze5uv3m

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 3

Multisig Notary address

f02049625

Client address

f1cvqsewsg7w5y3h7xwdsdd6qgsznx6ueacmparla

DataCap allocation requested

20TiB

Id

fff817a6-0e29-428d-b208-529257449e02

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f02049625

Client address

f1cvqsewsg7w5y3h7xwdsdd6qgsznx6ueacmparla

Rule to calculate the allocation request amount

200% of weekly dc amount requested

DataCap allocation requested

20TiB

Total DataCap granted for client so far

90.9YiB

Datacap to be granted to reach the total amount requested by the client (1PiB)

90.9YiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
0 0 10TiB NaN 15TiB
cryptowhizzard commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacea47j5zcmt5jjh5tntoisq55mq4fjj33riu3akwaqm26lkyfkqvuy

Address

f1cvqsewsg7w5y3h7xwdsdd6qgsznx6ueacmparla

Datacap Allocated

20.00TiB

Signer Address

f1krmypm4uoxxf3g7okrwtrahlmpcph3y7rbqqgfa

Id

fff817a6-0e29-428d-b208-529257449e02

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacea47j5zcmt5jjh5tntoisq55mq4fjj33riu3akwaqm26lkyfkqvuy

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

github-actions[bot] commented 11 months ago

This application has not seen any responses in the last 14 days, so for now it is being closed. Please feel free to contact the Fil+ Gov team to re-open the application if it is still being processed. Thank you!

-- Commented by Stale Bot.