search

keyko-io / filecoin-large-clients-onboarding

0 stars 0 forks source link

[DataCap Application] clriesco - test #1624

Closed clriesco closed 1 year ago

clriesco commented 1 year ago

Data Owner Name

Outercore - Network Growth - Engineering

What is your role related to the dataset

Data onramp entity that provides data onboarding services to multiple clients

Data Owner Country/Region

United States

Data Owner Industry

Web3 / Crypto

Website

https://fw.services

Social Media

https://twitter.com/OutercoreEng
https://twitter.com/Estuary_Tech
https://twitter.com/FilecoinTools

Total amount of DataCap being requested

1PiB

Expected size of single dataset (one copy)

166TiB

Number of replicas to store

6

Weekly allocation of DataCap requested

10TiB

On-chain address for first allocation

f1k2ltj5hut3rlrhdhjebidjhpliys3xqphtowhsq

Data Type of Application

Public, Open Dataset (Research/Non-Profit)

Custom multisig

Identifier

No response

Share a brief history of your project and organization

Root website: https://fw.services
Project website: https://delta.store

We are the Engineering team at Protocol Labs ➝ Outercore ➝ Engineering/Network Growth. Our goal is develop tooling for entire Filecoin Ecosystem to use to onboard data to the Filecoin Network.

This is the fourth application for our tool Delta. The first one is located here. 
https://github.com/filecoin-project/filecoin-plus-large-datasets/issues/1854
https://github.com/filecoin-project/filecoin-plus-large-datasets/issues/160. The second one is here: https://github.com/filecoin-project/filecoin-plus-large-datasets/issues/1602

#### Our mission

We want an internet where resources are owned & shared by everyone. Everyone shares consensus over a distributed ledger. Data storage is verifiably stored for the end-user, and more fault tolerant. There is no central authority of control and no single point of failure. And the security promise only improves as the network continues to grow. Just imagine all of the certifications that can be automatically generated for end users by a new network like this. This is how you achieve storage as a human right for anyone in the world.

We're going to prove that we can get here by building the tools that can support onboarding 10 PiBs a day of data to the Filecoin Network. But we're also not going to forget about retrievability and helping the Filecoin Network actually become usable.

#### Delta

Our solution to archival and cold storage use cases.

Use ∆ Delta to upload all of your useful public data to Filecoin storage providers. Delta is a straight-forward Filecoin storage deal making tool that manages deals, and does not do anything else. It is purely for the function of helping Storage Providers fill capacity either through online or offline methods. It is written in Go and designed to be paired well with bare-metal infrastructure.

Is this project associated with other projects/ecosystem stakeholders?

Yes

If answered yes, what are the other projects/ecosystem stakeholders

Protocol Labs -> Outercore -> Network Growth

Describe the data being stored onto Filecoin

We are working directly with Twala Sign (https://www.twalasign.io/). They have given us permission to upload their data to Filecoin on their behalf using our tooling, Delta. 

Twala Sign digitally transforms agreement workflows with a legally binding and secure esignature powered by blockchain technology.

The types of data that will be onboarded are publicly available user data and trending feeds for the user.

Where was the data currently stored in this dataset sourced from

AWS Cloud

If you answered "Other" in the previous question, enter the details here

S3

If you are a data preparer. What is your location (City and Country)

(Dallas) United States

If you are a data preparer, how will the data be prepared? Please include tooling used and technical details?

Our new tool Delta

Learn more at https://delta.store

If you are not preparing the data, who will prepare the data? (Provide name and business)

We are preparing the data.

Has this dataset been stored on the Filecoin network before? If so, please explain and make the case why you would like to store this dataset again to the network. Provide details on preparation and/or SP distribution.

No

Please share a sample of the data

[{"id":8826,"uid":"aff3fb2e-9411-410f-8634-3f4f8aa9f2e9","brand":"Whirlpool","equipment":"Forced-air"},{"id":1825,"uid":"79cfb53c-5e39-4bb0-acf8-6f643dd2e113","brand":"Blue Star","equipment":"Cold-pressed juicer"},{"id":2038,"uid":"0d4240e2-fb3c-4f54-803f-2c005220511c","brand":"LG","equipment":"Back boiler"},{"id":3833,"uid":"16f9bdce-adf4-43ab-bde4-cc2a636f783c","brand":"Bosch","equipment":"Air ioniser"},{"id":159,"uid":"3b9fec4f-dc96-4200-b6d0-fdddf3d73d2f","brand":"Admiral","equipment":"Bachelor griller"}]

[{"id":291,"uid":"3a6d3584-8c8b-40bd-b31a-e4b6e72b76a1","brand":"Pacifico","name":"La Fin Du Monde","style":"Pilsner","hop":"Warrior","yeast":"1318 - London Ale III","malts":"Wheat mal","ibu":"33 IBU","alcohol":"3.9%","blg":"14.6°Blg"},{"id":9319,"uid":"ba204042-f063-4d69-b497-de445ced0537","brand":"Becks","name":"Racer 5 India Pale Ale, Bear Republic Bre","style":"Smoke-flavored","hop":"Centennial","yeast":"1187 - Ringwood Ale","malts":"Caramel","ibu":"41 IBU","alcohol":"3.9%","blg":"14.3°Blg"},{"id":1391,"uid":"d936e2f9-a6d1-4dc2-bd25-ac980de79644","brand":"Amstel","name":"Samuel Smith’s Imperial IPA","style":"Scottish And Irish Ale","hop":"Summit","yeast":"1388 - Belgian Strong Ale","malts":"Chocolate malt","ibu":"18 IBU","alcohol":"3.0%","blg":"18.8°Blg"},{"id":4609,"uid":"536ea4a4-3d0a-4ed0-bfcb-4095a4336c39","brand":"Dos Equis","name":"Bourbon County Stout","style":"English Pale Ale","hop":"Palisade","yeast":"2001 - Urquell Lager","malts":"Caramel","ibu":"26 IBU","alcohol":"6.2%","blg":"8.2°Blg"},{"id":8989,"uid":"171aeaca-8c83-4fc3-9315-082b26e3a5f6","brand":"Murphys","name":"Delirium Noctorum","style":"German Wheat And Rye Beer","hop":"Saaz","yeast":"3787 - Trappist High Gravity","malts":"Roasted barley","ibu":"94 IBU","alcohol":"6.4%","blg":"19.9°Blg"},{"id":6264,"uid":"53b256c9-d950-4ddd-9cd4-c92f5bd611ee","brand":"Rolling Rock","name":"Celebrator Doppelbock","style":"Belgian And French Ale","hop":"Chelan","yeast":"1007 - German Ale","malts":"Caramel","ibu":"86 IBU","alcohol":"3.8%","blg":"7.3°Blg"},{"id":4178,"uid":"4d119de4-eece-4f12-9aaa-ab381332dd47","brand":"Stella Artois","name":"Sapporo Premium","style":"Pilsner","hop":"Golding","yeast":"5112 - Brettanomyces bruxellensis","malts":"Special roast","ibu":"81 IBU","alcohol":"9.7%","blg":"17.2°Blg"},{"id":9849,"uid":"b124aaa3-196f-4c49-9f68-060af4b03a27","brand":"Patagonia","name":"Péché Mortel","style":"Bock","hop":"Cascade","yeast":"2206 - Bavarian Lager","malts":"Carapils","ibu":"93 IBU","alcohol":"7.6%","blg":"15.4°Blg"},{"id":3291,"uid":"db7c903c-bdc8-461c-a72b-b423803e49aa","brand":"Amstel","name":"Westmalle Trappist Tripel","style":"Porter","hop":"Tettnang","yeast":"1099 - Whitbread Ale","malts":"Caramel","ibu":"40 IBU","alcohol":"4.9%","blg":"15.3°Blg"},{"id":8798,"uid":"0101baba-333e-4244-810d-6f139a31b03c","brand":"Stella Artois","name":"Westmalle Trappist Tripel","style":"German Wheat And Rye Beer","hop":"Golding","yeast":"2565 - Kölsch","malts":"Rye malt","ibu":"47 IBU","alcohol":"6.4%","blg":"19.8°Blg"}]

Confirm that this is a public dataset that can be retrieved by anyone on the Network

If you chose not to confirm, what was the reason

Delta is not focused on retrieval, its focused on storage onboarding. We are going to help all SPs onboard all of their data.
Estuary is focused on retrieval.

What is the expected retrieval frequency for this data

Yearly

For how long do you plan to keep this dataset stored on Filecoin

More than 3 years

In which geographies do you plan on making storage deals

North America

How will you be distributing your data to storage providers

Others

How do you plan to choose storage providers

Others

If you answered "Others" in the previous question, what is the tool or platform you plan to use

We will be developing our tool Delta

Learn more at https://delta.store

If you already have a list of storage providers to work with, fill out their names and provider IDs below

We have developed our own list at https://data.storage.market but we will work with everyone/anyone. We actively work with storage providers and we have a storage provider on our team.

93 of them that we plan to start with, obviously we'll work with more.

➝ https://data.storage.market/api/providers/f0840770
➝ https://data.storage.market/api/providers/f01624021
➝ https://data.storage.market/api/providers/f01806491
➝ https://data.storage.market/api/providers/f08399
➝ https://data.storage.market/api/providers/f01790264
➝ https://data.storage.market/api/providers/f0875769
➝ https://data.storage.market/api/providers/f01035680
➝ https://data.storage.market/api/providers/f033356
➝ https://data.storage.market/api/providers/f01683871
➝ https://data.storage.market/api/providers/f03488
➝ https://data.storage.market/api/providers/f030379
➝ https://data.storage.market/api/providers/f01466075
➝ https://data.storage.market/api/providers/f02301
➝ https://data.storage.market/api/providers/f010479
➝ https://data.storage.market/api/providers/f010088
➝ https://data.storage.market/api/providers/f0773157
➝ https://data.storage.market/api/providers/f0717969
➝ https://data.storage.market/api/providers/f0461791
➝ https://data.storage.market/api/providers/f01746964
➝ https://data.storage.market/api/providers/f01059489
➝ https://data.storage.market/api/providers/f023467
➝ https://data.storage.market/api/providers/f01392893
➝ https://data.storage.market/api/providers/f01736668
➝ https://data.storage.market/api/providers/f022352
➝ https://data.storage.market/api/providers/f01199430
➝ https://data.storage.market/api/providers/f09848
➝ https://data.storage.market/api/providers/f01222595
➝ https://data.storage.market/api/providers/f01794610
➝ https://data.storage.market/api/providers/f01443744
➝ https://data.storage.market/api/providers/f01199442
➝ https://data.storage.market/api/providers/f02401
➝ https://data.storage.market/api/providers/f01402814
➝ https://data.storage.market/api/providers/f0104671
➝ https://data.storage.market/api/providers/f01207045
➝ https://data.storage.market/api/providers/f0406703
➝ https://data.storage.market/api/providers/f01175097
➝ https://data.storage.market/api/providers/f010446
➝ https://data.storage.market/api/providers/f0724219
➝ https://data.storage.market/api/providers/f022142
➝ https://data.storage.market/api/providers/f058369
➝ https://data.storage.market/api/providers/f01201327
➝ https://data.storage.market/api/providers/f0406322
➝ https://data.storage.market/api/providers/f01278
➝ https://data.storage.market/api/providers/f0187709
➝ https://data.storage.market/api/providers/f01045784
➝ https://data.storage.market/api/providers/f0706693
➝ https://data.storage.market/api/providers/f024184
➝ https://data.storage.market/api/providers/f01385207
➝ https://data.storage.market/api/providers/f01652333
➝ https://data.storage.market/api/providers/f01319368
➝ https://data.storage.market/api/providers/f0214334
➝ https://data.storage.market/api/providers/f082635
➝ https://data.storage.market/api/providers/f0836160
➝ https://data.storage.market/api/providers/f0135078
➝ https://data.storage.market/api/providers/f039940
➝ https://data.storage.market/api/providers/f0408717
➝ https://data.storage.market/api/providers/f01345523
➝ https://data.storage.market/api/providers/f01108096
➝ https://data.storage.market/api/providers/f01611097
➝ https://data.storage.market/api/providers/f01208862
➝ https://data.storage.market/api/providers/f01662356
➝ https://data.storage.market/api/providers/f01423116
➝ https://data.storage.market/api/providers/f017665
➝ https://data.storage.market/api/providers/f01666984
➝ https://data.storage.market/api/providers/f0440429
➝ https://data.storage.market/api/providers/f01028552
➝ https://data.storage.market/api/providers/f0707721
➝ https://data.storage.market/api/providers/f099608
➝ https://data.storage.market/api/providers/f0142637
➝ https://data.storage.market/api/providers/f01127678
➝ https://data.storage.market/api/providers/f0501283
➝ https://data.storage.market/api/providers/f01133080
➝ https://data.storage.market/api/providers/f0127896
➝ https://data.storage.market/api/providers/f010617
➝ https://data.storage.market/api/providers/f097777
➝ https://data.storage.market/api/providers/f01367109
➝ https://data.storage.market/api/providers/f01225882
➝ https://data.storage.market/api/providers/f023971
➝ https://data.storage.market/api/providers/f0466405
➝ https://data.storage.market/api/providers/f01310564
➝ https://data.storage.market/api/providers/f02620
➝ https://data.storage.market/api/providers/f02576
➝ https://data.storage.market/api/providers/f01764587
➝ https://data.storage.market/api/providers/f019551
➝ https://data.storage.market/api/providers/f034258
➝ https://data.storage.market/api/providers/f0240185
➝ https://data.storage.market/api/providers/f01619524
➝ https://data.storage.market/api/providers/f01096124
➝ https://data.storage.market/api/providers/f01163272
➝ https://data.storage.market/api/providers/f01337533
➝ https://data.storage.market/api/providers/f01091851
➝ https://data.storage.market/api/providers/f01784458
➝ https://data.storage.market/api/providers/f08403

How do you plan to make deals to your storage providers

Others/custom tool

If you answered "Others/custom tool" in the previous question, enter the details here

Our new tool Delta, which we plan to release to the whole ecosystem.

Learn more at https://delta.store

Can you confirm that you will follow the Fil+ guideline

Yes

large-request[bot] commented 1 year ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

clriesco commented 1 year ago

Datacap Request Trigger

Total DataCap requested

1PiB

Expected weekly DataCap usage rate

10TiB

Client address

f1k2ltj5hut3rlrhdhjebidjhpliys3xqphtowhsq

large-request[bot] commented 1 year ago

DataCap Allocation requested

Multisig Notary address

t01021 #Normal

Client address

f1k2ltj5hut3rlrhdhjebidjhpliys3xqphtowhsq

DataCap allocation requested

5TiB

Id

3343b9e1-bb40-48f7-a27b-3bc21794a395

clriesco commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacecdg5iqwve7ililgshnrrc3qld5dgyjgtq6ceifa7vk6cqwdzv52s

Address

f1k2ltj5hut3rlrhdhjebidjhpliys3xqphtowhsq

Datacap Allocated

5.00TiB

Signer Address

f1yjhnsoga2ccnepb7t3p3ov5fzom3syhsuinxexa

Id

4c6dbc44-5597-41b3-a33a-862b7b6c93cb

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacecdg5iqwve7ililgshnrrc3qld5dgyjgtq6ceifa7vk6cqwdzv52s

clriesco commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacecx4nk3la7mugeel7u2dwi5nfyhf74ouj3w3cp5witqkigkf424je

Address

f1k2ltj5hut3rlrhdhjebidjhpliys3xqphtowhsq

Datacap Allocated

5.00TiB

Signer Address

f1irqs2gmctiv3jcdfwuch7oxvf4ixh3k4b2wc24i

Id

4c6dbc44-5597-41b3-a33a-862b7b6c93cb

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacecx4nk3la7mugeel7u2dwi5nfyhf74ouj3w3cp5witqkigkf424je

clriesco commented 1 year ago

.

clriesco commented 1 year ago

.