filecoin-project / filecoin-plus-large-datasets

Hub for client applications for DataCap at a large scale
110 stars 62 forks source link

[DataCap Application] FILTAKES - UK Biobank Pan-Ancestry Summary Statistics #2246

Closed filtakes closed 1 year ago

filtakes commented 1 year ago

Data Owner Name

Analytic and Translational Genetics Unit, Massachusetts General Hospital and the Broad Institute

What is your role related to the dataset

Data Preparer

Data Owner Country/Region

United Kingdom

Data Owner Industry

Life Science / Healthcare

Website

https://pan.ukbb.broadinstitute.org/

Social Media

http://www.filtakes.com

Total amount of DataCap being requested

5PiB

Expected size of single dataset (one copy)

508.6 TiB

Number of replicas to store

10

Weekly allocation of DataCap requested

600TiB

On-chain address for first allocation

f1l7b3sko5hqtpsb3pwjqqymh3ssoslucafzhd2jy

Data Type of Application

Public, Open Dataset (Research/Non-Profit)

Custom multisig

Identifier

No response

Share a brief history of your project and organization

Founded in 2021, filtakes is a technology company focusing on distributed storage technology, data computing and intelligent AI. We provide data storage and technical support services to various companies and have complete industry solutions.

We provide customers with technical services such as data collection and download, data storage, and big data model training to help customers gain deep insights from data and promote business decision-making and innovation.

Is this project associated with other projects/ecosystem stakeholders?

No

If answered yes, what are the other projects/ecosystem stakeholders

No response

Describe the data being stored onto Filecoin

UK Biobank Pan-Ancestry Summary Statistics
National Cancer Institute Imaging Data Commons (IDC) Collections
NASA Space Biology Open Science Data Repository (OSDR)
New Zealand Imagery
Indiana Statewide Digital Aerial Imagery Catalog
Downscaled Climate Data for Alaska

Where was the data currently stored in this dataset sourced from

AWS Cloud

If you answered "Other" in the previous question, enter the details here

No response

If you are a data preparer. What is your location (Country/Region)

China

If you are a data preparer, how will the data be prepared? Please include tooling used and technical details?

Singularity or go-graphsplit

If you are not preparing the data, who will prepare the data? (Provide name and business)

No response

Has this dataset been stored on the Filecoin network before? If so, please explain and make the case why you would like to store this dataset again to the network. Provide details on preparation and/or SP distribution.

According to my investigation, no organization or individual currently stores this data on the filecoin network.

Please share a sample of the data

https://registry.opendata.aws/broad-pan-ukb/
aws s3 ls --no-sign-request --recursive --human-readable --summarize s3://pan-ukb-us-east-1/
Total Size: 143.5 TiB

https://registry.opendata.aws/nci-imaging-data-commons/
aws s3 ls --no-sign-request --recursive --human-readable --summarize s3://idc-open-data/
Total Size: 61.3 TiB

https://registry.opendata.aws/nasa-osdr/
aws s3 ls --no-sign-request --recursive --human-readable --summarize s3://nasa-osdr/
Total Size: 133.3 TiB

https://registry.opendata.aws/nz-imagery/
aws s3 ls --no-sign-request --recursive --human-readable --summarize s3://nz-imagery/
Total Size: 18 TiB

https://registry.opendata.aws/in-imagery/
aws s3 ls --no-sign-request --recursive --human-readable --summarize s3://gisimageryingov/
Total Size: 72.9 TiB

https://registry.opendata.aws/wrf-alaska-snap/
aws s3 ls --no-sign-request --recursive --human-readable --summarize s3://wrf-ak-ar5/
Total Size: 79.9 TiB

Confirm that this is a public dataset that can be retrieved by anyone on the Network

If you chose not to confirm, what was the reason

No response

What is the expected retrieval frequency for this data

Yearly

For how long do you plan to keep this dataset stored on Filecoin

1.5 to 2 years

In which geographies do you plan on making storage deals

Greater China, Asia other than Greater China, North America, Europe

How will you be distributing your data to storage providers

HTTP or FTP server, Shipping hard drives, Lotus built-in data transfer

How do you plan to choose storage providers

Slack, Partners

If you answered "Others" in the previous question, what is the tool or platform you plan to use

No response

If you already have a list of storage providers to work with, fill out their names and provider IDs below

f02311133 Allen
f01946104 Chen Liu
f01153105 Toshiro

We will update here as we reach out to more SPs

How do you plan to make deals to your storage providers

Boost client, Singularity

If you answered "Others/custom tool" in the previous question, enter the details here

No response

Can you confirm that you will follow the Fil+ guideline

Yes

large-datacap-requests[bot] commented 1 year ago

Thanks for your request!

Heads up, you’re requesting more than the typical weekly onboarding rate of DataCap!
large-datacap-requests[bot] commented 1 year ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

herrehesse commented 1 year ago

Not allowed to do merged dataset requests.

Please do them seperately.

Sunnyiscoming commented 1 year ago

As notary said, merged datasets is not allowed. One application for one dataset.