filecoin-project / filecoin-plus-large-datasets

Hub for client applications for DataCap at a large scale
110 stars 62 forks source link

[DataCap Application] NIH NCBI Sequence Read Archive (SRA) src-1 #1884

Closed AgeeWeb3 closed 1 year ago

AgeeWeb3 commented 1 year ago

Data Owner Name

National Library of Medicine (NLM)

Data Owner Country/Region

United States

Data Owner Industry

Life Science / Healthcare

Website

https://www.ncbi.nlm.nih.gov/sra/docs/sra-cloud/

Social Media

https://www.nlm.nih.gov/
https://twitter.com/NLM_NIH

Total amount of DataCap being requested

5PiB

Weekly allocation of DataCap requested

800TiB

On-chain address for first allocation

f1g7y7hqjqyp4rkflqd3a2z5cr5olmtdvpj6oz5wq

Data Type of Application

Public, Open Dataset (Research/Non-Profit)

Custom multisig

Identifier

No response

Share a brief history of your project and organization

The Sequence Read Archive (SRA), produced by the National Center for Biotechnology Information (NCBI) at the National Library of Medicine (NLM) at the National Institutes of Health (NIH), stores raw DNA sequencing data and alignment information from high-throughput sequencing platforms. The SRA provides open access to these biological sequence data to support the research community's efforts to enhance reproducibility and make new discoveries by comparing data sets. Buckets in this registry contain public SRA data in the original (user submitted) format from select high value and newly-released studies as well as all public-access SRA formatted ETL+BQS data. Also included is all SRA metadata that can be leveraged for attribute-based data discovery.

Is this project associated with other projects/ecosystem stakeholders?

No

If answered yes, what are the other projects/ecosystem stakeholders

No response

Describe the data being stored onto Filecoin

.bam, .cram, and .fastq files in a public S3 bucket. This is the first of two S3 buckets for source submissions from sequencing methodologies such as PacBio, Oxford Nanopore Technologies, and 10X Genomics.

Where was the data currently stored in this dataset sourced from

AWS Cloud

If you answered "Other" in the previous question, enter the details here

No response

How do you plan to prepare the dataset

lotus, singularity

If you answered "other/custom tool" in the previous question, enter the details here

No response

Please share a sample of the data

aws s3 ls --no-sign-request s3://sra-pub-src-1/

Confirm that this is a public dataset that can be retrieved by anyone on the Network

If you chose not to confirm, what was the reason

No response

What is the expected retrieval frequency for this data

Sporadic

For how long do you plan to keep this dataset stored on Filecoin

1 to 1.5 years

In which geographies do you plan on making storage deals

Asia other than Greater China, North America, South America, Europe

How will you be distributing your data to storage providers

HTTP or FTP server, Shipping hard drives, Lotus built-in data transfer

How do you plan to choose storage providers

Slack, Filmine, Big data exchange

If you answered "Others" in the previous question, what is the tool or platform you plan to use

No response

If you already have a list of storage providers to work with, fill out their names and provider IDs below

No response

How do you plan to make deals to your storage providers

Lotus client

If you answered "Others/custom tool" in the previous question, enter the details here

No response

Can you confirm that you will follow the Fil+ guideline

Yes

large-datacap-requests[bot] commented 1 year ago

Thanks for your request!

Heads up, you’re requesting more than the typical weekly onboarding rate of DataCap!
large-datacap-requests[bot] commented 1 year ago

Thanks for your request!

Heads up, you’re requesting more than the typical weekly onboarding rate of DataCap!
large-datacap-requests[bot] commented 1 year ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

large-datacap-requests[bot] commented 1 year ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

Sunnyiscoming commented 1 year ago

Can you introduce your organizaion? Please share more detailed information about sps you will cooperate with.

AgeeWeb3 commented 1 year ago

Hi @Sunnyiscoming I had participated in hackathon, and after learning about it I discovered filecoin and interested in it. Also, I took part in Slingshot and keep an eye on the community for a long time. I'm looking for SPs who have good reputation and experience about dealing. That's in the progress.

Sunnyiscoming commented 1 year ago

Datacap Request Trigger

Total DataCap requested

5PiB

Expected weekly DataCap usage rate

800TiB

Client address

f1g7y7hqjqyp4rkflqd3a2z5cr5olmtdvpj6oz5wq

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Multisig Notary address

f02049625

Client address

f1g7y7hqjqyp4rkflqd3a2z5cr5olmtdvpj6oz5wq

DataCap allocation requested

256TiB

Id

7daa2567-aa95-4ecf-a4aa-5461a5078383

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Multisig Notary address

f02049625

Client address

f1g7y7hqjqyp4rkflqd3a2z5cr5olmtdvpj6oz5wq

DataCap allocation requested

256TiB

Id

548218af-c034-48c7-9300-5d9bb1ccc190

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report[^1]

No application info found for this issue on https://filplus.d.interplanetary.one/clients.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

TakiChain commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacebt7ou3moza242rwc3ul45isdjl6uu7dglizgoh5gaylymws2b5ha

Address

f1g7y7hqjqyp4rkflqd3a2z5cr5olmtdvpj6oz5wq

Datacap Allocated

256.00TiB

Signer Address

f15impf3j2zcaex4lhyxndxswuuhv24vzstuqtxsi

Id

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebt7ou3moza242rwc3ul45isdjl6uu7dglizgoh5gaylymws2b5ha

Bennyyangpu commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacebyulq7mp6xk76ts4hnk7lqgae3k653ggm4wpdaxsmfx5cbgfhvj4

Address

f1g7y7hqjqyp4rkflqd3a2z5cr5olmtdvpj6oz5wq

Datacap Allocated

256.00TiB

Signer Address

f174fg3bqbln3zjnkxtyf6s54txqkr7yqkj6cig7y

Id

548218af-c034-48c7-9300-5d9bb1ccc190

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebyulq7mp6xk76ts4hnk7lqgae3k653ggm4wpdaxsmfx5cbgfhvj4

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 2

Multisig Notary address

f02049625

Client address

f1g7y7hqjqyp4rkflqd3a2z5cr5olmtdvpj6oz5wq

DataCap allocation requested

512TiB

Id

ee773c65-f904-49e2-8a44-a903a6b9eadf

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f02049625

Client address

f1g7y7hqjqyp4rkflqd3a2z5cr5olmtdvpj6oz5wq

Rule to calculate the allocation request amount

10% of total dc amount requested

DataCap allocation requested

512TiB

Total DataCap granted for client so far

256TiB

Datacap to be granted to reach the total amount requested by the client (5PiB)

4.75PiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
null null 256TiB null 60.56TiB
Casey-PG commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzaceabb62u7c3xeapsw76h6x5zuzg6xmxccw2ie7hgg52c7bpqzc32bu

Address

f1g7y7hqjqyp4rkflqd3a2z5cr5olmtdvpj6oz5wq

Datacap Allocated

512.00TiB

Signer Address

f1d4yb3wags3mtddzesxoo63jv7dmlec3bq4yteni

Id

ee773c65-f904-49e2-8a44-a903a6b9eadf

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceabb62u7c3xeapsw76h6x5zuzg6xmxccw2ie7hgg52c7bpqzc32bu

BobbyChoii commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaceck7irvp6567w7iwixoeqx4unc7bz75b3lcmg6heqv2ml67egc5l4

Address

f1g7y7hqjqyp4rkflqd3a2z5cr5olmtdvpj6oz5wq

Datacap Allocated

512.00TiB

Signer Address

f1irqs2gmctiv3jcdfwuch7oxvf4ixh3k4b2wc24i

Id

ee773c65-f904-49e2-8a44-a903a6b9eadf

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceck7irvp6567w7iwixoeqx4unc7bz75b3lcmg6heqv2ml67egc5l4

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 3

Multisig Notary address

f02049625

Client address

f1g7y7hqjqyp4rkflqd3a2z5cr5olmtdvpj6oz5wq

DataCap allocation requested

1PiB

Id

f08ba355-cda1-4dcb-910d-edc3cd395c84

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f02049625

Client address

f1g7y7hqjqyp4rkflqd3a2z5cr5olmtdvpj6oz5wq

Rule to calculate the allocation request amount

20% of total dc amount requested

DataCap allocation requested

1PiB

Total DataCap granted for client so far

465661.3YiB

Datacap to be granted to reach the total amount requested by the client (5PiB)

-5.62B

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
null null 512TiB null 109.62TiB
TakiChain commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacedkj44rgaw3lhl37bbengelpibncttdhybnl4hn4fobvolrlzqnfu

Address

f1g7y7hqjqyp4rkflqd3a2z5cr5olmtdvpj6oz5wq

Datacap Allocated

1.00PiB

Signer Address

f15impf3j2zcaex4lhyxndxswuuhv24vzstuqtxsi

Id

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedkj44rgaw3lhl37bbengelpibncttdhybnl4hn4fobvolrlzqnfu

BobbyChoii commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaceca5q64nh5krjayoyh35rieajuwbrdesg6hlpljfafda7ftjyl4x2

Address

f1g7y7hqjqyp4rkflqd3a2z5cr5olmtdvpj6oz5wq

Datacap Allocated

1.00PiB

Signer Address

f1irqs2gmctiv3jcdfwuch7oxvf4ixh3k4b2wc24i

Id

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceca5q64nh5krjayoyh35rieajuwbrdesg6hlpljfafda7ftjyl4x2

cryptowhizzard commented 1 year ago

Hi,

None of the data stored here is retrievable. This is against the FIL+ Guidelines. Secondly there is no diversity, everything is in HongKong, also against Fil+ Guidelines.

Can you please correct this?

Scherm­afbeelding 2023-05-05 om 19 47 32
large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 4

Multisig Notary address

f02049625

Client address

f1g7y7hqjqyp4rkflqd3a2z5cr5olmtdvpj6oz5wq

DataCap allocation requested

2PiB

Id

97a92c76-8df9-4a3a-8233-5b3178b5c051

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f02049625

Client address

f1g7y7hqjqyp4rkflqd3a2z5cr5olmtdvpj6oz5wq

Rule to calculate the allocation request amount

40% of total dc amount requested

DataCap allocation requested

2PiB

Total DataCap granted for client so far

931322574615478927360.0YiB

Datacap to be granted to reach the total amount requested by the client (5PiB)

-1.12B

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
null null 1PiB null 246.06TiB
Casey-PG commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzaceblkx6syzonqfmeavc5vknxf6bftaj3gw6mbakxaf5lpggvhz4gf6

Address

f1g7y7hqjqyp4rkflqd3a2z5cr5olmtdvpj6oz5wq

Datacap Allocated

2.00PiB

Signer Address

f1d4yb3wags3mtddzesxoo63jv7dmlec3bq4yteni

Id

97a92c76-8df9-4a3a-8233-5b3178b5c051

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceblkx6syzonqfmeavc5vknxf6bftaj3gw6mbakxaf5lpggvhz4gf6

AthSmith commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacedjh2p26reb7sqjyeos3pnjwao6rqnqmxxvxxls7rpwm5wlusnqz2

Address

f1g7y7hqjqyp4rkflqd3a2z5cr5olmtdvpj6oz5wq

Datacap Allocated

2.00PiB

Signer Address

f1vxbqrf7rfum3n6m5u6eb4re6xj7amvsaqnzu64y

Id

97a92c76-8df9-4a3a-8233-5b3178b5c051

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedjh2p26reb7sqjyeos3pnjwao6rqnqmxxvxxls7rpwm5wlusnqz2

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

AgeeWeb3 commented 1 year ago

Please remain open.

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 14 days, so for now it is being closed. Please feel free to contact the Fil+ Gov team to re-open the application if it is still being processed. Thank you!

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

⚠️ 55.67% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

cryptowhizzard commented 1 year ago

No need to keep this application open.

The Sp's in red are involved in CID sharing. The Sp's with AT3 ( in colum AT ) don't support retrieval. Akin, nothing works.

Scherm­afbeelding 2023-07-31 om 18 00 40
large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 5

Multisig Notary address

f02049625

Client address

f1g7y7hqjqyp4rkflqd3a2z5cr5olmtdvpj6oz5wq

DataCap allocation requested

1.25PiB

Id

c5e5c7b2-2cd8-4704-b676-6b08495ab545

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f02049625

Client address

f1g7y7hqjqyp4rkflqd3a2z5cr5olmtdvpj6oz5wq

Rule to calculate the allocation request amount

400% weekly > 2PiB, requesting 2PiB

DataCap allocation requested

1.25PiB

Total DataCap granted for client so far

1.862645149230957e+37YiB

Datacap to be granted to reach the total amount requested by the client (5PiB)

1.862645149230957e+37YiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
93703 18 2PiB 17.34 0B
AgeeWeb3 commented 1 year ago

@cryptowhizzard Your words mean nothing.

Wengeding commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

⚠️ 55.67% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

Wengeding commented 1 year ago

have done DD with the applicant, report looks normal to me. some indicators could be improved a bit more.

Wengeding commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzaceae2m2sfdy2n6bfbfyiapc66lkcj3ob3cczqufnlvlqpla5pv42ui

Address

f1g7y7hqjqyp4rkflqd3a2z5cr5olmtdvpj6oz5wq

Datacap Allocated

1.25PiB

Signer Address

f1txfsjmix4vlzido4dkildrnbw26owtlbslexmpa

Id

c5e5c7b2-2cd8-4704-b676-6b08495ab545

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceae2m2sfdy2n6bfbfyiapc66lkcj3ob3cczqufnlvlqpla5pv42ui

AthSmith commented 1 year ago

In light of the applicant's long history of engagement and response record, this is the kind of consistent DD response attitude that file plus requires. Thank you for integrating more trustworthy data into the network and please continue to follow fileplus' guidelines.

AthSmith commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacebk5o5j5xcwahkeek5imjfsxxtr5l3ek7xnxt2q7ir2lpkr7qp3om

Address

f1g7y7hqjqyp4rkflqd3a2z5cr5olmtdvpj6oz5wq

Datacap Allocated

1.25PiB

Signer Address

f1vxbqrf7rfum3n6m5u6eb4re6xj7amvsaqnzu64y

Id

c5e5c7b2-2cd8-4704-b676-6b08495ab545

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebk5o5j5xcwahkeek5imjfsxxtr5l3ek7xnxt2q7ir2lpkr7qp3om

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 14 days, so for now it is being closed. Please feel free to contact the Fil+ Gov team to re-open the application if it is still being processed. Thank you!

clriesco commented 1 year ago

Removed stale label and reopened issue :)

cryptowhizzard commented 1 year ago

https://github.com/filecoin-project/filecoin-plus-large-datasets/issues/1898#issuecomment-1701036576

Scherm­afbeelding 2023-08-31 om 15 33 22

large-datacap-requests[bot] commented 1 year ago

The issue reached the total datacap requested. This should be closed

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f02049625

Client address

f1g7y7hqjqyp4rkflqd3a2z5cr5olmtdvpj6oz5wq

Rule to calculate the allocation request amount

total dc reached

DataCap allocation requested

0

Total DataCap granted for client so far

1.1641532182693484e+53YiB

Datacap to be granted to reach the total amount requested by the client (5PiB)

1.1641532182693484e+53YiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
112763 24 1.25PiB 14.92 499.37TiB
github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 14 days, so for now it is being closed. Please feel free to contact the Fil+ Gov team to re-open the application if it is still being processed. Thank you!

-- Commented by Stale Bot.