search

filecoin-project / filecoin-plus-large-datasets

Hub for client applications for DataCap at a large scale
110 stars 62 forks source link

[DataCap Application] <BossDB Open Neuroimagery Datasets><1/2> #1835

Closed caoxinhe0108 closed 8 months ago

caoxinhe0108 commented 1 year ago

Data Owner Name

Johns Hopkins University Applied Physics Laboratory

Data Owner Country/Region

United States

Data Owner Industry

Life Science / Healthcare

Website

https://registry.opendata.aws/bossdb

Social Media

https://registry.opendata.aws/bossdb

Total amount of DataCap being requested

5PiB

Weekly allocation of DataCap requested

600TiB

On-chain address for first allocation

f1fji3yg3t5xyhuzhq34yl7kihbe4jnuyzf55jt7y

Data Type of Application

Public, Open Dataset (Research/Non-Profit)

Custom multisig

Identifier

No response

Share a brief history of your project and organization

This data ecosystem, Brain Observatory Storage Service & Database (BossDB), contains several neuro-imaging datasets across multiple modalities and scales, ranging from nanoscale (electron microscopy), to microscale (cleared lightsheet microscopy and array tomography), and mesoscale (structural and functional magnetic resonance imaging). Additionally, many of the datasets include dense segmentation and meshes.

Is this project associated with other projects/ecosystem stakeholders?

Yes

If answered yes, what are the other projects/ecosystem stakeholders

No response

Describe the data being stored onto Filecoin

This data ecosystem, Brain Observatory Storage Service & Database (BossDB), contains several neuro-imaging datasets across multiple modalities and scales, ranging from nanoscale (electron microscopy), to microscale (cleared lightsheet microscopy and array tomography), and mesoscale (structural and functional magnetic resonance imaging). Additionally, many of the datasets include dense segmentation and meshes.

Where was the data currently stored in this dataset sourced from

AWS Cloud

If you answered "Other" in the previous question, enter the details here

No response

How do you plan to prepare the dataset

lotus

If you answered "other/custom tool" in the previous question, enter the details here

No response

Please share a sample of the data

aws s3 ls --no-sign-request s3://bossdb-open-data/

2022-02-26 11:02:10 1.5 KiB witvliet2020/Dataset_8_Segmentation/synapses/info
2022-02-26 11:02:11 1.5 KiB witvliet2020/Dataset_8_Segmentation/synapses/provenance
2022-02-18 05:20:18 446 Bytes witvliet2020/Dataset_8_Segmentation/template/info
2022-06-21 10:00:04 3.8 KiB witvliet2020/Dataset_8_Segmentation_Properties/info

Total Objects: 664318037
Total Size: 849.7 TiB

Confirm that this is a public dataset that can be retrieved by anyone on the Network

If you chose not to confirm, what was the reason

No response

What is the expected retrieval frequency for this data

Yearly

For how long do you plan to keep this dataset stored on Filecoin

1.5 to 2 years

In which geographies do you plan on making storage deals

Greater China, Asia other than Greater China, North America, South America

How will you be distributing your data to storage providers

Cloud storage (i.e. S3)

How do you plan to choose storage providers

Slack, Big data exchange

If you answered "Others" in the previous question, what is the tool or platform you plan to use

No response

If you already have a list of storage providers to work with, fill out their names and provider IDs below

No response

How do you plan to make deals to your storage providers

No response

If you answered "Others/custom tool" in the previous question, enter the details here

No response

Can you confirm that you will follow the Fil+ guideline

Yes

large-datacap-requests[bot] commented 1 year ago

Thanks for your request!

Heads up, you’re requesting more than the typical weekly onboarding rate of DataCap!
large-datacap-requests[bot] commented 1 year ago

Thanks for your request!

Heads up, you’re requesting more than the typical weekly onboarding rate of DataCap!
large-datacap-requests[bot] commented 1 year ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

large-datacap-requests[bot] commented 1 year ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

Sunnyiscoming commented 1 year ago

Can you list sps you will cooperate with?

caoxinhe0108 commented 1 year ago

This is the SP that I am currently cooperating with, please check it out, thank you

f02050599 Bang Sai, Phra Nakhon Si Ayutthaya, TH f02063202 Mueang Nonthaburi, Nonthaburi, TH f02064089 Singapore, Singapore, SG f02058557 Kuala Lumpur, Kuala Lumpur, MY f02045964 Sham Shui Po, Sham Shui Po, HK f02055638 Kuala Lumpur, Kuala Lumpur, MY f02056391 Hong Kong, Central and Western, HK f02048990 Hong Kong, Central and Western, HK f02041085 Tokyo, Tokyo, JP f02044834 Singapore, Singapore, SG f02046736 Jakarta, Jakarta, ID f02030031 Singapore, Singapore, SG f02029895 Seoul, Seoul, KR f01170282 Hong Kong, Central and Western, HK f02059055 Seoul, Seoul, KR f02042992 Singapore, Singapore, SG

Sunnyiscoming commented 1 year ago

Datacap Request Trigger

Total DataCap requested

5PiB

Expected weekly DataCap usage rate

600TiB

Client address

f1fji3yg3t5xyhuzhq34yl7kihbe4jnuyzf55jt7y

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Multisig Notary address

f02049625

Client address

f1fji3yg3t5xyhuzhq34yl7kihbe4jnuyzf55jt7y

DataCap allocation requested

256TiB

Id

6a95efe4-bdcd-45e9-a255-76d5e7a83eb0

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Multisig Notary address

f02049625

Client address

f1fji3yg3t5xyhuzhq34yl7kihbe4jnuyzf55jt7y

DataCap allocation requested

256TiB

Id

c210aba1-56b3-4777-b0cd-a78385bad5ae

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report[^1]

No application info found for this issue on https://filplus.d.interplanetary.one/clients.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report[^1]

No application info found for this issue on https://filplus.d.interplanetary.one/clients.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

TimGuo7 commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzaceb7kuuwdn35dizbsokfmdsdzljsblq4bsyykcyqwbhtgdamym3dey

Address

f1fji3yg3t5xyhuzhq34yl7kihbe4jnuyzf55jt7y

Datacap Allocated

256.00TiB

Signer Address

f1yslbnnqzrjlyuxsmyxfbqcc7xthcavgpripjevi

Id

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceb7kuuwdn35dizbsokfmdsdzljsblq4bsyykcyqwbhtgdamym3dey

caoxinhe0108 commented 1 year ago

Thank you for your support @TimGuo7

psh0691 commented 1 year ago

I'll sign the first round and make sure it meets the Fileplus regulations.

psh0691 commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacedcphgffxtg2lqvbiddpla4yfanhhwkyvrb577e7sxqffjcblt5s6

Address

f1fji3yg3t5xyhuzhq34yl7kihbe4jnuyzf55jt7y

Datacap Allocated

256.00TiB

Signer Address

f1qdko4jg25vo35qmyvcrw4ak4fmuu3f5rif2kc7i

Id

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedcphgffxtg2lqvbiddpla4yfanhhwkyvrb577e7sxqffjcblt5s6

cryptowhizzard commented 1 year ago

Hi @caoxinhe0108

Thank you for applying for datacap. As Filecoin FIL+ notary i am screening your application and conducting due diligence.

Looking at your application i have some questions: Why did you choose to only store your application only Asia (based on above SPs)?

What hardware do you have over there to prepare this dataset? What is your internet connection speed? Do you already have the data downloaded on the premises? Can you show us some visible proof of the size of your data and the storage / packing hardware you have there?

As last question i would like you to fill out this form to provide us with the necessary information to make a educated decision on your LDN request if we would like to support it.

Thanks!

caoxinhe0108 commented 1 year ago

Thank you for your attention. We are also looking for SPs from outside Asia and will add them later. We will launch this project soon. Thank you

caoxinhe0108 commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report[^1]

No active deals found for this client.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

caoxinhe0108 commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the full report.

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 2

Multisig Notary address

f02049625

Client address

f1fji3yg3t5xyhuzhq34yl7kihbe4jnuyzf55jt7y

DataCap allocation requested

512TiB

Id

97436521-8b47-40f6-a079-6aae69668685

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f02049625

Client address

f1fji3yg3t5xyhuzhq34yl7kihbe4jnuyzf55jt7y

Rule to calculate the allocation request amount

100% weekly > 0.5PiB, requesting 0.5PiB

DataCap allocation requested

512TiB

Total DataCap granted for client so far

256TiB

Datacap to be granted to reach the total amount requested by the client (5PiB)

4.75PiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
2669 7 256TiB 18.28 64.37TiB
mikezli commented 1 year ago

e86bcd726a9e10afe6e4659c6f9fd75

lotus client retrieve --provider f02052217 QmaV1G63CVnmuT6WhxVDbnsJ1z3EUbwmMhjf3nDyyYWgmt QmaV1G63CVnmuT6WhxVDbnsJ1z3EUbwmMhjf3nDyyYWgmt.temp

mikezli commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacec4udp4d7ufpd47vyacctl7k4pkaurmaqez5afz665ujoc4l5z5bi

Address

f1fji3yg3t5xyhuzhq34yl7kihbe4jnuyzf55jt7y

Datacap Allocated

512.00TiB

Signer Address

f1dnb3uz7sylxk6emti3ififcvu3nlufnnsjui6ea

Id

97436521-8b47-40f6-a079-6aae69668685

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacec4udp4d7ufpd47vyacctl7k4pkaurmaqez5afz665ujoc4l5z5bi

igoovo commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval report.

igoovo commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaceayyh2oj4slwaolpq54q4k5merfbwo2qqkck4imqahlzzc65zhlj6

Address

f1fji3yg3t5xyhuzhq34yl7kihbe4jnuyzf55jt7y

Datacap Allocated

512.00TiB

Signer Address

f1shnsfayxqll77svffaxnjenms7bbbysbqcatrpy

Id

97436521-8b47-40f6-a079-6aae69668685

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceayyh2oj4slwaolpq54q4k5merfbwo2qqkck4imqahlzzc65zhlj6

caoxinhe0108 commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval report.

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 3

Multisig Notary address

f02049625

Client address

f1fji3yg3t5xyhuzhq34yl7kihbe4jnuyzf55jt7y

DataCap allocation requested

1PiB

Id

f05c5b3f-c213-4f86-bbfb-90585a0569d5

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f02049625

Client address

f1fji3yg3t5xyhuzhq34yl7kihbe4jnuyzf55jt7y

Rule to calculate the allocation request amount

200% weekly > 1PiB, requesting 1PiB

DataCap allocation requested

1PiB

Total DataCap granted for client so far

465661.3YiB

Datacap to be granted to reach the total amount requested by the client (5PiB)

465661.3YiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
10225 7 512TiB 20.81 127.68TiB
spaceT9 commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval report.

spaceT9 commented 1 year ago

The CID report looks great, hope you can communicate with your SPs to increase the http retreival sucess rate in the future. Support this round

spaceT9 commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacebev3262xguqzvw77bswnx3r3lkny7bcbitvi5ceswjt73tjh5kdw

Address

f1fji3yg3t5xyhuzhq34yl7kihbe4jnuyzf55jt7y

Datacap Allocated

1.00PiB

Signer Address

f1dktm3abgn7wkpepxab4q3eaw24qktizk4esfzyy

Id

f05c5b3f-c213-4f86-bbfb-90585a0569d5

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebev3262xguqzvw77bswnx3r3lkny7bcbitvi5ceswjt73tjh5kdw

luobin544 commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval report.

luobin544 commented 1 year ago

The data backup continues to grow, and each node can be retrieved, and the initial report is healthy without any issues.Willing to support this round

luobin544 commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaceasc4nkahwyco24v27l6mkxcebzv6xgqajcd7ceq7axckx4adzz6y

Address

f1fji3yg3t5xyhuzhq34yl7kihbe4jnuyzf55jt7y

Datacap Allocated

1.00PiB

Signer Address

f1tbd632f6w62glfaf7wjpimacbnjiz26poyoes2q

Id

f05c5b3f-c213-4f86-bbfb-90585a0569d5

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceasc4nkahwyco24v27l6mkxcebzv6xgqajcd7ceq7axckx4adzz6y

github-actions[bot] commented 11 months ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

caoxinhe0108 commented 11 months ago

checker:manualTrigger

filplus-checker-app[bot] commented 11 months ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

github-actions[bot] commented 11 months ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

caoxinhe0108 commented 11 months ago

checker:manualTrigger

filplus-checker-app[bot] commented 11 months ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

large-datacap-requests[bot] commented 11 months ago

DataCap Allocation requested

Request number 4

Multisig Notary address

f02049625

Client address

f1fji3yg3t5xyhuzhq34yl7kihbe4jnuyzf55jt7y

DataCap allocation requested

2PiB

Id

80e2647b-59a5-449c-a2a5-afc4b8eb2fd0

large-datacap-requests[bot] commented 11 months ago

Stats & Info for DataCap Allocation

Multisig Notary address

f02049625

Client address

f1fji3yg3t5xyhuzhq34yl7kihbe4jnuyzf55jt7y

Rule to calculate the allocation request amount

400% weekly > 2PiB, requesting 2PiB

DataCap allocation requested

2PiB

Total DataCap granted for client so far

931322574615478927360.0YiB

Datacap to be granted to reach the total amount requested by the client (5PiB)

931322574615478927360.0YiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
29938 16 1PiB 16.58 271.15TiB
SuperChaiChai commented 11 months ago

checker:manualTrigger

filplus-checker-app[bot] commented 11 months ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

⚠️ 34.14% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.