filecoin-project / filecoin-plus-large-datasets

Hub for client applications for DataCap at a large scale
110 stars 62 forks source link

[DataCap Application] <Advanced Bioimaging Center> - <Imaging Proof of Concept> 1/3 #903

Closed MattHazard closed 1 year ago

MattHazard commented 2 years ago

Large Dataset Notary Application

To apply for DataCap to onboard your dataset to Filecoin, please fill out the following.

Core Information

Project details

Share a brief history of your project and organization

Advanced Bioimaging Center’s goals are to make advanced imaging systems and resources accessible to scientists with broad specialties to extract biologically meaningful insights. Cutting-edge microscopy technology is being built to tackle the data challenges posed by terabyte to petabyte scale imaging projects.

What is the primary source of funding for this project?

Advanced Bioimaging Center at UC Berkeley.

What other projects/ecosystem stakeholders is this project associated with?

Chan Zuckerberg Initiative, Howard Hughes Medical Institute, Philomathia Foundation, University of California, Berkeley

Use-case details

Describe the data being stored onto Filecoin

Imaging across scales spanning several orders of magnitude in space (specimens up to several millimeters in size) and time (imaging sessions lasting multiple days). One of the several types of imaging include imaging of living cells within transparent organisms without harming them and removes blurriness due to surrounding cells and tissue.

Where was the data in this dataset sourced from?

This data was sourced from the imaging systems built by The Advanced Bioimaging Center. 

Can you share a sample of the data? A link to file would work.

https://drive.google.com/drive/folders/1z1nB_DRgXYWwuUBEHYvj5hVotnAlR3W4

Confirm that this is a public dataset that can be retrieved by anyone on the Network (i.e., no specific permissions or access rights are required to view the data).

This is public, unencrypted data for any researcher to use.

What is the expected retrieval frequency for this data?

Rarely

For how long do you plan to keep this dataset stored on Filecoin?

As long as possible

DataCap allocation plan

In which geographies (countries, regions) do you plan on making storage deals?

Globally

How will you be distributing your data to storage providers? Is there an offline data transfer process?

Offline transfer

How do you plan on choosing the storage providers with whom you will be making deals? This should include a plan to ensure the data is retrievable in the future both by you and others.

We would like to select reputable storage providers with low slashing rates and are able to store our large amounts of data. We welcome storage providers to inquire about storing our data but we will be very selective based on reputation, sealing rate and age. Storage providers must have a track record of storing previous FIL Plus data and may be subject to customary enterprise-tier certifications. We also hope these SPs will work with us to pilot novel use cases for visualizations atop the IPFS protocol.

How will you be distributing deals across storage providers?

The amount of data any one storage provider will receive shall depend on the quality and availability of their node and team.

Do you have the resources/funding to start making deals as soon as you receive DataCap? What support from the community would help you onboard onto Filecoin?

No additional support is needed, though any development support with Globus Integration and funding would be appreciated. Storage providers with customary data mover tools would be greatly appreciated for onboarding onto Filecoin.
large-datacap-requests[bot] commented 2 years ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

Sunnyiscoming commented 2 years ago

What's the relationship between you and organization? Can you provide more detailed information about storage providers participated in this program, such as you can list SPs you have contacted with at present? Could you send an email to filplus-app-review@fil.org with your official domain in order to confirm your identity? Email name should includes the issue id #903.

Sunnyiscoming commented 1 year ago

Any update here?

MattHazard commented 1 year ago

I am a part of the computational staff for the lab. Other than this, we are entirely self-sufficient from our own internal storage.

raghavrmadya commented 1 year ago

Datacap Request Trigger

Total DataCap requested

5PiB

Expected weekly DataCap usage rate

100TiB

Client address

f1whxi4iekt3ejdaagsynryjhg6qstrudhghvm3by

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Multisig Notary address

f01858410

Client address

f1whxi4iekt3ejdaagsynryjhg6qstrudhghvm3by

DataCap allocation requested

50TiB

liyunzhi-666 commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacedze7wflce6ea25dketbptu5oizs2immjgwx4z5gexjdpuvise5ic

Address

f1whxi4iekt3ejdaagsynryjhg6qstrudhghvm3by

Datacap Allocated

50.00TiB

Signer Address

f1pszcrsciyixyuxxukkvtazcokexbn54amf7gvoq

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedze7wflce6ea25dketbptu5oizs2immjgwx4z5gexjdpuvise5ic

psh0691 commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacebuz5tcaxwbsl27fywhv27vvvmetlo5mssvcs5s2ugy3itelav4mu

Address

f1whxi4iekt3ejdaagsynryjhg6qstrudhghvm3by

Datacap Allocated

50.00TiB

Signer Address

f1qdko4jg25vo35qmyvcrw4ak4fmuu3f5rif2kc7i

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebuz5tcaxwbsl27fywhv27vvvmetlo5mssvcs5s2ugy3itelav4mu

BDE-io commented 1 year ago

@MattHazard Hi! Great to see you have gotten approval for DataCap and advancing the mission of preserving humanity’s most important information. If you are looking for storage providers to store these data or have any questions, please visit #bigdata-exchange on Filecoin Slack or reply here.

We have strong demand from a diverse group of SPs, who are actively looking to onboard more data.

lvschouwen commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report[^1]

No active deals found for this client.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

lvschouwen commented 1 year ago

Same request as https://github.com/filecoin-project/filecoin-plus-large-datasets/issues/905 Wonder where 2 out 3 is and I also wonder if when we can expect deals on chain for this request.

jamerduhgamer commented 1 year ago

@lvschouwen, I believe this LDN is the 2 out of 3 - https://github.com/filecoin-project/filecoin-plus-large-datasets/issues/904

Sunnyiscoming commented 1 year ago

Is there any problem with using datacap?

flyworker commented 1 year ago

What's the relationship between you and organization? Can you provide more detailed information about storage providers participated in this program, such as you can list SPs you have contacted with at present? Could you send an email to filplus-app-review@fil.org with your official domain in order to confirm your identity? Email name should includes the issue id #903.

question: who has the right to access this account? filplus-app-review@fil.org

large-datacap-requests[bot] commented 1 year ago

Thanks for your request! :exclamation: We have found some problems in the information provided. We could not find Organization Name field in the information provided We could not find Website \/ Social Media field in the information provided We could not find Total amount of DataCap being requested (between 500 TiB and 5 PiB) field in the information provided We could not find Weekly allocation of DataCap requested (usually between 1-100TiB) field in the information provided We could not find On-chain address for first allocation field in the information provided We could not find Data Type of Application field in the information provided

Please, take a look at the request and edit the body of the issue providing all the required information.
jamerduhgamer commented 1 year ago

Hi @fabriziogianni7, any idea why the above message would shown up after all this time? We already had datacap granted previously so syntax errors now don't make sense xD

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

jamerduhgamer commented 1 year ago

@simonkim0515, @panges2 could you take a look at this LDN as well? It seems to have moved back into an errored state for some reason without any edits to the LDN info above.

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

panges2 commented 1 year ago

@jamerduhgamer Fabri and Huseyin investigated this and suggested opening a new application with the new LDN template.

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 14 days, so for now it is being closed. Please feel free to contact the Fil+ Gov team to re-open the application if it is still being processed. Thank you!

aggregation-and-compliance-bot[bot] commented 10 months ago
Client f01904627 does not follow the datacap usage rules. More info here. This application has been failing the requirements for 7 days. Please take appropiate action to fix the following DataCap usage problems. Criteria Treshold Reason
Percent of used DataCap stored with top provider < 75 The percent of Data from the client that is stored with their top provider is 100%. This should be less than 75%