filecoin-project / filecoin-plus-large-datasets

Hub for client applications for DataCap at a large scale
110 stars 62 forks source link

[DataCap Application]EMPIAR(1/3) #2152

Closed TOPPOOL-LEE closed 10 months ago

TOPPOOL-LEE commented 1 year ago

Data Owner Name

EMPIAR

What is your role related to the dataset

Data Preparer

Data Owner Country/Region

United Kingdom

Data Owner Industry

Life Science / Healthcare

Website

https://www.ebi.ac.uk/empiar/

Social Media

https://www.ebi.ac.uk/empiar/

Total amount of DataCap being requested

15PiB

Expected size of single dataset (one copy)

1P

Number of replicas to store

10

Weekly allocation of DataCap requested

1PiB

On-chain address for first allocation

f1dwes3ykaliqbvfgp6x4a22uhiepy24ruti3ohli

Data Type of Application

Public, Open Dataset (Research/Non-Profit)

Custom multisig

Identifier

No response

Share a brief history of your project and organization

EMPIAR, the Electron Microscopy Public Image Archive, is a public resource for raw images underpinning 3D cryo-EM maps and tomograms (themselves archived in EMDB). EMPIAR also accommodates 3D datasets obtained with volume EM techniques and soft and hard X-ray tomography. More ...
As of 2023-08-15, EMPIAR contains 1391 entries, taking up 3.15 PB of storage.

Is this project associated with other projects/ecosystem stakeholders?

No

If answered yes, what are the other projects/ecosystem stakeholders

No response

Describe the data being stored onto Filecoin

Founded in 1956, NRAO provides the most advanced radio telescope facilities and information to the international scientific community.

Where was the data currently stored in this dataset sourced from

AWS Cloud

If you answered "Other" in the previous question, enter the details here

No response

If you are a data preparer. What is your location (City and Country)

china

If you are a data preparer, how will the data be prepared? Please include tooling used and technical details?

No response

If you are not preparing the data, who will prepare the data? (Provide name and business)

No response

Has this dataset been stored on the Filecoin network before? If so, please explain and make the case why you would like to store this dataset again to the network. Provide details on preparation and/or SP distribution.

No response

Please share a sample of the data

https://www.ebi.ac.uk/empiar/

Confirm that this is a public dataset that can be retrieved by anyone on the Network

If you chose not to confirm, what was the reason

No response

What is the expected retrieval frequency for this data

Monthly

For how long do you plan to keep this dataset stored on Filecoin

1.5 to 2 years

In which geographies do you plan on making storage deals

Greater China, Asia other than Greater China, North America, Europe

How will you be distributing your data to storage providers

Cloud storage (i.e. S3), HTTP or FTP server, IPFS, Shipping hard drives, Lotus built-in data transfer

How do you plan to choose storage providers

Slack, Filmine

If you answered "Others" in the previous question, what is the tool or platform you plan to use

No response

If you already have a list of storage providers to work with, fill out their names and provider IDs below

No response

How do you plan to make deals to your storage providers

Boost client, Lotus client, Singularity

If you answered "Others/custom tool" in the previous question, enter the details here

No response

Can you confirm that you will follow the Fil+ guideline

Yes

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 6

Multisig Notary address

f02049625

Client address

f1dwes3ykaliqbvfgp6x4a22uhiepy24ruti3ohli

DataCap allocation requested

2PiB

Id

7d959730-914f-4966-8aff-fcb8c35b03d3

TOPPOOL-LEE commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

⚠️ 33.39% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

AlanGreaterheat commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

⚠️ 34.19% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

AlanGreaterheat commented 1 year ago

Please note that Deal Data Replication!

AlanGreaterheat commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacecfpxqz6e4o6xcmwg22ncslburw4vhocifgp5mmcimmf4cefkok24

Address

f1dwes3ykaliqbvfgp6x4a22uhiepy24ruti3ohli

Datacap Allocated

2.00PiB

Signer Address

f1pnmzlxj7cfeo2v6oj5nco46hkg2l46wj7o4xxui

Id

7d959730-914f-4966-8aff-fcb8c35b03d3

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacecfpxqz6e4o6xcmwg22ncslburw4vhocifgp5mmcimmf4cefkok24

METAVERSEDATAMINING commented 1 year ago

Retrieval is good.Hope to see improvements regarding the issue of duplicate data in the next round.

METAVERSEDATAMINING commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaceadimiqkupzrwirk3ay5cgi7gfysvkqm6dt6l7wrodpyrqditjgt2

Address

f1dwes3ykaliqbvfgp6x4a22uhiepy24ruti3ohli

Datacap Allocated

2.00PiB

Signer Address

f17idrnfnxl2mbgcgr57a6z2c6lj2qx56gvm3336i

Id

7d959730-914f-4966-8aff-fcb8c35b03d3

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceadimiqkupzrwirk3ay5cgi7gfysvkqm6dt6l7wrodpyrqditjgt2

github-actions[bot] commented 11 months ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

TOPPOOL-LEE commented 11 months ago

keep open

herrehesse commented 11 months ago

checker:manualTrigger

filplus-checker-app[bot] commented 11 months ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

⚠️ 36.75% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard.

large-datacap-requests[bot] commented 11 months ago

DataCap Allocation requested

Request number 7

Multisig Notary address

f02049625

Client address

f1dwes3ykaliqbvfgp6x4a22uhiepy24ruti3ohli

DataCap allocation requested

2PiB

Id

cc39a231-b1f8-4893-ba35-cfa6f6d69101

mikezli commented 11 months ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzaced4m4rhtdulzhmwu4o23hayfrd2mwth5x2tajqpvcxmrvj3gyp5yc

Address

f1dwes3ykaliqbvfgp6x4a22uhiepy24ruti3ohli

Datacap Allocated

2.00PiB

Signer Address

f1dnb3uz7sylxk6emti3ififcvu3nlufnnsjui6ea

Id

cc39a231-b1f8-4893-ba35-cfa6f6d69101

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaced4m4rhtdulzhmwu4o23hayfrd2mwth5x2tajqpvcxmrvj3gyp5yc

nj-steve commented 11 months ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaced46u2fdqolavfrefceyzkzkinmpr7dalcyj7uafxoovtwogbd3ng

Address

f1dwes3ykaliqbvfgp6x4a22uhiepy24ruti3ohli

Datacap Allocated

2.00PiB

Signer Address

f1xx6555qijma7igpnjspyvdunc4vfxkawnpqy5ii

Id

cc39a231-b1f8-4893-ba35-cfa6f6d69101

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaced46u2fdqolavfrefceyzkzkinmpr7dalcyj7uafxoovtwogbd3ng

nj-steve commented 11 months ago

The reports look nice. The retrieval rate is 43.02%.Willing to support. 1698304200094

WechatIMG2014
kevzak commented 11 months ago

SPs taking deals

f02002141 | Seoul, Seoul, KRAmazon.com, Inc. | 329.94 TiB | 5.59% | 329.94 TiB | 0.00% f02005421 | Seoul, Seoul, KRAmazon.com, Inc. | 329.94 TiB | 5.59% | 329.94 TiB | 0.00% f02229545 | Los Angeles, California, USCNSERVERS LLC | 328.13 TiB | 5.56% | 328.13 TiB | 0.00% f02252024 | New York City, New York, USCologix, Inc | 230.00 TiB | 3.90% | 230.00 TiB | 0.00% f02252111 | Kuala Lumpur, Kuala Lumpur, MYExtreme Broadband - Total Broadband Experience | 625.69 TiB | 10.61% | 625.69 TiB | 0.00% f01110088new | Hong Kong, Central and Western, HKGigabitbank Global | 1.35 PiB | 23.50% | 1.35 PiB | 0.00% f02252023 | Hong Kong, Central and Western, HKHGC Global Communications Limited | 230.00 TiB | 3.90% | 230.00 TiB | 0.00% f02809374 | Tokyo, Tokyo, JPIPTELECOM Global | 579.97 TiB | 9.83% | 579.97 TiB | 0.00% f02809382 | Singapore, Singapore, SGIPTELECOM Global | 579.81 TiB | 9.83% | 579.81 TiB | 0.00% f02230309 | Putra Heights, Selangor, MYTechAvenue Malaysia | 329.38 TiB | 5.58% | 329.38 TiB | 0.00% f01422327 | Yokohama, Kanagawa, JPTOKAI Communications Corporation | 323.38 TiB | 5.48% | 323.38 TiB | 0.00% f02252097 | Hanoi, Hanoi, VNVNPT Corp | 625.69 TiB | 10.61% | 625.69 TiB | 0.00%

SPs listed upfront:

265682647-e5703fc3-3102-436b-95d3-2c04ac6dda5e
kevzak commented 11 months ago

Please confirm all SP enities @liou38469 many are not listed

TOPPOOL-LEE commented 11 months ago

Dear@kevzak , as you can see, we listed 6 SPs, and then we cooperated with these 6 SPs, and added 6 new SPs. This is because we previously promised to find 10 SPs for backup, so when our original 6 SPs stopped cooperating, we contacted the new 6 SPs, and we worked hard to make the data backup look better.

kevzak commented 11 months ago

All SPs need to be shared. Please provide information about entity and location for the 6 new SPs. Thanks

TOPPOOL-LEE commented 11 months ago

Hey, @kevzak we have submitted all the SPs we cooperate with according to your requirements. Thank you.

WX20231030-181617@2x
github-actions[bot] commented 11 months ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

TOPPOOL-LEE commented 11 months ago

keep it open

NewHuoPool commented 11 months ago

checker:manualTrigger

filplus-checker-app[bot] commented 11 months ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

⚠️ 49.23% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard.

zcfil commented 11 months ago

checker:manualTrigger

filplus-checker-app[bot] commented 11 months ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

⚠️ 49.23% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard.

zcfil commented 11 months ago

Is there an improved program for this problem? ⚠️ 49.23% of deals are for data replicated across less than 4 storage providers.

TOPPOOL-LEE commented 11 months ago

Thank you for your question. Due to the different speed of hard disk mailing, SP downloading data and encapsulation speed, the data backup is not synchronized. Next, we will pay more attention to the storage progress of each SP. Thank you.

NewHuoPool commented 11 months ago

Everything looks fine.

NewHuoPool commented 11 months ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacebcnpgnobrughp6lulmxkerery3p54t6ffk56sxpsguhdlysguere

Address

f1dwes3ykaliqbvfgp6x4a22uhiepy24ruti3ohli

Datacap Allocated

2.00PiB

Signer Address

f16karfxq7lxdy7izqrzrk75jf3not34k6sg6zvcy

Id

cc39a231-b1f8-4893-ba35-cfa6f6d69101

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebcnpgnobrughp6lulmxkerery3p54t6ffk56sxpsguhdlysguere

zcfil commented 11 months ago

Thank you for your question. Due to the different speed of hard disk mailing, SP downloading data and encapsulation speed, the data backup is not synchronized. Next, we will pay more attention to the storage progress of each SP. Thank you.

Ok, hopefully adjustments will be made soon, will follow up on the bot report!

zcfil commented 11 months ago

Viewed historical information, willing to support this round, note data distribution

zcfil commented 11 months ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacea6nvknlqrupk7mkk5xs3zzpgoo67e7upwecec2z7wuwxdvq5jzlk

Address

f1dwes3ykaliqbvfgp6x4a22uhiepy24ruti3ohli

Datacap Allocated

2.00PiB

Signer Address

f1cjzbiy5xd4ehera4wmbz63pd5ku4oo7g52cldga

Id

cc39a231-b1f8-4893-ba35-cfa6f6d69101

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacea6nvknlqrupk7mkk5xs3zzpgoo67e7upwecec2z7wuwxdvq5jzlk

large-datacap-requests[bot] commented 11 months ago

DataCap Allocation requested

Request number 8

Multisig Notary address

f02049625

Client address

f1dwes3ykaliqbvfgp6x4a22uhiepy24ruti3ohli

DataCap allocation requested

2PiB

Id

5c19fd79-a407-4ab6-af7b-2e6d8a61b874

SuperChaiChai commented 10 months ago

checker:manualTrigger

filplus-checker-app[bot] commented 10 months ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

⚠️ 49.23% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard.

SuperChaiChai commented 10 months ago

Further encapsulation requires increasing the number of copies

SuperChaiChai commented 10 months ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzaceb5rpfunahld6k4vdx4it77sl6gfv2hhrheos3f366mgqcofba4di

Address

f1dwes3ykaliqbvfgp6x4a22uhiepy24ruti3ohli

Datacap Allocated

2.00PiB

Signer Address

f12mckci3omexgzoeosjvstcfxfe4vqw7owdia3da

Id

5c19fd79-a407-4ab6-af7b-2e6d8a61b874

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceb5rpfunahld6k4vdx4it77sl6gfv2hhrheos3f366mgqcofba4di

a1991car commented 10 months ago

Except for the data backup issue, everything else looks good and I hope to see improvements.

a1991car commented 10 months ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacedilexqj3ohocxvtmdhegqqe46bt7ysv4cszvhq5vyttsqwfdwvfq

Address

f1dwes3ykaliqbvfgp6x4a22uhiepy24ruti3ohli

Datacap Allocated

2.00PiB

Signer Address

f1qnumecdypgrbaebtkdfjnwt5ndacadcuas3deiq

Id

5c19fd79-a407-4ab6-af7b-2e6d8a61b874

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedilexqj3ohocxvtmdhegqqe46bt7ysv4cszvhq5vyttsqwfdwvfq

Sunnyiscoming commented 10 months ago

Some sps outside the form participated in @liou38469 Can you explain about that?

TOPPOOL-LEE commented 10 months ago

Sorry for the late reply. We have cooperated with all the SPs we listed. We have added new SPs and we have listed the company, location, and node number. Please check previous reply records, thank you.

large-datacap-requests[bot] commented 10 months ago

DataCap Allocation requested

Request number 10

Multisig Notary address

f02049625

Client address

f1dwes3ykaliqbvfgp6x4a22uhiepy24ruti3ohli

DataCap allocation requested

1PiB

Id

a30ab29e-a496-4c88-9272-47b5c11e67a6

TOPPOOL-LEE commented 10 months ago

checker:manualTrigger

filplus-checker-app[bot] commented 10 months ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

⚠️ 49.66% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard.

filplus-checker-app[bot] commented 10 months ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

⚠️ 49.66% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard.

sxxfuture-official commented 10 months ago

Need to increase the number of data replicas, everything else seems to be fine