search

filecoin-project / filecoin-plus-large-datasets

Hub for client applications for DataCap at a large scale
110 stars 62 forks source link

[DataCap Application] <Venus Team > - <DealAccelerator 3> #1725

Closed Joss-Hua closed 8 months ago

Joss-Hua commented 1 year ago

Data Owner Name

Venus team

Data Owner Country/Region

American Samoa

Data Owner Industry

Other

Website

https://venushub.io

Social Media

https://linktr.ee/filecoinvenus

Total amount of DataCap being requested

5PiB

Weekly allocation of DataCap requested

200TiB

On-chain address for first allocation

f1fppo2nhn3zd2zfpa6fdqwuqwlho3nerbkrenquq

Custom multisig

Identifier

No response

Share a brief history of your project and organization

The team: Venus team

Venus team leads the development and practice of Filecoin Venus, established in 2020, Shanghai, China. Now we have decades of members from all over the world focusing on code dev and community.

Through the core dev and ecological activities and programs, we hope (and already) to have more storage service providers, users, and enthusiasts join Filecoin or provide more contributions after joining Filecoin.

The project: Venus Deal Accelerator (https://venushub.io/accelerator/)

Venus is committed to offer a fully functional deal-making experience for both storage clients and storage providers on the scale. As the Filecoin network grows and the community strives towards a more storage deal weighted growth than committed capacity growth, the Venus community takes on the challenge to help shape this vision with the Venus Deal Accelerator program.

The goal of the Venus Deal Accelerator program is to distribute as much storage deals as it can to the broader storage provider community with focuses on seamlessly bridging the sealing experience that storage providers are already familiar with to the Filecoin deal taking experience. Venus Deal Accelerator program will be responsible for applying large datacap with approved open datasets from fil-plus program and distribute storage deals to its participants running Venus storage systems.

Is this project associated with other projects/ecosystem stakeholders?

No

If answered yes, what are the other projects/ecosystem stakeholders

No response

Describe the data being stored onto Filecoin

The data stored in Filecoin is from publicly available datasets in various machine learning.

Where was the data currently stored in this dataset sourced from

AWS Cloud

If you answered "Other" in the previous question, enter the details here

No response

How do you plan to prepare the dataset

graphsplit

If you answered "other/custom tool" in the previous question, enter the details here

No response

Please share a sample of the data

https://registry.opendata.aws/nasa-gcms/
s3://gcms-samdata-mlchallenge/
https://registry.opendata.aws/comonscreens/
s3://common-screens/
https://registry.opendata.aws/allenai-tqa/
3://ai2-public-datasets/
https://registry.opendata.aws/multi-token-completion/
s3://multi-token-completion/

Confirm that this is a public dataset that can be retrieved by anyone on the Network

If you chose not to confirm, what was the reason

No response

What is the expected retrieval frequency for this data

Monthly

For how long do you plan to keep this dataset stored on Filecoin

1 to 1.5 years

In which geographies do you plan on making storage deals

Greater China, Asia other than Greater China, Africa, North America, Europe

How will you be distributing your data to storage providers

Cloud storage (i.e. S3), HTTP or FTP server, Shipping hard drives, Others

How do you plan to choose storage providers

Slack, Filmine, Big data exchange, Others

If you answered "Others" in the previous question, what is the tool or platform you plan to use

We define VenusHub as a platform for community projects. The Deal Accelerator mentioned here is one of them. The Deal Accelerator is aimed at the storage providers of real data storage. We complete the SP screening through they' application and screening rules, and they come from the Filecoin community, so they have no interest relationship.

If you already have a list of storage providers to work with, fill out their names and provider IDs below

This list is a part of it. We are still expanding the available real data storage providers.
https://github.com/data-preservation-programs/filplus-checker-assets/tree/main/filecoin-project/filecoin-plus-large-datasets/issues/345
https://github.com/data-preservation-programs/filplus-checker-assets/tree/main/filecoin-project/filecoin-plus-large-datasets/issues/1444
We will focus on more new SPs (miners) and hope that more miners will real data provide storage for the client.

How do you plan to make deals to your storage providers

Others/custom tool

If you answered "Others/custom tool" in the previous question, enter the details here

venus-market (Droplet). The support of venus-market(Droplet) for real data storage is very mature.

Can you confirm that you will follow the Fil+ guideline

Yes

large-datacap-requests[bot] commented 1 year ago

Thanks for your request!

Heads up, you’re requesting more than the typical weekly onboarding rate of DataCap!
large-datacap-requests[bot] commented 1 year ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

Sunnyiscoming commented 1 year ago

Datacap Request Trigger

Total DataCap requested

5PiB

Expected weekly DataCap usage rate

200TiB

Client address

f1fppo2nhn3zd2zfpa6fdqwuqwlho3nerbkrenquq

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Multisig Notary address

f02049625

Client address

f1fppo2nhn3zd2zfpa6fdqwuqwlho3nerbkrenquq

DataCap allocation requested

100TiB

Id

a85399b4-3db7-45f1-a4e4-7536d6f0f4c6

newwebgroup commented 1 year ago

Tim Venus memiliki banyak pengalaman Fil+ sebelumnya dan sejarah kinerja yang baik dan, karena ini adalah putaran pertama, Bersedia mendukung mereka di babak ini.

newwebgroup commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacea4k6hr7h6fdb3duanunbe6jo7rfbuzffxtfsw52wpljwm4sijf3e

Address

f1fppo2nhn3zd2zfpa6fdqwuqwlho3nerbkrenquq

Datacap Allocated

100.00TiB

Signer Address

f1e77zuityhvvw6u2t6tb5qlnsegy2s67qs4lbbbq

Id

a85399b4-3db7-45f1-a4e4-7536d6f0f4c6

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacea4k6hr7h6fdb3duanunbe6jo7rfbuzffxtfsw52wpljwm4sijf3e

sxxfuture-official commented 1 year ago

The Venus team is a trustworthy team. After checking the disclosed information, it is a public data set, and the volume of the data meets the requirements. I will support this round.

sxxfuture-official commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacedw3zdlwb6fe3osvz2qnrepxutrpbt3ny6vs5i2tdcraqx5pcmfmg

Address

f1fppo2nhn3zd2zfpa6fdqwuqwlho3nerbkrenquq

Datacap Allocated

100.00TiB

Signer Address

f1foiomqlmoshpuxm6aie4xysffqezkjnokgwcecq

Id

a85399b4-3db7-45f1-a4e4-7536d6f0f4c6

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedw3zdlwb6fe3osvz2qnrepxutrpbt3ny6vs5i2tdcraqx5pcmfmg

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 2

Multisig Notary address

f02049625

Client address

f1fppo2nhn3zd2zfpa6fdqwuqwlho3nerbkrenquq

DataCap allocation requested

200TiB

Id

2d6df82d-7d41-4f43-aa9a-570cdcfc588b

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f02049625

Client address

f1fppo2nhn3zd2zfpa6fdqwuqwlho3nerbkrenquq

Rule to calculate the allocation request amount

100% of weekly dc amount requested

DataCap allocation requested

200TiB

Total DataCap granted for client so far

100TiB

Datacap to be granted to reach the total amount requested by the client (5PiB)

4.90PiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
1719 4 100TiB 34.03 27.75TiB
Joss-Hua commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

⚠️ 2 storage providers have unknown IP location - f01874063, f01874059

Deal Data Replication

⚠️ 100.00% of deals are for data replicated across less than 3 storage providers.

Deal Data Shared with other Clients[^3]

⚠️ CID sharing has been observed. (Top 3)

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the full report.

Joss-Hua commented 1 year ago

Here is a supplementary explanation:

If you have any further questions, please feel free to contact me at Slack @Joss-Venus

DaYouGroup commented 1 year ago

How to solve the situation of CID duplication and excessive proportion?

Joss-Hua commented 1 year ago

‘The high proportion’ is mainly due to the fact that the first round has just ended.

I am not planning to delete the stored CIDs here, as these LDNs are used for VDA projects and were planned to be the same batch of SPs (possibly 20, or 30, or more). If 'deletion' is only to adjust the results of the report and does not change the facts, but brings unnecessary costs to SPs, it is meaningless.

By the way, is it possible to merge multiple LDNs now, just like the reason for this proposal

DaYouGroup commented 1 year ago

Please strengthen the number of copies and increase sp diversity as soon as possible.

DaYouGroup commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacebgzxjkezquwj2zfgvfjddpwyboem6iiaclsgm52kx4bh7crlpe5c

Address

f1fppo2nhn3zd2zfpa6fdqwuqwlho3nerbkrenquq

Datacap Allocated

200.00TiB

Signer Address

f1nwjsd2mc6hu4qrwnmd6ukrfkuu4h5fhs7u3exii

Id

2d6df82d-7d41-4f43-aa9a-570cdcfc588b

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebgzxjkezquwj2zfgvfjddpwyboem6iiaclsgm52kx4bh7crlpe5c

laurarenpanda commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacealslf3nc6mvp5ru3r3wkpo6uoyn6uvdnbnynolz3lxaud3moblgk

Address

f1fppo2nhn3zd2zfpa6fdqwuqwlho3nerbkrenquq

Datacap Allocated

200.00TiB

Signer Address

f1bp3tzp536edm7dodldceekzbsx7zcy7hdfg6uzq

Id

2d6df82d-7d41-4f43-aa9a-570cdcfc588b

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacealslf3nc6mvp5ru3r3wkpo6uoyn6uvdnbnynolz3lxaud3moblgk

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 3

Multisig Notary address

f02049625

Client address

f1fppo2nhn3zd2zfpa6fdqwuqwlho3nerbkrenquq

DataCap allocation requested

400TiB

Id

82ddefe2-70c0-4926-b9c1-29582d98a17c

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f02049625

Client address

f1fppo2nhn3zd2zfpa6fdqwuqwlho3nerbkrenquq

Rule to calculate the allocation request amount

200% of weekly dc amount requested

DataCap allocation requested

400TiB

Total DataCap granted for client so far

181898.9YiB

Datacap to be granted to reach the total amount requested by the client (5PiB)

-2.19B

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
3673 7 200TiB 30.4 143.29TiB
Joss-Hua commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

⚠️ 2 storage providers have unknown IP location - f01874063, f01874059

Deal Data Replication

⚠️ 100.00% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

⚠️ CID sharing has been observed. (Top 3)

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the full report.

kernelogic commented 1 year ago

Long term reputable client and CID report looks good.

kernelogic commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacea43ncg62z3vl4z4jzgzjuwiu6fdatygi5hgtwyvxo26lfr44jdzc

Address

f1fppo2nhn3zd2zfpa6fdqwuqwlho3nerbkrenquq

Datacap Allocated

400.00TiB

Signer Address

f1yjhnsoga2ccnepb7t3p3ov5fzom3syhsuinxexa

Id

82ddefe2-70c0-4926-b9c1-29582d98a17c

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacea43ncg62z3vl4z4jzgzjuwiu6fdatygi5hgtwyvxo26lfr44jdzc

newwebgroup commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaceableyqnnqy4apw6ri2q777w4f2svni3w7spydzm5hwr5msofyryy

Address

f1fppo2nhn3zd2zfpa6fdqwuqwlho3nerbkrenquq

Datacap Allocated

400.00TiB

Signer Address

f1e77zuityhvvw6u2t6tb5qlnsegy2s67qs4lbbbq

Id

82ddefe2-70c0-4926-b9c1-29582d98a17c

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceableyqnnqy4apw6ri2q777w4f2svni3w7spydzm5hwr5msofyryy

Joss-Hua commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

⚠️ All retrieval success ratios are below 1%.

Storage Provider Distribution

⚠️ 2 storage providers have unknown IP location - f01874063, f01874059

Deal Data Replication

⚠️ 100.00% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

⚠️ CID sharing has been observed. (Top 3)

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval report.

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 4

Multisig Notary address

f02049625

Client address

f1fppo2nhn3zd2zfpa6fdqwuqwlho3nerbkrenquq

DataCap allocation requested

800TiB

Id

ad7ae559-eba7-4ef2-b405-f29bdcb85762

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f02049625

Client address

f1fppo2nhn3zd2zfpa6fdqwuqwlho3nerbkrenquq

Rule to calculate the allocation request amount

400% of weekly dc amount requested

DataCap allocation requested

800TiB

Total DataCap granted for client so far

363797880709171445760.0YiB

Datacap to be granted to reach the total amount requested by the client (5PiB)

363797880709171445760.0YiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
16697 10 400TiB 16.06 108.92TiB
psh0691 commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

⚠️ All retrieval success ratios are below 1%.

Storage Provider Distribution

⚠️ 2 storage providers have unknown IP location - f01874063, f01874059

Deal Data Replication

⚠️ 100.00% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

⚠️ CID sharing has been observed. (Top 3)

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval report.

Joss-Hua commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

⚠️ All retrieval success ratios are below 1%.

Storage Provider Distribution

⚠️ 2 storage providers have unknown IP location - f01874063, f01874059

Deal Data Replication

⚠️ 100.00% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

⚠️ CID sharing has been observed. (Top 3)

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval report.

Joss-Hua commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

⚠️ All retrieval success ratios are below 1%.

Storage Provider Distribution

⚠️ 2 storage providers have unknown IP location - f01874063, f01874059

Deal Data Replication

⚠️ 100.00% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

⚠️ CID sharing has been observed. (Top 3)

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval report.

Joss-Hua commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

Storage Provider Distribution

⚠️ 2 storage providers have unknown IP location - f01874063, f01874059

Deal Data Replication

⚠️ 100.00% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

⚠️ CID sharing has been observed. (Top 3)

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval report.

Joss-Hua commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

Storage Provider Distribution

⚠️ 2 storage providers have unknown IP location - f01874063, f01874059

Deal Data Replication

⚠️ 100.00% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

⚠️ CID sharing has been observed. (Top 3)

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval report.

DaYouGroup commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

Storage Provider Distribution

⚠️ 2 storage providers have unknown IP location - f01874063, f01874059

Deal Data Replication

⚠️ 100.00% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

⚠️ CID sharing has been observed. (Top 3)

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval report.

DaYouGroup commented 1 year ago

HTTP retrieval results are well optimized. Trust Venus as a reputable member of the community. Willing to support this round.

DaYouGroup commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacedyo6x6ozuz2djrbga2o5u57e2br3gm22aeymu2kkqfce3dh3sl72

Address

f1fppo2nhn3zd2zfpa6fdqwuqwlho3nerbkrenquq

Datacap Allocated

800.00TiB

Signer Address

f1nwjsd2mc6hu4qrwnmd6ukrfkuu4h5fhs7u3exii

Id

ad7ae559-eba7-4ef2-b405-f29bdcb85762

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedyo6x6ozuz2djrbga2o5u57e2br3gm22aeymu2kkqfce3dh3sl72

Joss-Hua commented 1 year ago

Thank @DaYouGroup so much

Tom-OriginStorage commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

Storage Provider Distribution

⚠️ 2 storage providers have unknown IP location - f01874063, f01874059

Deal Data Replication

⚠️ 100.00% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

⚠️ CID sharing has been observed. (Top 3)

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval report.

Tom-OriginStorage commented 1 year ago

The HTTP retrieval rate has been optimized, and we have communicated with Joss that CID is shared as the same type of project. We are willing to support this round, but it seems that the number of SPs is problematic for robots

Tom-OriginStorage commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacebsprvlxz7y4hsu6eb3l6zncg3tv6v52p7cv6jb7tll6coqbs7zji

Address

f1fppo2nhn3zd2zfpa6fdqwuqwlho3nerbkrenquq

Datacap Allocated

800.00TiB

Signer Address

f1q6bpjlqia6iemqbrdaxr2uehrhpvoju3qh4lpga

Id

ad7ae559-eba7-4ef2-b405-f29bdcb85762

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebsprvlxz7y4hsu6eb3l6zncg3tv6v52p7cv6jb7tll6coqbs7zji

Joss-Hua commented 1 year ago

thank @Tom-OriginStorage , due to the fact that all SPs come from the community, the expansion speed is slow, but we will increase the number of SPs in the long term.

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 5

Multisig Notary address

f02049625

Client address

f1fppo2nhn3zd2zfpa6fdqwuqwlho3nerbkrenquq

DataCap allocation requested

800TiB

Id

110d9612-c8ff-46e2-ad18-841189e3b01c