search

filecoin-project / filecoin-plus-large-datasets

Hub for client applications for DataCap at a large scale
109 stars 62 forks source link

[DataCap Application] <Venus Team > - <DealAccelerator 5> #1727

Open Joss-Hua opened 1 year ago

Joss-Hua commented 1 year ago

Data Owner Name

Venus team

Data Owner Country/Region

Hong Kong

Data Owner Industry

Environment

Website

https://venushub.io

Social Media

https://linktr.ee/filecoinvenus

Total amount of DataCap being requested

5PiB

Weekly allocation of DataCap requested

200TiB

On-chain address for first allocation

f1rge7ye6y2vvrytjmvr3po7co52wa3oitrucy5sq

Custom multisig

Identifier

No response

Share a brief history of your project and organization

The team: Venus team

Venus team leads the development and practice of Filecoin Venus, established in 2020, Shanghai, China. Now we have decades of members from all over the world focusing on code dev and community.

Through the core dev and ecological activities and programs, we hope (and already) to have more storage service providers, users, and enthusiasts join Filecoin or provide more contributions after joining Filecoin.

The project: Venus Deal Accelerator (https://venushub.io/accelerator/)

Venus is committed to offer a fully functional deal-making experience for both storage clients and storage providers on the scale. As the Filecoin network grows and the community strives towards a more storage deal weighted growth than committed capacity growth, the Venus community takes on the challenge to help shape this vision with the Venus Deal Accelerator program.

The goal of the Venus Deal Accelerator program is to distribute as much storage deals as it can to the broader storage provider community with focuses on seamlessly bridging the sealing experience that storage providers are already familiar with to the Filecoin deal taking experience. Venus Deal Accelerator program will be responsible for applying large datacap with approved open datasets from fil-plus program and distribute storage deals to its participants running Venus storage systems.

Is this project associated with other projects/ecosystem stakeholders?

No

If answered yes, what are the other projects/ecosystem stakeholders

No response

Describe the data being stored onto Filecoin

The data stored in Filecoin is from publicly available datasets with important information for events or living environments.

Where was the data currently stored in this dataset sourced from

AWS Cloud

If you answered "Other" in the previous question, enter the details here

No response

How do you plan to prepare the dataset

graphsplit

If you answered "other/custom tool" in the previous question, enter the details here

No response

Please share a sample of the data

https://registry.opendata.aws/emearth/
s3://emearth/
https://registry.opendata.aws/epa-rsei-pds/
s3://epa-rsei-pds/
https://registry.opendata.aws/gnss-ro-opendata/
s3://gnss-ro-data/
https://registry.opendata.aws/gdelt/
s3://gdelt-open-data/

Confirm that this is a public dataset that can be retrieved by anyone on the Network

If you chose not to confirm, what was the reason

No response

What is the expected retrieval frequency for this data

Monthly

For how long do you plan to keep this dataset stored on Filecoin

1 to 1.5 years

In which geographies do you plan on making storage deals

Greater China, Asia other than Greater China, North America, Europe

How will you be distributing your data to storage providers

Cloud storage (i.e. S3), HTTP or FTP server, Shipping hard drives, Others

How do you plan to choose storage providers

Slack, Filmine, Big data exchange, Others

If you answered "Others" in the previous question, what is the tool or platform you plan to use

We define VenusHub as a platform for community projects. The Deal Accelerator mentioned here is one of them. The Deal Accelerator is aimed at the storage providers of real data storage. We complete the SP screening through they' application and screening rules, and they come from the Filecoin community, so they have no interest relationship.

If you already have a list of storage providers to work with, fill out their names and provider IDs below

This list is a part of it. We are still expanding the available real data storage providers.
https://github.com/data-preservation-programs/filplus-checker-assets/tree/main/filecoin-project/filecoin-plus-large-datasets/issues/345
https://github.com/data-preservation-programs/filplus-checker-assets/tree/main/filecoin-project/filecoin-plus-large-datasets/issues/1444
We will focus on more new SPs (miners) and hope that more miners will real data provide storage for the client.

How do you plan to make deals to your storage providers

Others/custom tool

If you answered "Others/custom tool" in the previous question, enter the details here

venus-market (Droplet). The support of venus-market(Droplet) for real data storage is very mature.

Can you confirm that you will follow the Fil+ guideline

Yes

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

Joss-Hua commented 1 year ago

keep going

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

Joss-Hua commented 1 year ago

keep going

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 4

Multisig Notary address

f02049625

Client address

f1rge7ye6y2vvrytjmvr3po7co52wa3oitrucy5sq

DataCap allocation requested

800TiB

Id

cd88ed83-f05c-4ca0-81bd-2583ad738868

1ane-1 commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

Storage Provider Distribution

⚠️ 1 storage providers sealed more than 30% of total datacap - f01114587: 39.03%

⚠️ 1 storage providers have unknown IP location - f02365245

Deal Data Replication

⚠️ 100.00% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

⚠️ CID sharing has been observed. (Top 3)

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

Joss-Hua commented 1 year ago
1ane-1 commented 1 year ago
  • Unknown IP: f02365245 is a new participant and sp of VDA, currently only encapsulating 1T deals for testing purposes. I will contact this SP to support retrieval before continuing sealing;
  • Sealed more than 30%: currently in the early stages of allocation and will be improved in the next round;
  • Other issues: have been explained multiple times under the VDA series LDN;

Just support for current stage and will keep eye on your next stage report. And pls add more SPs in the next round. It will be more distributed.

1ane-1 commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacebh5xcm56ci3fbuhizt6qvuyei3oxm57t7yek4cthbeoyszgkiz56

Address

f1rge7ye6y2vvrytjmvr3po7co52wa3oitrucy5sq

Datacap Allocated

800.00TiB

Signer Address

f1mdk7s2vntzm6hu35yuo6vjubtrpfnb2awhgvrri

Id

cd88ed83-f05c-4ca0-81bd-2583ad738868

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebh5xcm56ci3fbuhizt6qvuyei3oxm57t7yek4cthbeoyszgkiz56

Joss-Hua commented 1 year ago
  • Unknown IP: f02365245 is a new participant and sp of VDA, currently only encapsulating 1T deals for testing purposes. I will contact this SP to support retrieval before continuing sealing;
  • Sealed more than 30%: currently in the early stages of allocation and will be improved in the next round;
  • Other issues: have been explained multiple times under the VDA series LDN;

Just support for current stage and will keep eye on your next stage report. And pls add more SPs in the next round. It will be more distributed.

thanks

kernelogic commented 1 year ago

The OP's explanation sounds good to me. We all know how merged dataset works.

kernelogic commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaced3or3azaccyycylungfno5lpoibov3m4mxeozwwlr3dwv7x5np2o

Address

f1rge7ye6y2vvrytjmvr3po7co52wa3oitrucy5sq

Datacap Allocated

800.00TiB

Signer Address

f1yjhnsoga2ccnepb7t3p3ov5fzom3syhsuinxexa

Id

cd88ed83-f05c-4ca0-81bd-2583ad738868

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaced3or3azaccyycylungfno5lpoibov3m4mxeozwwlr3dwv7x5np2o

kevzak commented 1 year ago

need to see some deal distribution here @Joss-Hua , one SP storing 40%

Sunnyiscoming commented 1 year ago

Hello, @Joss-Hua per the https://github.com/filecoin-project/notary-governance/issues/922 for Open, Public Dataset applicants, please complete the following Fil+ registration form to identify yourself as the applicant and also please add the contact information of the SP entities you are working with to store copies of the data.

This information will be reviewed by Fil+ Governance team to confirm validity and then the application will be allowed to move forward for additional notary review.

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 5

Multisig Notary address

f02049625

Client address

f1rge7ye6y2vvrytjmvr3po7co52wa3oitrucy5sq

DataCap allocation requested

800TiB

Id

7b11e3de-81e4-41a1-9cfb-8dc7c8ab7d61

Joss-Hua commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

⚠️ 1 storage providers sealed more than 30% of total datacap - f02813417: 33.81%

⚠️ 1 storage providers have unknown IP location - f02365245

Deal Data Replication

⚠️ 100.00% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

⚠️ CID sharing has been observed. (Top 3)

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard.

Joss-Hua commented 1 year ago

need to see some deal distribution here @Joss-Hua , one SP storing 40%

hi sir @kevzak , yes, we understand. There has been improvement and we will continue to see changes.

Joss-Hua commented 1 year ago

image

Sunnyiscoming commented 1 year ago

Yes. Received that.

SuperChaiChai commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacec6dwftooj46nt5zsgvxmlje3ncspwmq52cvejmm7kjgx4oquar3q

Address

f1rge7ye6y2vvrytjmvr3po7co52wa3oitrucy5sq

Datacap Allocated

800.00TiB

Signer Address

f12mckci3omexgzoeosjvstcfxfe4vqw7owdia3da

Id

7b11e3de-81e4-41a1-9cfb-8dc7c8ab7d61

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacec6dwftooj46nt5zsgvxmlje3ncspwmq52cvejmm7kjgx4oquar3q

herrehesse commented 12 months ago

checker:manualTrigger

filplus-checker-app[bot] commented 12 months ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

⚠️ 1 storage providers sealed more than 30% of total datacap - f02813417: 33.01%

⚠️ 1 storage providers have unknown IP location - f02365245

Deal Data Replication

⚠️ 100.00% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

⚠️ CID sharing has been observed. (Top 3)

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard.

herrehesse commented 12 months ago

Screenshot 2023-11-14 at 11 17 32

60% of all deals are to the same entity, self-dealing. Not OK. @Joss-Hua explain?

Joss-Hua commented 11 months ago

checker:manualTrigger

filplus-checker-app[bot] commented 11 months ago

DataCap and CID Checker Report Summary[^1]

Other Addresses[^2]

Storage Provider Distribution

⚠️ 1 storage providers sealed more than 30% of total datacap - f02813417: 32.12%

⚠️ 1 storage providers have unknown IP location - f02365245

Deal Data Replication

⚠️ 100.00% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

⚠️ CID sharing has been observed. (Top 3)

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard.

Joss-Hua commented 11 months ago

checker:manualTrigger

filplus-checker-app[bot] commented 11 months ago

DataCap and CID Checker Report Summary[^1]

Other Addresses[^2]

Storage Provider Distribution

⚠️ 1 storage providers sealed more than 30% of total datacap - f02813417: 32.12%

⚠️ 1 storage providers have unknown IP location - f02365245

Deal Data Replication

⚠️ 100.00% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

⚠️ CID sharing has been observed. (Top 3)

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard.

Joss-Hua commented 11 months ago

Screenshot 2023-11-14 at 11 17 32

60% of all deals are to the same entity, self-dealing. Not OK. @Joss-Hua explain?

Please Stop Imagining. That's all.

kernelogic commented 11 months ago

I looked across CID reports for the whole series and they are diverse. No reason to not support Venus.

kernelogic commented 11 months ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacebyisjihze5mqmd6ejo4zaa3qmd2jgvz2lmytksebo4jjfbzph6ts

Address

f1rge7ye6y2vvrytjmvr3po7co52wa3oitrucy5sq

Datacap Allocated

800.00TiB

Signer Address

f1yjhnsoga2ccnepb7t3p3ov5fzom3syhsuinxexa

Id

7b11e3de-81e4-41a1-9cfb-8dc7c8ab7d61

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebyisjihze5mqmd6ejo4zaa3qmd2jgvz2lmytksebo4jjfbzph6ts

Bitengine-reeta commented 11 months ago

https://github.com/filecoin-project/filecoin-plus-large-datasets/issues/1727#issuecomment-1815641349 Based on the above description, and Venus is a trustworthy organization in the community, I will support this round

Bitengine-reeta commented 11 months ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzaceb3nlac5go6oxaga2jb7bx32bfoslyugdrxqijso7gvgkqtk4rxrg

Address

f1rge7ye6y2vvrytjmvr3po7co52wa3oitrucy5sq

Datacap Allocated

800.00TiB

Signer Address

f1jyvhxp4kmwreo22ke4itspraznpudw3uqaink5i

Id

7b11e3de-81e4-41a1-9cfb-8dc7c8ab7d61

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceb3nlac5go6oxaga2jb7bx32bfoslyugdrxqijso7gvgkqtk4rxrg

laurarenpanda commented 11 months ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacecssvh4c4yvdtaeqfx7zps76kmdn5ls6dfw7h7awt4xczauh7d66u

Address

f1rge7ye6y2vvrytjmvr3po7co52wa3oitrucy5sq

Datacap Allocated

800.00TiB

Signer Address

f1bp3tzp536edm7dodldceekzbsx7zcy7hdfg6uzq

Id

7b11e3de-81e4-41a1-9cfb-8dc7c8ab7d61

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacecssvh4c4yvdtaeqfx7zps76kmdn5ls6dfw7h7awt4xczauh7d66u

github-actions[bot] commented 11 months ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

Joss-Hua commented 11 months ago

keep going

Sunnyiscoming commented 11 months ago

SP List provided: [{"providerID":"f02813417","City":"Beijing","Country":"CN","SPOrg","shiliu"}, {"providerID":"f020522","City":"Shenzhen","Country":"CN","SPOrg","wenchu"}, {"providerID":"f02104858","City":"HongKong","Country":"CN","SPOrg","Sanchuang"}, {"providerID":"f01968296","City":"Beijing","Country":"CN","SPOrg","YueKeYun"}, {"providerID":"f034548","City":"Texas","Country":"US","SPOrg","ByteByte"},]

github-actions[bot] commented 10 months ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

josshrb commented 10 months ago

keep going

github-actions[bot] commented 10 months ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

Joss-Hua commented 10 months ago

keep going

github-actions[bot] commented 9 months ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

Joss-Hua commented 9 months ago

keep going

a1991car commented 9 months ago

checker:manualTrigger

filplus-checker-app[bot] commented 9 months ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

⚠️ 6 storage providers have unknown IP location - f02013534, f02886944, f02887147, f02886969, f02931119, f02365245

Deal Data Replication

⚠️ 63.95% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

⚠️ CID sharing has been observed. (Top 3)

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard.

a1991car commented 9 months ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzaceaaeywscmescbh2gr6eaf2bwhhel3bf6qs6tulv2clhod3cfgbxdq

Address

f1rge7ye6y2vvrytjmvr3po7co52wa3oitrucy5sq

Datacap Allocated

800.00TiB

Signer Address

f1qnumecdypgrbaebtkdfjnwt5ndacadcuas3deiq

Id

7b11e3de-81e4-41a1-9cfb-8dc7c8ab7d61

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceaaeywscmescbh2gr6eaf2bwhhel3bf6qs6tulv2clhod3cfgbxdq

nj-steve commented 9 months ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacebipuruiqe7hx5nj2wcdc5bresgacpkvzbprijeojcxfxw6bco246

Address

f1rge7ye6y2vvrytjmvr3po7co52wa3oitrucy5sq

Datacap Allocated

800.00TiB

Signer Address

f1xx6555qijma7igpnjspyvdunc4vfxkawnpqy5ii

Id

7b11e3de-81e4-41a1-9cfb-8dc7c8ab7d61

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebipuruiqe7hx5nj2wcdc5bresgacpkvzbprijeojcxfxw6bco246

Tom-OriginStorage commented 9 months ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacebwa25poovrm47riezphgmdqx6vxydo45s4cduodwr4o5naptotqg

Address

f1rge7ye6y2vvrytjmvr3po7co52wa3oitrucy5sq

Datacap Allocated

800.00TiB

Signer Address

f1q6bpjlqia6iemqbrdaxr2uehrhpvoju3qh4lpga

Id

7b11e3de-81e4-41a1-9cfb-8dc7c8ab7d61

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebwa25poovrm47riezphgmdqx6vxydo45s4cduodwr4o5naptotqg

large-datacap-requests[bot] commented 9 months ago

DataCap Allocation requested

Request number 7

Multisig Notary address

f02049625

Client address

f1rge7ye6y2vvrytjmvr3po7co52wa3oitrucy5sq

DataCap allocation requested

800TiB

Id

719c9a87-5dc2-4b31-a61e-e970a8c659ff