filecoin-project / filecoin-plus-large-datasets

Hub for client applications for DataCap at a large scale
109 stars 62 forks source link

[DataCap Application] <Shenzhen Weite Network Culture Co., Ltd.> - <video archive> #1436

Closed hhhhxiao closed 11 months ago

hhhhxiao commented 1 year ago

Data Owner Name

Shenzhen Weite Network Culture Co., Ltd.

Data Owner Country/Region

China

Data Owner Industry

Information, Media & Telecommunications

Website

http://www.oktrust.com.cn/

Social Media

http://www.oktrust.com.cn/

Total amount of DataCap being requested

5PiB

Weekly allocation of DataCap requested

300TiB

On-chain address for first allocation

f16xlgzogkkd5qm44uftfprkvh44gu4ruo7dmngca

Custom multisig

Identifier

No response

Share a brief history of your project and organization

Shenzhen Weite Network Culture Co., Ltd. was established in 2017. We have been engaged in e-commerce, live broadcast, short video shooting, corporate promotional videos and other businesses.
We are a comprehensive advertising communication agency with creative images as the core and all-media "one-stop" communication as the channel. The integrated brand communication work we are engaged in covers creative film and television production, corporate brand building, new media interactive marketing, etc.
Video: As a professional commercial video production and dissemination agency, Witt Media always takes the maximization of brand communication as the basis of creation. It has industry-leading directors and production team resources. Foreign large and medium-sized enterprises have provided hundreds of excellent film and television works.
Brand: From communication strategy to advertisement creation and placement, Weite Media provides customers with one-stop full-service services. With accurate consumer insight, strategic thinking, creative expression and integrated communication capabilities, we will create more prominent business value and social influence for customers in the all-media era.
New Media: Weite Media focuses on the most advanced new media marketing model in the "Internet +" era, integrates interactive media, advertising creativity and innovative technology, and provides the most complete innovative marketing services in the industry with rigorous execution and a professional attitude . Connect customers, brands, products, users, and society, and create new brand experiences through communication in the interactive field.

Is this project associated with other projects/ecosystem stakeholders?

No

If answered yes, what are the other projects/ecosystem stakeholders

No response

Describe the data being stored onto Filecoin

We have a lot of live, e-commerce videos that need to be archived。

Where was the data currently stored in this dataset sourced from

My Own Storage Infra

If you answered "Other" in the previous question, enter the details here

No response

How do you plan to prepare the dataset

lotus

If you answered "other/custom tool" in the previous question, enter the details here

No response

Please share a sample of the data

https://drive.google.com/file/d/1XAqPnIuVMC8tisHbDmcZgVsnHltJfObl/view?usp=share_link
https://drive.google.com/file/d/15A9rGG0mUr3DTUD8TWg7nVuZErdRCGcv/view?usp=share_link
https://drive.google.com/file/d/1AW4BXp_uc0ZdhbKXYxzE_F3acesZaA_4/view?usp=share_link

Confirm that this is a public dataset that can be retrieved by anyone on the Network

If you chose not to confirm, what was the reason

No response

What is the expected retrieval frequency for this data

Yearly

For how long do you plan to keep this dataset stored on Filecoin

More than 3 years

In which geographies do you plan on making storage deals

Greater China

How will you be distributing your data to storage providers

Shipping hard drives

How do you plan to choose storage providers

Slack

If you answered "Others" in the previous question, what is the tool or platform you plan to use

No response

If you already have a list of storage providers to work with, fill out their names and provider IDs below

No response

How do you plan to make deals to your storage providers

Boost client, Lotus client

If you answered "Others/custom tool" in the previous question, enter the details here

No response

Can you confirm that you will follow the Fil+ guideline

Yes

zcfil commented 1 year ago

The report shows that the data distribution is relatively concentrated. What is the reason for this and how to avoid it in the future?

maxvint commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

⚠️ 1 storage providers sealed more than 30% of total datacap - f02121645: 42.82%

Deal Data Replication

⚠️ 100.00% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the full report.

hhhhxiao commented 1 year ago

The report shows that the data distribution is relatively concentrated. What is the reason for this and how to avoid it in the future?

Some SPs have faster encapsulation speed, and some are slower. However, the ratio has improved compared with the previous round of signatures. Next, we will optimize this to ensure that it meets the requirements.

zcfil commented 1 year ago

企业微信截图_16833645891375

zcfil commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacebk376wtgg3qzmaegsjkhj7ue3dcmyvq72myykeiermk2vodijglg

Address

f16xlgzogkkd5qm44uftfprkvh44gu4ruo7dmngca

Datacap Allocated

1.17PiB

Signer Address

f1cjzbiy5xd4ehera4wmbz63pd5ku4oo7g52cldga

Id

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebk376wtgg3qzmaegsjkhj7ue3dcmyvq72myykeiermk2vodijglg

Joyologist commented 1 year ago

0254d1e70c4085006ee89c8109e123c

Joyologist commented 1 year ago

Retrieval looks ok, willing to support this round

Joyologist commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaceavcynpcdwubmbq6a5kuuwmuj7rs2lhns3csnfevwcw4vcpdkhmhw

Address

f16xlgzogkkd5qm44uftfprkvh44gu4ruo7dmngca

Datacap Allocated

1.17PiB

Signer Address

f1txboxiscdm4f3okffyjmbilmwjtbisrbee72j6q

Id

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceavcynpcdwubmbq6a5kuuwmuj7rs2lhns3csnfevwcw4vcpdkhmhw

lyjmry commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

Storage Provider Distribution

⚠️ 1 storage providers sealed more than 30% of total datacap - f01261113: 33.72%

Deal Data Replication

⚠️ 76.80% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval report.

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

hhhhxiao commented 1 year ago

thanks please keep it open

cryptowhizzard commented 1 year ago

Can you check retrievals please?

Not all your miners are online at the moment.

cat 1436-f02060232-f0130556-34793173-baga6ea4seaql42yqhusbdveijmmvgdlfyaj44nseaehv7bhfh2x5lih6jmqqwhi.log 2023-07-27T06:20:59.711+0200 WARN rpc go-jsonrpc@v0.3.1/client.go:615 unmarshaling failed {"message": "{\"Err\":\"exhausted 5 attempts but failed to open stream, err: failed to dial 12D3KooWMAvCj89Vc9bYL7F41swmHV1YGLck6nSfhKQPPugZAeBW:\n [/ip4/120.232.233.80/tcp/24001] dial tcp4 120.232.233.80:24001: connect: connection refused\",\"Root\":null,\"Piece\":null,\"Size\":0,\"MinPrice\":\"\u003cnil\u003e\",\"UnsealPrice\":\"\u003cnil\u003e\",\"PricePerByte\":\"\u003cnil\u003e\",\"PaymentInterval\":0,\"PaymentIntervalIncrease\":0,\"Miner\":\"f0130556\",\"MinerPeer\":{\"Address\":\"f0130556\",\"ID\":\"12D3KooWMAvCj89Vc9bYL7F41swmHV1YGLck6nSfhKQPPugZAeBW\",\"PieceCID\":null}}"} 2023-07-27T06:20:59.711+0200 INFO retry retry/retry.go:17 Retrying after error:RPC client error: unmarshaling result: failed to parse big string: '"\u003cnil\u003e"' 2023-07-27T06:22:13.005+0200 WARN rpc go-jsonrpc@v0.3.1/client.go:615 unmarshaling failed {"message": "{\"Err\":\"exhausted 5 attempts but failed to open stream, err: failed to dial 12D3KooWMAvCj89Vc9bYL7F41swmHV1YGLck6nSfhKQPPugZAeBW:\n [/ip4/120.232.233.80/tcp/24001] dial tcp4 120.232.233.80:24001: connect: connection refused\",\"Root\":null,\"Piece\":null,\"Size\":0,\"MinPrice\":\"\u003cnil\u003e\",\"UnsealPrice\":\"\u003cnil\u003e\",\"PricePerByte\":\"\u003cnil\u003e\",\"PaymentInterval\":0,\"PaymentIntervalIncrease\":0,\"Miner\":\"f0130556\",\"MinerPeer\":{\"Address\":\"f0130556\",\"ID\":\"12D3KooWMAvCj89Vc9bYL7F41swmHV1YGLck6nSfhKQPPugZAeBW\",\"PieceCID\":null}}"} 2023-07-27T06:22:13.005+0200 INFO retry retry/retry.go:17 Retrying after error:RPC client error: unmarshaling result: failed to parse big string: '"\u003cnil\u003e"' 2023-07-27T06:24:57.602+0200 WARN rpc go-jsonrpc@v0.3.1/client.go:615 unmarshaling failed {"message": "{\"Err\":\"exhausted 5 attempts but failed to open stream, err: failed to dial 12D3KooWMAvCj89Vc9bYL7F41swmHV1YGLck6nSfhKQPPugZAeBW:\n [/ip4/120.232.233.80/tcp/24001] dial tcp4 120.232.233.80:24001: connect: connection refused\",\"Root\":null,\"Piece\":null,\"Size\":0,\"MinPrice\":\"\u003cnil\u003e\",\"UnsealPrice\":\"\u003cnil\u003e\",\"PricePerByte\":\"\u003cnil\u003e\",\"PaymentInterval\":0,\"PaymentIntervalIncrease\":0,\"Miner\":\"f0130556\",\"MinerPeer\":{\"Address\":\"f0130556\",\"ID\":\"12D3KooWMAvCj89Vc9bYL7F41swmHV1YGLck6nSfhKQPPugZAeBW\",\"PieceCID\":null}}"} 2023-07-27T06:24:57.602+0200 INFO retry retry/retry.go:17 Retrying after error:RPC client error: unmarshaling result: failed to parse big string: '"\u003cnil\u003e"' 2023-07-27T06:27:17.107+0200 WARN rpc go-jsonrpc@v0.3.1/client.go:615 unmarshaling failed {"message": "{\"Err\":\"exhausted 5 attempts but failed to open stream, err: failed to dial 12D3KooWMAvCj89Vc9bYL7F41swmHV1YGLck6nSfhKQPPugZAeBW:\n [/ip4/120.232.233.80/tcp/24001] dial tcp4 120.232.233.80:24001: connect: connection refused\",\"Root\":null,\"Piece\":null,\"Size\":0,\"MinPrice\":\"\u003cnil\u003e\",\"UnsealPrice\":\"\u003cnil\u003e\",\"PricePerByte\":\"\u003cnil\u003e\",\"PaymentInterval\":0,\"PaymentIntervalIncrease\":0,\"Miner\":\"f0130556\",\"MinerPeer\":{\"Address\":\"f0130556\",\"ID\":\"12D3KooWMAvCj89Vc9bYL7F41swmHV1YGLck6nSfhKQPPugZAeBW\",\"PieceCID\":null}}"} 2023-07-27T06:27:17.107+0200 INFO retry retry/retry.go:17 Retrying after error:RPC client error: unmarshaling result: failed to parse big string: '"\u003cnil\u003e"' 2023-07-27T06:29:43.846+0200 WARN rpc go-jsonrpc@v0.3.1/client.go:615 unmarshaling failed {"message": "{\"Err\":\"exhausted 5 attempts but failed to open stream, err: failed to dial 12D3KooWMAvCj89Vc9bYL7F41swmHV1YGLck6nSfhKQPPugZAeBW:\n * [/ip4/120.232.233.80/tcp/24001] dial tcp4 120.232.233.80:24001: connect: connection refused\",\"Root\":null,\"Piece\":null,\"Size\":0,\"MinPrice\":\"\u003cnil\u003e\",\"UnsealPrice\":\"\u003cnil\u003e\",\"PricePerByte\":\"\u003cnil\u003e\",\"PaymentInterval\":0,\"PaymentIntervalIncrease\":0,\"Miner\":\"f0130556\",\"MinerPeer\":{\"Address\":\"f0130556\",\"ID\":\"12D3KooWMAvCj89Vc9bYL7F41swmHV1YGLck6nSfhKQPPugZAeBW\",\"PieceCID\":null}}"} 2023-07-27T06:29:43.847+0200 ERROR retry retry/retry.go:29 Failed after 5 attempts, last error: RPC client error: unmarshaling result: failed to parse big string: '"\u003cnil\u003e"' ERROR: RPC client error: unmarshaling result: failed to parse big string: '"\u003cnil\u003e"'

Scherm­afbeelding 2023-07-28 om 14 30 43

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

hhhhxiao commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

⚠️ 64.93% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

hhhhxiao commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

⚠️ 65.56% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

cryptowhizzard commented 1 year ago

Scherm­afbeelding 2023-08-21 om 14 00 18

Only 3 of your SP's are online at the moment. Why?

hhhhxiao commented 1 year ago

Scherm­afbeelding 2023-08-21 om 14 00 18

Only 3 of your SP's are online at the moment. Why?

Thanks, we are contacting SP to find out why.

hhhhxiao commented 1 year ago

Some SPs could not be contacted, and one of them said that their technology encountered a problem and could not be solved temporarily. We decided to suspend cooperation with the problematic SP and wait for their repair to be completed before re-cooperating. @cryptowhizzard

cryptowhizzard commented 1 year ago

You have only 3 Sp's left , only 42% of your data is retrievable at maximum. :(

Let me know who you consider for replacement. I hope SP's with reputation this time?

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 5

Multisig Notary address

f02049625

Client address

f16xlgzogkkd5qm44uftfprkvh44gu4ruo7dmngca

DataCap allocation requested

1.17PiB

Id

c3f95fcb-b8ad-458e-b685-2cbfa298420f

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f02049625

Client address

f16xlgzogkkd5qm44uftfprkvh44gu4ruo7dmngca

Rule to calculate the allocation request amount

400% of weekly dc amount requested

DataCap allocation requested

1.17PiB

Total DataCap granted for client so far

1.0896474123001104e+36YiB

Datacap to be granted to reach the total amount requested by the client (5PiB)

1.0896474123001104e+36YiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
53254 10 1.17PiB 27.33 302.92TiB
zcfil commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

⚠️ 40.75% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

cryptowhizzard commented 1 year ago

I downloaded the data and made it available here:

http://www.datasetcreators.com/downloadedcarfiles/1436-f02060232-f01261113-50835053-baga6ea4seaqac2kioha5z47g6mwii4dvep3qhzqwrdizmq4siq52qcgoqed2aaq

The data does not match the data applicant indicated he is storing.

Scherm­afbeelding 2023-09-05 om 14 28 17

hhhhxiao commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

⚠️ 31.91% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

hhhhxiao commented 1 year ago

Project in progress

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

hhhhxiao commented 1 year ago

Project in progress

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

hhhhxiao commented 1 year ago

Project in progress

Sunnyiscoming commented 1 year ago

Hello, @hhhhxiao per the https://github.com/filecoin-project/notary-governance/issues/922 for Open, Public Dataset applicants, please complete the following Fil+ registration form to identify yourself as the applicant and also please add the contact information of the SP entities you are working with to store copies of the data.

This information will be reviewed by Fil+ Governance team to confirm validity and then the application will be allowed to move forward for additional notary review.

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

hhhhxiao commented 1 year ago

@Sunnyiscoming Sorry, it took me so long to reply to your message. Fil+ registration form has been submitted. Please check it. Thank you.

ghost commented 11 months ago

SP list provided f0130556 | Lanhe | CN | no | no f02051716 | US | no | no f02040772 | HB | UK | no | no f02320270 | R1 | US | no | no

ghost commented 11 months ago

checker:manualTrigger

filplus-checker-app[bot] commented 11 months ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

⚠️ 31.91% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard.

ghost commented 11 months ago

SPs taking deals: 130556 | Guangzhou, Guangdong, CNChina Mobile communications corporation | 145.06 TiB | 6.81% | 144.97 TiB | 0.06% f02226869 | Nanchang, Jiangxi, CNCHINA UNICOM China169 Backbone | 463.34 TiB | 21.74% | 463.34 TiB | 0.00% f01261113 | Shenzhen, Guangdong, CNCHINANET-BACKBONE | 451.22 TiB | 21.18% | 438.78 TiB | 2.76% f02059207 | Shenzhen, Guangdong, CNCHINANET-BACKBONE | 126.03 TiB | 5.91% | 126.03 TiB | 0.00% f01261218 | Shenzhen, Guangdong, CNCHINANET-BACKBONE | 50.28 TiB | 2.36% | 50.28 TiB | 0.00% f02121645 | Hanoi, Hanoi, VNDREAM POWER TECHNOLOGY LIMITED | 370.69 TiB | 17.40% | 370.69 TiB | 0.00% f02329561new | Hong Kong, Central and Western, HKDREAM POWER TECHNOLOGY LIMITED | 303.00 TiB | 14.22% | 303.00 TiB | 0.00% f02051716new | Seattle, Washington, USGTHost | 116.13 TiB | 5.45% | 116.13 TiB | 0.00% f02040772new | London, England, GBGTHost | 105.06 TiB | 4.93% | 105.06 TiB | 0.00%

SPs don't match list provided. Closing until a clear list of SP entities and distribution is provided

hhhhxiao commented 11 months ago

Sorry for replying so late, please confirm, thank you @Filplus-govteam

f02226869 | Nanchang | Frogbt | Darren | mantttsun@outlook.com f012611139 | Shenzhen | LunaTec | Lou | 1971197597@qq.com f021216459 | Hanoi | InfinityGroup | Eric | xvecos@163.com f02329561new9 | Hong Kong | AlphaConnections | Connor | xigancheng@gmail.com f02051716new9 | Seattle | StellarTech | Lucien | toneysgp@gmail.com f02040772new9 | London | APWS | Ken | furo2861@gmail.com

ghost commented 11 months ago

OK. Please collect SP entity and location information and send to filplus.govteam@gmail.com

hhhhxiao commented 10 months ago

@Filplus-govteam The email has been sent, please confirm

hhhhxiao commented 8 months ago

@Filplus-govteam @Sunnyiscoming Who do I need to ask for help to use it properly?