search

filecoin-project / filecoin-plus-large-datasets

Hub for client applications for DataCap at a large scale
110 stars 62 forks source link

Niwan Dao / CabrinaHuang - ChiSheng Digital Human #2140

Open NiwanDao opened 11 months ago

NiwanDao commented 11 months ago

Data Owner Name

ChiSheng Digital Human

What is your role related to the dataset

Data onramp entity that provides data onboarding services to multiple clients

Data Owner Country/Region

China

Data Owner Industry

IT & Technology Services

Website

https://www.anyimeta.com/

Social Media

https://dstorage.cabrina.xyz/

Total amount of DataCap being requested

15PiB

Expected size of single dataset (one copy)

1P

Number of replicas to store

10

Weekly allocation of DataCap requested

500TiB

On-chain address for first allocation

f1tuf6hs6jjigjodjiir5dxbn7rxjn6kaavjatura

Data Type of Application

Public, Open Commercial/Enterprise

Custom multisig

Identifier

No response

Share a brief history of your project and organization

As an active participant in the Slingshot project since 2021, I have gained extensive experience and knowledge every single point in the flow of data preparation, deal-making, deal sealing and retrieval. Throughout my involvement, I have established fruitful relationships with fellow community members and successfully conducted numerous deal-making, primarily focused on open-source scientific data, with over 100 SPs globally.

By combining the mission of Filecoin to store humanity's most important data with the rapid advancements in the era of artificial intelligence, I have engaged with and explored various companies in the AIGC domain over the past few months. Based on my comprehensive research, I firmly believe that storing the vast amounts of data generated by artificial intelligence on a decentralized network is an inevitable choice.

To demonstrate compliance with the FIL Plus rules, I have showcased the datasets and SP partners I previously onboarded on the website https://dstorage.cabrina.xyz/, as a gesture of respect towards the regulations and guidelines imposed.

Is this project associated with other projects/ecosystem stakeholders?

No

If answered yes, what are the other projects/ecosystem stakeholders

No response

Describe the data being stored onto Filecoin

ChiSheng Digital human is an industry leading AIGC Digital Human Incubation Platform based in China. It creates engaging videos with AI-powered digital human. It provides human digital cloning service: simply shooting the model in front of green canvas studio and using the AI model, the model's meta human will create in hours. Video Generation Service: Simply type in your text and choose digital human and voice to create a professional video in minutes. 

ChiSheng Digital human will onboard the AI generated video and relative materials including pictures, video, text, and audio  in Filecoin.

Where was the data currently stored in this dataset sourced from

My Own Storage Infra

If you answered "Other" in the previous question, enter the details here

No response

If you are a data preparer. What is your location (City and Country)

Sichuan, China

If you are a data preparer, how will the data be prepared? Please include tooling used and technical details?

I will use Singularity to generate the CAR file. Additionally, I have developed my own software that incorporates features such as SP partner communication for deal agreement, deal distribution, deal status tracking, and deal-resend mechanism to ensure compliance with the FilPlus rule requirements.

If you are not preparing the data, who will prepare the data? (Provide name and business)

No response

Has this dataset been stored on the Filecoin network before? If so, please explain and make the case why you would like to store this dataset again to the network. Provide details on preparation and/or SP distribution.

This is the first time.

Please share a sample of the data

https://drive.google.com/file/d/1xpOitdo5rqHal1kgcNPf60WeDXN8_Ut0/view?usp=drive_link

I put a few video generated by ChiSheng Platform as sample.

Confirm that this is a public dataset that can be retrieved by anyone on the Network

If you chose not to confirm, what was the reason

No response

What is the expected retrieval frequency for this data

Monthly

For how long do you plan to keep this dataset stored on Filecoin

1 to 1.5 years

In which geographies do you plan on making storage deals

Greater China, Asia other than Greater China, North America

How will you be distributing your data to storage providers

HTTP or FTP server, Shipping hard drives

How do you plan to choose storage providers

Big Data Exchange, Partners

If you answered "Others" in the previous question, what is the tool or platform you plan to use

No response

If you already have a list of storage providers to work with, fill out their names and provider IDs below

I work with SP including but not limited to FL cloud, GreaterHeat, Ciic, Kinx. xingjiliangzi, Mayi,  

The partner I previously worked with are listed in https://dstorage.cabrina.xyz/sp/

How do you plan to make deals to your storage providers

Boost client

If you answered "Others/custom tool" in the previous question, enter the details here

No response

Can you confirm that you will follow the Fil+ guideline

Yes

large-datacap-requests[bot] commented 11 months ago

Thanks for your request!

Heads up, you’re requesting more than the typical weekly onboarding rate of DataCap!
large-datacap-requests[bot] commented 11 months ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

Sunnyiscoming commented 11 months ago
Sunnyiscoming commented 11 months ago

Hi @NiwanDao

Per the https://github.com/filecoin-project/notary-governance/issues/922 for Open, Public Dataset applicants, please complete the following Fil+ registration form to identify yourself as the applicant and also please add the contact information of the SP entities you are working with to store copies of the data.

This information will be reviewed by Fil+ Governance team to confirm validity and then the application will be triggered for notary review. Let us know if you have any questions.

NiwanDao commented 11 months ago

Business Entity | Location | Miner ID GREATERHEAT| Dallas | USA | f0** .... Mayi | Singapore| Singapore| f01808139 ... FL cloud |yangzhou| China| f0161916 ... TianJi Data| Hongkong| China| f02202943 ... Enjoycloud | guizhou| China | f02229279 xingjiliangzi | chongqing | china | f01845912 .....

Sunnyiscoming commented 11 months ago

Received business license. Have you submitted Fil+ registration form ?

NiwanDao commented 11 months ago
截屏2023-08-09 下午11 52 01

@Sunnyiscoming Yes, I did.

ghost commented 11 months ago

SPs confirmed as listed above

Sunnyiscoming commented 10 months ago

Datacap Request Trigger

Total DataCap requested

15PiB

Expected weekly DataCap usage rate

500TiB

Client address

f1tuf6hs6jjigjodjiir5dxbn7rxjn6kaavjatura

large-datacap-requests[bot] commented 10 months ago

DataCap Allocation requested

Multisig Notary address

f02049625

Client address

f1tuf6hs6jjigjodjiir5dxbn7rxjn6kaavjatura

DataCap allocation requested

250TiB

Id

09e0ba18-676f-4764-b3e9-d1ffb6f196c6

kernelogic commented 10 months ago

Very interesting dataset, willing to support and further verify what AI videos it can generate.

kernelogic commented 10 months ago

I hope no one will jump out and say AIGC does not qualify FIL+!

kernelogic commented 10 months ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacedfwnlbnieuf57ie5sjbluz3dgz35tbtssj4n4unckb2qgwm7wkhc

Address

f1tuf6hs6jjigjodjiir5dxbn7rxjn6kaavjatura

Datacap Allocated

250.00TiB

Signer Address

f1yjhnsoga2ccnepb7t3p3ov5fzom3syhsuinxexa

Id

09e0ba18-676f-4764-b3e9-d1ffb6f196c6

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedfwnlbnieuf57ie5sjbluz3dgz35tbtssj4n4unckb2qgwm7wkhc

nj-steve commented 10 months ago

Hello, how much data do you have prepared? Have you learned how to store data on the filecoin network?

NiwanDao commented 10 months ago

Hello, how much data do you have prepared? Have you learned how to store data on the filecoin network?

Thanks for asking and I have all knowledge in data perpetration. I have onboarded at least 30P to the network over the past year. @nj-steve

newwebgroup commented 10 months ago

This Client has rich experience in storing data on Filecoin and has a good track record.

The information of ChiSheng Digital Human has been checked, KYC verification has been completed, and the SP list has been provided. AIGC is a recent hot topic and very interesting.

Willing to provide support in the first round and looking forward to future compliance performance.

newwebgroup commented 10 months ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaced7jjiny6uhlnswqcb4enziuu5kk6mflgmtiiffcbnkgwzithji3k

Address

f1tuf6hs6jjigjodjiir5dxbn7rxjn6kaavjatura

Datacap Allocated

250.00TiB

Signer Address

f1e77zuityhvvw6u2t6tb5qlnsegy2s67qs4lbbbq

Id

09e0ba18-676f-4764-b3e9-d1ffb6f196c6

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaced7jjiny6uhlnswqcb4enziuu5kk6mflgmtiiffcbnkgwzithji3k

github-actions[bot] commented 10 months ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

NiwanDao commented 10 months ago

WIP

github-actions[bot] commented 10 months ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

large-datacap-requests[bot] commented 10 months ago

DataCap Allocation requested

Request number 2

Multisig Notary address

f02049625

Client address

f1tuf6hs6jjigjodjiir5dxbn7rxjn6kaavjatura

DataCap allocation requested

500TiB

Id

6af5f611-6c59-46e0-a6de-59ffd5160328

large-datacap-requests[bot] commented 10 months ago

Stats & Info for DataCap Allocation

Multisig Notary address

f02049625

Client address

f1tuf6hs6jjigjodjiir5dxbn7rxjn6kaavjatura

Rule to calculate the allocation request amount

100% of weekly dc amount requested

DataCap allocation requested

500TiB

Total DataCap granted for client so far

250TiB

Datacap to be granted to reach the total amount requested by the client (15PiB)

14.75PiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
0 0 250TiB NaN 250TiB
NiwanDao commented 10 months ago

WIP

NewHuoPool commented 9 months ago

checker:manualTrigger

filplus-checker-app[bot] commented 9 months ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

⚠️ All retrieval success ratios are below 1%.

Storage Provider Distribution

⚠️ All storage providers are located in the same region.

Deal Data Replication

⚠️ 100.00% of deals are for data replicated across less than 3 storage providers.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

NewHuoPool commented 9 months ago

Storing AI data on IPFS is a very good idea. I've already checked the data samples and everything looks fine. Would you mind sharing the plan for adding SP in the later stages?

NiwanDao commented 9 months ago

@NewHuoPool thanks for your asking. As I disclose in the application on the SP partner I work with, there is no much difference from what will happen. Other SPs are in the data transfer phase. The statistics will look much better in the next round.

NewHuoPool commented 9 months ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacedkezjpshftjutlkjs7oac2qrlesjmdejqfts6ozxdhtrkvh6iyj2

Address

f1tuf6hs6jjigjodjiir5dxbn7rxjn6kaavjatura

Datacap Allocated

500.00TiB

Signer Address

f16karfxq7lxdy7izqrzrk75jf3not34k6sg6zvcy

Id

6af5f611-6c59-46e0-a6de-59ffd5160328

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedkezjpshftjutlkjs7oac2qrlesjmdejqfts6ozxdhtrkvh6iyj2

METAVERSEDATAMINING commented 9 months ago

Keenly interested in this project and anticipate the smooth loading of the data.

METAVERSEDATAMINING commented 9 months ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaceabhs2ukl2ivqt6aaomsleri3eiqoeuryb3hhdes64jitv6262w3m

Address

f1tuf6hs6jjigjodjiir5dxbn7rxjn6kaavjatura

Datacap Allocated

500.00TiB

Signer Address

f17idrnfnxl2mbgcgr57a6z2c6lj2qx56gvm3336i

Id

6af5f611-6c59-46e0-a6de-59ffd5160328

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceabhs2ukl2ivqt6aaomsleri3eiqoeuryb3hhdes64jitv6262w3m

NiwanDao commented 9 months ago

checker:manualTrigger

filplus-checker-app[bot] commented 9 months ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

⚠️ All retrieval success ratios are below 1%.

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

⚠️ 78.78% of deals are for data replicated across less than 3 storage providers.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

NiwanDao commented 9 months ago

I talked to SPs offline, and the retrieval statistics report will look much better once the SP fixes the router problem. Also, this round is pretty urgent as other SPs are in the process of sealing data.

luobin544 commented 9 months ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacebpywrwddqhogejl2dly5nztirsjcfxsrd3rnua6hozbzvdbsfmd4

Address

f1tuf6hs6jjigjodjiir5dxbn7rxjn6kaavjatura

Datacap Allocated

500.00TiB

Signer Address

f1tbd632f6w62glfaf7wjpimacbnjiz26poyoes2q

Id

6af5f611-6c59-46e0-a6de-59ffd5160328

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebpywrwddqhogejl2dly5nztirsjcfxsrd3rnua6hozbzvdbsfmd4

nj-steve commented 9 months ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaceavjhlot6uwzqumrwlffbnxmapjtkgmtrejtaux3x4kqg57egto6s

Address

f1tuf6hs6jjigjodjiir5dxbn7rxjn6kaavjatura

Datacap Allocated

500.00TiB

Signer Address

f1xx6555qijma7igpnjspyvdunc4vfxkawnpqy5ii

Id

6af5f611-6c59-46e0-a6de-59ffd5160328

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceavjhlot6uwzqumrwlffbnxmapjtkgmtrejtaux3x4kqg57egto6s

nj-steve commented 9 months ago

hope you solve the ⚠️ as soon as possible. I will keep eyes on it later.

NiwanDao commented 9 months ago

checker:manualTrigger

filplus-checker-app[bot] commented 9 months ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

⚠️ All retrieval success ratios are below 1%.

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

NiwanDao commented 9 months ago

All SPs are checking the low retrieval rate. It should get better in the future.

NiwanDao commented 9 months ago

checker:manualTrigger

filplus-checker-app[bot] commented 9 months ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

Bitrise0111 commented 9 months ago

Client contacted me through Slack and promised to improve their retrieval rate. Since their rate has met the standard at this round, we'd like to give a support. Meanwhile, we'll continue to check it to make sure they are working on improving retrieval rate.

Bitrise0111 commented 9 months ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacecshuosnopkb44l6x2kngnxywwpbt3sndngetqjrafo6zprs5wfni

Address

f1tuf6hs6jjigjodjiir5dxbn7rxjn6kaavjatura

Datacap Allocated

500.00TiB

Signer Address

f1nknj7ayq4o43czrtdoauggtwl43fbqatmqis3yy

Id

6af5f611-6c59-46e0-a6de-59ffd5160328

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacecshuosnopkb44l6x2kngnxywwpbt3sndngetqjrafo6zprs5wfni

igoovo commented 9 months ago

checker:manualTrigger

filplus-checker-app[bot] commented 9 months ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

igoovo commented 9 months ago

All SPs are checking the low retrieval rate. It should get better in the future.

The client has a good reputation in the historical records. The retrieval rate has improved compared to the check results of yesterday. The client has also promised to make improvements,There is no CID sharing, and the distribution of SPs is also even. The replication of the replicas is also reasonable. so we are willing to support moving forward in this round.

igoovo commented 9 months ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacebvfohziaev7kl73sfpmtjvvyju5u332k2tskzfdpvm747gikfozu

Address

f1tuf6hs6jjigjodjiir5dxbn7rxjn6kaavjatura

Datacap Allocated

500.00TiB

Signer Address

f1shnsfayxqll77svffaxnjenms7bbbysbqcatrpy

Id

6af5f611-6c59-46e0-a6de-59ffd5160328

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebvfohziaev7kl73sfpmtjvvyju5u332k2tskzfdpvm747gikfozu

github-actions[bot] commented 9 months ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

NiwanDao commented 9 months ago

checker:manualTrigger

filplus-checker-app[bot] commented 9 months ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.