filecoin-project / filecoin-plus-large-datasets

Hub for client applications for DataCap at a large scale
109 stars 62 forks source link

[DataCap Application] < AOLIGEI > #2077

Closed CaptaiNLP closed 1 year ago

CaptaiNLP commented 1 year ago

Data Owner Name

AOLIGEI

What is your role related to the dataset

Dataset Owner

Data Owner Country/Region

China

Data Owner Industry

Information, Media & Telecommunications

Website

www.aoligei-Fighting.com

Social Media

none

Total amount of DataCap being requested

4PiB

Expected size of single dataset (one copy)

450T

Number of replicas to store

10

Weekly allocation of DataCap requested

500TiB

On-chain address for first allocation

f1ofjicywsxwx6nzuz36upvsziibcsw2c7jsfrola

Data Type of Application

Public, Open Commercial/Enterprise

Custom multisig

Identifier

No response

Share a brief history of your project and organization

Zhejiang AOLIGEI Culture Media Co., Ltd is a cultural media platform focused on the development of artists. With accurate market positioning, we’ve won a certain market share out of many competitors. Engaged in the filed of the image promotion of internet influencers, short video planning, filming training consulting, etc., not only do we have a large number of cultural media resources, but also create their own flow of IP with the professional operation teams, helping them achieve their own dreams!
Our company currently has more than 100 artists under contract, and our current business has close ties with film and television companies, social media platforms, and e-commerce.
1、Cooperating with some film and TV companies to shoot TV series and advertisement promotion works.
2、Cooperating with some social media platforms to publish videos, pictures and advertisements of our contracted artists.
3、We also do e-commerce by selling a full range of products through live-stream media.

Is this project associated with other projects/ecosystem stakeholders?

No

If answered yes, what are the other projects/ecosystem stakeholders

No response

Describe the data being stored onto Filecoin

The data stored on Filecoin are our business videos, commercials, image and client’s video materials.

Where was the data currently stored in this dataset sourced from

My Own Storage Infra

If you answered "Other" in the previous question, enter the details here

No response

How do you plan to prepare the dataset

IPFS, lotus, singularity

If you answered "other/custom tool" in the previous question, enter the details here

No response

Please share a sample of the data

Douying:
https://www.douyin.com/user/MS4wLjABAAAAqHIH7OBGnA77xHvRJNYaNXRS5MAm-2lDxNIk3dJSHtU?modal_id=7117310005006568740
https://www.douyin.com/user/MS4wLjABAAAAqHIH7OBGnA77xHvRJNYaNXRS5MAm-2lDxNIk3dJSHtU?modal_id=7116789513619361032
https://www.douyin.com/user/MS4wLjABAAAAqHIH7OBGnA77xHvRJNYaNXRS5MAm-2lDxNIk3dJSHtU?modal_id=7118665932796431647
https://www.douyin.com/user/MS4wLjABAAAAAD-lSehi3CNwG1I8bTSfQUZ-Ak2j9Rxdulx6TZNFsAc?modal_id=7026563322648579362
https://www.douyin.com/user/MS4wLjABAAAAAD-lSehi3CNwG1I8bTSfQUZ-Ak2j9Rxdulx6TZNFsAc?modal_id=7062130813122481423
https://www.douyin.com/user/MS4wLjABAAAA5Zr5L2owYCE9QuAWtxzbYfBc2Nhj0b4Pf8FBk-xyUu4?modal_id=6651515890045160711
https://www.douyin.com/user/MS4wLjABAAAA5Zr5L2owYCE9QuAWtxzbYfBc2Nhj0b4Pf8FBk-xyUu4?modal_id=7134982490384616743
https://www.douyin.com/user/MS4wLjABAAAA5Zr5L2owYCE9QuAWtxzbYfBc2Nhj0b4Pf8FBk-xyUu4?modal_id=7135387938128497958
https://www.douyin.com/user/MS4wLjABAAAA5Zr5L2owYCE9QuAWtxzbYfBc2Nhj0b4Pf8FBk-xyUu4?modal_id=7136491982939049227
https://www.douyin.com/user/MS4wLjABAAAA_UAtaMCagWq8N1RPE-KCyW9VIyt6wwhhJ3J4l7pI-tXCiK6I1TQvV1Adv9PkQj2a?modal_id=6837725081230200071

Confirm that this is a public dataset that can be retrieved by anyone on the Network

If you chose not to confirm, what was the reason

No response

What is the expected retrieval frequency for this data

Yearly

For how long do you plan to keep this dataset stored on Filecoin

1 to 1.5 years

In which geographies do you plan on making storage deals

Greater China, Asia other than Greater China, North America

How will you be distributing your data to storage providers

HTTP or FTP server, IPFS

How do you plan to choose storage providers

Slack, Filmine, Big Data Exchange

If you answered "Others" in the previous question, what is the tool or platform you plan to use

No response

If you already have a list of storage providers to work with, fill out their names and provider IDs below

f099132 Guangzhou, Guangdong, CN
Hangzhou Alibaba Advertising Co.,Ltd.   4.72 TiB    10.07%  4.69 TiB    0.66%
f021704new  Shanghai, Shanghai, CN
Hangzhou Alibaba Advertising Co.,Ltd.   2.47 TiB    5.27%   2.47 TiB    0.00%
f01878534   Los Angeles, California, US
Zenlayer Inc    11.81 TiB   25.22%  11.81 TiB   0.00%
f01853104   Los Angeles, California, US
Zenlayer Inc    11.16 TiB   23.82%  11.16 TiB   0.00%
f01893023   Los Angeles, California, US
Zenlayer Inc    9.94 TiB    21.21%  9.94 TiB    0.00%
f01890456   Los Angeles, California, US
Zenlayer Inc

------
These are the storage providers we have contacted previously, and we plan to contact a few more for joint storage

How do you plan to make deals to your storage providers

Boost client, Lotus client

If you answered "Others/custom tool" in the previous question, enter the details here

No response

Can you confirm that you will follow the Fil+ guideline

Yes

large-datacap-requests[bot] commented 1 year ago

Thanks for your request!

Heads up, you’re requesting more than the typical weekly onboarding rate of DataCap!
large-datacap-requests[bot] commented 1 year ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

CaptaiNLP commented 1 year ago

https://github.com/filecoin-project/filecoin-plus-large-datasets/issues/880

The above is our previous application form, because LDN reported an error, so we can not continue, hope to resubmit a new application form to continue

Sunnyiscoming commented 1 year ago

https://github.com/filecoin-project/filecoin-plus-large-datasets/issues/880#issuecomment-1612251369

cryptowhizzard commented 1 year ago

Hi there,

f099132 Guangzhou, Guangdong, CN Hangzhou Alibaba Advertising Co.,Ltd. 4.72 TiB 10.07% 4.69 TiB 0.66% f021704new Shanghai, Shanghai, CN Hangzhou Alibaba Advertising Co.,Ltd. 2.47 TiB 5.27% 2.47 TiB 0.00% f01878534 Los Angeles, California, US Zenlayer Inc 11.81 TiB 25.22% 11.81 TiB 0.00% f01853104 Los Angeles, California, US Zenlayer Inc 11.16 TiB 23.82% 11.16 TiB 0.00% f01893023 Los Angeles, California, US Zenlayer Inc 9.94 TiB 21.21% 9.94 TiB 0.00% f01890456 Los Angeles, California, US Zenlayer Inc


These are the storage providers we have contacted previously, and we plan to contact a few more for joint storage

This list has a most providers flagged in applications with FIL+ not following on the rules.

Sunnyiscoming commented 1 year ago

Any update here?

herrehesse commented 1 year ago

Please close this application.

CaptaiNLP commented 1 year ago

@Sunnyiscoming The 2077 application is a continuation of the 880 LDN, please help me close the 880 and open the 2077.

@herrehesse I'll close 880.

herrehesse commented 1 year ago

@CaptaiNLP Please respond to the above: https://github.com/filecoin-project/filecoin-plus-large-datasets/issues/2077#issuecomment-1615154347

CaptaiNLP commented 1 year ago

@herrehesse Hi, thanks for the kind reminder.

As sps, they have done ok with 880, no mixing of my datastores, etc. they are qualified sps to me. as for your comment about them not following the rules of the community as sps, it could be some other project they are working with that has nothing to do with my application.

I will however contact other sps again for storage of our data to reduce the percentage of storage for these sps you mentioned. If the sps is not following the rules of the community, we will stop working with them.

cryptowhizzard commented 1 year ago

@herrehesse Hi, thanks for the kind reminder.

As sps, they have done ok with 880, no mixing of my datastores, etc. they are qualified sps to me. as for your comment about them not following the rules of the community as sps, it could be some other project they are working with that has nothing to do with my application.

I will however contact other sps again for storage of our data to reduce the percentage of storage for these sps you mentioned.

If the sps is not following the rules of the community, we will stop working with them.

This is what i meant to say ;)

CaptaiNLP commented 1 year ago

@herrehesse That's for sure.

Sunnyiscoming commented 1 year ago

Best practice for storing large datasets includes ideally, storing it in 3 or more regions, with 4 or more storage provider operators or owners. You should list Miner ID, Business Entity, Location of sps in one more region.

CaptaiNLP commented 1 year ago

@Sunnyiscoming f02185816 Hong Kong f01972110 Beijing f02037700 South Korea f02192496 Guangdong f02199413 Vietnam

In addition to the SPs we have cooperated with before, the following are also the SPs we plan to cooperate with, and they are located in different regions. And as I mentioned above, I will no longer work with these sps if they have violations in my LDN reports

Sunnyiscoming commented 1 year ago
herrehesse commented 1 year ago

The initial request for 4 PiB using a newly created account and without establishing trust within the community is concerning. I highly recommend close the application and reconsider by starting with a smaller amount to build credibility and trust before making larger requests.

Advising notaries not to engage.

cryptowhizzard commented 1 year ago

SP f02199413 is not online and open for deals. It does not have it's public address set on chain yet.

CaptaiNLP commented 1 year ago

@Sunnyiscoming 1, yes 2、Yes, #880 is our previous project 3、The current data are stored in our own server, we will provide the download method to the cooperating sp. The sp will assist us to store the data 4、The data will be prepared by us and downloaded and stored by the sp. 5、The data was not stored on filecoin before, because our server cost is too high, we need to do some space release.

@cryptowhizzard Apart from this one sp node, our other nodes have met the requirements in 4 different geographies, and we will be contacting other sp nodes in the future.

cryptowhizzard commented 1 year ago

Hello,

Zenlayer does not provide physical hosting for equipment of miners and storage. It is a cloud/ VPN provider. Please provide us with the right locations of the SP's you intend to work with.

ghost commented 1 year ago

Hello @CaptaiNLP per the new guidelines https://github.com/filecoin-project/notary-governance/issues/922 for Open Dataset applicants, please complete the following Fil+ registration form to identify yourself as the applicant and also please add the contact information of the SP entities you are working with to store copies of the data.

This information will be reviewed by Fil+ Governance team to confirm validity toward the Fil+ guideline of a distributed storage plan and then the application will be approved for notary review. Let us know if you have any questions.

Sunnyiscoming commented 1 year ago

Any update here?

CaptaiNLP commented 1 year ago

@Sunnyiscoming @Filplus-govteam Thanks for the reply, I've submitted the form, please check it out.

cf16350fdf9bcf9265a5e7864e981f7

ghost commented 1 year ago

We received the same SP list posted above with no way to contact these SPs: f02185816 Hong Kong f01972110 Beijing f02037700 South Korea f02192496 Guangdong f02199413 Vietnam

Any notaries can do due diligence here on SP entities? Do they have a good record? Do they support retrievals?

cryptowhizzard commented 1 year ago

In my system I have f01972110 flagged in #1084 for CID sharing and non-retrievability.

f02185816, f02037700, f02192496 are in the same /24 as other SP's flagged for CID sharing and non-retrievability

f02199413 is clean and ok

ghost commented 1 year ago

@CaptaiNLP there seems to be some concerns over your SPs. Please ensure you are choosing SPs that meet distribution and retrievability of Fil+ guidelines.

Sunnyiscoming commented 1 year ago

Datacap Request Trigger

Total DataCap requested

4PiB

Expected weekly DataCap usage rate

500TiB

Client address

f1ofjicywsxwx6nzuz36upvsziibcsw2c7jsfrola

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Multisig Notary address

f02049625

Client address

f1ofjicywsxwx6nzuz36upvsziibcsw2c7jsfrola

DataCap allocation requested

204.79TiB

Id

2f1a478a-c12f-487e-9fcb-5a38a02a2470

CaptaiNLP commented 1 year ago

@Filplus-govteam Regarding the application for 880, you can see that our storage project inspection reports fully comply with the Fil+ guidelines for distribution and retrievability. This is because we have rigorously monitored the entire process. Due to the issues with LDN reporting, our project has been put on hold for a long time. We hope to receive the support of community members and also look forward to seeing the results in the next round of the check bot. Thank you.

CaptaiNLP commented 1 year ago

Since the previously partnered SPs couldn't initiate immediately due to issues like pledge coins and data center adjustments, we now have an update on SPs that can be collaborated with immediately: f02212669 china hangzhou Cryptomage f0411877 hongkong FTM f02226892 china shenzhen Blackhole f02320270 USA R1

ghost commented 1 year ago

OK @CaptaiNLP let's check the storage progress after your first allocation.

joshua-ne commented 1 year ago

It seems @Filplus-govteam has cleared all the questions and since this is the first round with reasonable amount of DC requested, I will support this round.

joshua-ne commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacebn6nww3jazqemqlpzsekbqydk376ljt2amjlsunb62ro6wjx457u

Address

f1ofjicywsxwx6nzuz36upvsziibcsw2c7jsfrola

Datacap Allocated

204.79TiB

Signer Address

f1xzff5xup63o5sygr2swp4zvcajg54lotliimdty

Id

2f1a478a-c12f-487e-9fcb-5a38a02a2470

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebn6nww3jazqemqlpzsekbqydk376ljt2amjlsunb62ro6wjx457u

1ane-1 commented 1 year ago

Support based on the information that you offered!

1ane-1 commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacedbcdzpdaufaa2viiu4hoa4oukvfg27crrgg5lo2qvuiqtidfspqa

Address

f1ofjicywsxwx6nzuz36upvsziibcsw2c7jsfrola

Datacap Allocated

204.79TiB

Signer Address

f1mdk7s2vntzm6hu35yuo6vjubtrpfnb2awhgvrri

Id

2f1a478a-c12f-487e-9fcb-5a38a02a2470

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedbcdzpdaufaa2viiu4hoa4oukvfg27crrgg5lo2qvuiqtidfspqa

CaptaiNLP commented 1 year ago

Since the previously partnered SPs couldn't initiate immediately due to issues like pledge coins and data center adjustments, we now have an update on SPs that can be collaborated with immediately: f02212669 china hangzhou Cryptomage f0411877 hongkong FTM f02226892 china shenzhen Blackhole f02320270 USA R1

update: f02212669 china hangzhou Cryptomage --shortage of collateral f0411877 hongkong FTM --shortage of collateral f02226892 china shenzhen Blackhole --shortage of collateral

update to: f02329119 - HZ - Cryptomage f02274508 - HK - H&W f02211572 - HK - MicroAnt

@Filplus-govteam

cryptowhizzard commented 1 year ago

Scherm­afbeelding 2023-08-15 om 15 43 55

f02211572 is not in HongKong. You are not storing according to your storage plan and not following distribution rules and guidelines.

ghost commented 1 year ago

checker:manualTrigger

CaptaiNLP commented 1 year ago

@cryptowhizzard @Filplus-govteam

Hello, I apologize for the confusion. It was just an editing error. We haven't started the allocation yet. I was simply disclosing the SPs we are preparing to cooperate with in advance, as requested by the community. Here is the updated list: f02329119 - china hangzhou - Cryptomage f02274508 - HK - H&W f02211572 - china sichuan - MicroAnt

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 2

Multisig Notary address

f02049625

Client address

f1ofjicywsxwx6nzuz36upvsziibcsw2c7jsfrola

DataCap allocation requested

500TiB

Id

43cf7a13-2291-4536-a499-c24727969374

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f02049625

Client address

f1ofjicywsxwx6nzuz36upvsziibcsw2c7jsfrola

Rule to calculate the allocation request amount

100% of weekly dc amount requested

DataCap allocation requested

500TiB

Total DataCap granted for client so far

204.78TiB

Datacap to be granted to reach the total amount requested by the client (4PiB)

3.80PiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
3691 6 204.78TiB 28.17 66.32TiB
CaptaiNLP commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

⚠️ 100.00% of deals are for data replicated across less than 3 storage providers.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

cryptowhizzard commented 1 year ago

Dear CaptaiNLP,

As notary I am doing due diligence on your LDN. I could not get retrieval to work. Can you please upload the car file of CID baga6ea4seaqhq7v72wn6732tc5idehru7ecqzoylggy6x5stfm4aveabqpnxqky ?

You can use our upload system at http://send.datasetcreators.com. Please select 7 days for the system to keep the file and post the link you received here so I (and other notaries) can download your content.

kevzak commented 1 year ago

SPs provied: f02329119 - china hangzhou - Cryptomage f02274508 - HK - H&W f02211572 - china sichuan - MicroAnt

SPs currently taking Deals: f02211572 | Chengdu, Sichuan, CNChina Mobile Communications Group Co., Ltd. | 11.59 TiB | 9.08% | 11.59 TiB | 0.00% f02226869new | Nanchang, Jiangxi, CNCHINA UNICOM China169 Backbone | 256.00 GiB | 0.20% | 256.00 GiB | 0.00% f02274508 | Hong Kong, Central and Western, HKChina Unicom Global | 30.00 TiB | 23.51% | 30.00 TiB | 0.00% f02329119 | Hangzhou, Zhejiang, CNCT-HangZhou-IDC | 29.94 TiB | 23.46% | 29.94 TiB | 0.00% f01853077 | Singapore, Singapore, SGZenlayer Inc | 35.25 TiB | 27.62% | 35.25 TiB | 0.00% f01852363 | Singapore, Singapore, SGZenlayer Inc | 20.59 TiB | 16.14% | 20.59 TiB | 0.00%

@CaptaiNLP Seeing more SPs taking deals than originally provided. Please confirm who they are and please insure deal distribution is increased across all SPs

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

CaptaiNLP commented 1 year ago

Hello, @kevzak As the storage project progresses, more service providers will be added according to community rules, so this is reasonable. Additionally, here is the primary information for these service providers: f02226869new | Nanchang, Jiangxi, CNCHINA UNICOM China169 Backbone | LuckyMine f01853077 | Singapore, Singapore, SGZenlayer Inc | Alpha100 f01852363 | Singapore, Singapore, SGZenlayer Inc | HectorLi

CaptaiNLP commented 1 year ago

@cryptowhizzard I'm sorry, I can't upload this file. It's showing that the file is too large.

cryptowhizzard commented 1 year ago

@CaptaiNLP

Then use Winzip or something to split it in 2 or more pieces.

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

-- Commented by Stale Bot.

CaptaiNLP commented 1 year ago

@cryptowhizzard I'm sorry. I'm not a technical person.