search

filecoin-project / filecoin-plus-large-datasets

Hub for client applications for DataCap at a large scale
110 stars 62 forks source link

[DataCap Allocation] - cloud native #1485

Closed hail100 closed 1 year ago

hail100 commented 1 year ago

name: Large Dataset Notary application about: Clients should use this application form to request a DataCap allocation via a LDN for a dataset title: "cloud native" labels: 'application, Phase: Diligence' assignees: ''

Large Dataset Notary Application

To apply for DataCap to onboard your dataset to Filecoin, please fill out the following.

Core Information

Please respond to the questions below by replacing the text saying "Please answer here". Include as much detail as you can in your answer.

Project details

Share a brief history of your project and organization.

       OpsMind (Shanghai Lingmi Information Technology Co., Ltd.), founded in 2017, is an open operation and maintenance capability building platform-as-a-service provider for the cloud-native era. Adhering to the corporate mission of ‘making IT information technology popular’ and the core values of ‘customer success, seeking truth from facts, long-termism, and first innovation’, OpsMind is committed to empowering enterprises to transform IT operation and maintenance into SRE with an operation and maintenance self-service platform and improving the penetration rate of business self-service and accelerate the process of enterprise digital transformation.
    With the advent of the cloud-native era, we firmly believe that IT operation and maintenance has transformed from the manager of the infrastructure to a part of the infrastructure. By using OpsMind, the enterprise IT operation and maintenance team can quickly and cost-effectively build personalized and customized self-service solutions that meet business needs. The service platform enables internal and external personnel of the company to complete query retrieval, observation analysis and operation control of operation and maintenance objects from a business perspective.
    We are a professional ITSM technology service provider, providing solutions such as full-stack monitoring and alarming, resource configuration management, multi-cloud management platform, automated operation platform, and release process management. Most importantly, we also provide a low-code secondary development platform for the above solutions, making the construction and changes of solutions more flexible and agile.
    The OpsMind R&D service center is located in Shanghai. The core team comes from well-known technology companies such as Qiniuyun, UCloud, Baidu, and Byte. Rich insights and practical experience, corporate customers widely cover media entertainment, corporate services, retail distribution, financial technology and other industries.

What is the primary source of funding for this project?

corporate income

What other projects/ecosystem stakeholders is this project associated with?

None.

Use-case details

Describe the data being stored onto Filecoin

We need to store a large number of surveillance video files, database data and various business information to meet customer business monitoring, real-time alarm and so on.

Where was the data in this dataset sourced from?

These data come from the company's business development, and a large amount of video surveillance data will be generated every day, which will provide business security for customers in various industries.

Can you share a sample of the data? A link to a file, an image, a table, etc., are good ways to do this. 

https://drive.google.com/drive/folders/1spEV-g75BmkSIapXOVf9e3UKkzThOqeM?usp=share_link
https://drive.google.com/drive/folders/1jGQDW8G0ViAFscYC2OSoMOBnOvE7xjeR?usp=share_link

Confirm that this is a public dataset that can be retrieved by anyone on the Network (i.e., no specific permissions or access rights are required to view the data).

Confirm

What is the expected retrieval frequency for this data?

No need frequent, 1-2 times a day

For how long do you plan to keep this dataset stored on Filecoin?

2 year

DataCap allocation plan

In which geographies (countries, regions) do you plan on making storage deals?

all regions

How will you be distributing your data to storage providers? Is there an offline data transfer process?

We have our own IDC room, can open the external network for customers downloading.

How do you plan on choosing the storage providers with whom you will be making deals? This should include a plan to ensure the data is retrievable in the future both by you and others.

We will choose a storage provider with high reputation and strong transaction confidentiality. Before we cooperate with them, we should know the size of their business, reliability, whether they have core hardware and professional technical team. Because we know the confidentiality of the data is very important

How will you be distributing deals across storage providers?

We will distribute equally to high-quality storage providers

Do you have the resources/funding to start making deals as soon as you receive DataCap? What support from the community would help you onboard onto Filecoin?

When we get our datacap, we make the transaction immediately. And there is enough money to pay for it.
large-datacap-requests[bot] commented 1 year ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

large-datacap-requests[bot] commented 1 year ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

large-datacap-requests[bot] commented 1 year ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

large-datacap-requests[bot] commented 1 year ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

Sunnyiscoming commented 1 year ago

surveillance video files are not suitable as public data stores. Can you explain your data composition? How many copies will you store? What's the relationship between you and the organization? Can you provide more detailed information about other storage providers participated in this program, such as you can list SPs you have contacted with at present? Whether the sps you choose can support data retrieval?

hail100 commented 1 year ago

@Sunnyiscoming Hello, Sunny iscoming. We have re-updated the public data. The original surveillance video is required for our business operation and maintenance IDC. We will monitor some risk factors of IDC through intelligent algorithms, and some surveillance videos have been removed; I am the project manager of the company, mainly responsible for the company's project development and implementation. Most of our data is generated by business development, including but not limited to cloud resource monitoring data, operation and maintenance object observation and analysis data, operation and maintenance object operation audit and cloud resources data such as cost analysis. At present, there are some SP service providers connected. f01949018, f01947280;

herrehesse commented 1 year ago

Dear Applicant,

Due to the increased amount of erroneous/wrong Filecoin+ data recently, on behalf of the entire community, we feel compelled to go deeper into datacap requests. Hereby to ensure that the overall value of the Filecoin network and Filecoin+ program increases and is not abused.

Please answer the questions below as comprehensively as possible.

Customer data

We expect that for the onboarding of customers with the scale of an LDN there would have been at least multiple email and perhaps several chat conversations preceding it. A single email with an agreement does not qualify here.

Should this only be soley for acquiring datacap this is of course out of the question. The customer must have a legitimate reason for wanting to use the Filecoin+ program which is intended as a program to store useful and public datasets on the network.

(As an intermediate solution Filecoin offers the FIL-E program or the glif.io website for business datasets that do not meet the requirements for a Filecoin+ dataset)

Files and Processing

Hopefully you understand the caution the overall community has for onboarding the wrong data. We understand the increased need for Filecoin+, however, we must not allow the program to be misused. Everything depends on a valuable and useful network, let's do our best to make this happen. Together.

simonkim0515 commented 1 year ago

Datacap Request Trigger

Total DataCap requested

5PiB

Expected weekly DataCap usage rate

100TiB

Client address

f1ewvlonhvzu2d377z3dvzo5wk4dh3rkccp6j234q

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Multisig Notary address

f02049625

Client address

f1ewvlonhvzu2d377z3dvzo5wk4dh3rkccp6j234q

DataCap allocation requested

50TiB

Id

1d1cbac5-bb29-42be-b902-838323a079a4

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report[^1]

There is no previous allocation for this issue.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

cryptowhizzard commented 1 year ago

Dear applicant,

Thank you for applying for datacap. As Filecoin FIL+ notary i am screening your application and conducting due diligence.

Looking at your application i have some questions: As you are brand new on Github and have no history of past applications it seems to me that applying for 5PB of datacap is a lot. One needs comprehensive knowledge of Filecoin, packing of data, distribution of data and all it's requirements coming with it. Are you brand new in the Filecoin space or have you applied for datacap in the past on different Github account names?

Can you show us visible proof of the size of your data and the storage systems you have there?

As last question i would like you to fill out this form to provide us with the necessary information to make a educated decision on your LDN request if we would like to support it.

Thanks!

hail100 commented 1 year ago

@cryptowhizzard hi，Completed the form

cryptowhizzard commented 1 year ago

Seems we have not received anything. Can you retry please?

hail100 commented 1 year ago

@cryptowhizzard Hello, I have conducted another handover here, and I have worked hard to check it image

hail100 commented 1 year ago

@cryptowhizzard Hello, haven't received it yet?

hail100 commented 1 year ago

@fabriziogianni7 Hello, can you verify it?

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

hail100 commented 1 year ago

This apply is using.

1ane-1 commented 1 year ago

Can you provide more detailed information about other storage providers participated in this program, such as you can list SPs you have contacted with at present? And how many copies do you want to store?

hail100 commented 1 year ago

@1ane-1 Hello, the relevant sps are f02094335, f02029743, f02114868, f02199393, f02121645.store 3-5 servings

1ane-1 commented 1 year ago

@1ane-1 Hello, the relevant sps are f02094335, f02029743, f02114868, f02199393, f02121645.store 3-5 servings

Thanks, for the first round, i am willing to support you. Hope you can obey the fil+ rules.

1ane-1 commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzaceaain37fmmxz6p2almxye5kdxvlyvuk2xoxpfsckxvrq6htsrkcmi

Address

f1ewvlonhvzu2d377z3dvzo5wk4dh3rkccp6j234q

Datacap Allocated

50.00TiB

Signer Address

f1mdk7s2vntzm6hu35yuo6vjubtrpfnb2awhgvrri

Id

1d1cbac5-bb29-42be-b902-838323a079a4

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceaain37fmmxz6p2almxye5kdxvlyvuk2xoxpfsckxvrq6htsrkcmi

sxxfuture-official commented 1 year ago

The SPs provided are all reachable. In the first round, I will support and continue to pay attention.

sxxfuture-official commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaceb6lv4upvnumid5mzeqlwii6pbepit66mcj6j4de34zfa42oquxqo

Address

f1ewvlonhvzu2d377z3dvzo5wk4dh3rkccp6j234q

Datacap Allocated

50.00TiB

Signer Address

f1foiomqlmoshpuxm6aie4xysffqezkjnokgwcecq

Id

1d1cbac5-bb29-42be-b902-838323a079a4

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceb6lv4upvnumid5mzeqlwii6pbepit66mcj6j4de34zfa42oquxqo

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 7

Multisig Notary address

f02049625

Client address

f1ewvlonhvzu2d377z3dvzo5wk4dh3rkccp6j234q

DataCap allocation requested

400TiB

Id

81ffda26-146d-4631-a418-8d2d2e004f2d

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f02049625

Client address

f1ewvlonhvzu2d377z3dvzo5wk4dh3rkccp6j234q

Rule to calculate the allocation request amount

400% of weekly dc amount requested

DataCap allocation requested

400TiB

Total DataCap granted for client so far

4.54747350886467e+44YiB

Datacap to be granted to reach the total amount requested by the client (5PiB)

4.54747350886467e+44YiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
0 0 50TiB NaN 16GiB
Chuangshi1 commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

⚠️ All retrieval success ratios are below 1%.

Storage Provider Distribution

⚠️ 1 storage providers sealed more than 70% of total datacap - f02233608: 100.00%

⚠️ All storage providers are located in the same region.

Deal Data Replication

⚠️ 100.00% of deals are for data replicated across less than 3 storage providers.

Deal Data Shared with other Clients[^3]

⚠️ CID sharing has been observed. (Top 3)

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

Chuangshi1 commented 1 year ago

Pls check your report.

large-datacap-requests[bot] commented 1 year ago

We have found some problems in the information provided in the Approved Comment. We could not find Address** field in the information provided

Please, take a look at the comment and edit the body of the comment providing all the required information.
cryptowhizzard commented 1 year ago

The SPs provided are all reachable. In the first round, I will support and continue to pay attention.

🥳🥳🥳🥳🥳🥳🥳🥳🥳🥳

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

⚠️ All retrieval success ratios are below 1%.

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

⚠️ CID sharing has been observed. (Top 3)

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

hail100 commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

⚠️ All retrieval success ratios are below 1%.

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

⚠️ CID sharing has been observed. (Top 3)

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

hail100 commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

⚠️ All retrieval success ratios are below 1%.

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

⚠️ CID sharing has been observed. (Top 3)

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

twosson commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

⚠️ All retrieval success ratios are below 1%.

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

⚠️ CID sharing has been observed. (Top 3)

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

hail100 commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

⚠️ All retrieval success ratios are below 1%.

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

⚠️ CID sharing has been observed. (Top 3)

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

twosson commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

⚠️ All retrieval success ratios are below 1%.

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

⚠️ CID sharing has been observed. (Top 3)

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

twosson commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

⚠️ All retrieval success ratios are below 1%.

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

⚠️ CID sharing has been observed. (Top 3)

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

twosson commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

⚠️ CID sharing has been observed. (Top 3)

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

hail100 commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

⚠️ CID sharing has been observed. (Top 3)

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

hail100 commented 1 year ago

The CID sharing issue was caused by the SP test during their initial storage/code optimization. This problem has been discussed with the SP, and it will not occur again in the future.

SuperChaiChai commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacedhkjnmhkpu6cn5twhlrk2sw2fmr3ierl72hoeiirid4qqhmj5yhu

Address

f1ewvlonhvzu2d377z3dvzo5wk4dh3rkccp6j234q

Datacap Allocated

400.00TiB

Signer Address

f12mckci3omexgzoeosjvstcfxfe4vqw7owdia3da

Id

81ffda26-146d-4631-a418-8d2d2e004f2d

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedhkjnmhkpu6cn5twhlrk2sw2fmr3ierl72hoeiirid4qqhmj5yhu