search

filecoin-project / filecoin-plus-large-datasets

Hub for client applications for DataCap at a large scale
110 stars 62 forks source link

[DataCap Application]All Blue #389

Closed Pearl918 closed 1 year ago

Pearl918 commented 2 years ago

Large Dataset Notary Application

To apply for DataCap to onboard your dataset to Filecoin, please fill out the following.

Core Information

Please respond to the questions below by replacing the text saying "Please answer here". Include as much detail as you can in your answer.

Project details

Share a brief history of your project and organization.

All blue solves the problem with traditional banking .We are building credit, savings, and investment products. Democratizing financial services in ways that are equitable and transparent. To redefine people’s relationship with money through simple, accessible credit, personal finance, digital banking offerings, and much more.

What is the primary source of funding for this project?

Company's income.

What other projects/ecosystem stakeholders is this project associated with?

We were the only company involved in the project.

Use-case details

Describe the data being stored onto Filecoin

Dataset include Hong Kong stock data, foreign exchange, American stocks, Shanghai and Shenzhen stocks.

Where was the data in this dataset sourced from?

The data comes from the historical data during 2000 to 2019, including :
1032TiB of Shanghai and Shenzhen stock analysis data, 
223TiB of foreign exchange analysis data, 
10Tib of Hong Kong stock analysis data, 
15TiB of American stock analysis data,
Total 1280TiB data.

Can you share a sample of the data? A link to a file, an image, a table, etc., are good ways to do this. 

[^GSPC.csv](https://github.com/filecoin-project/filecoin-plus-large-datasets/files/8830450/GSPC.csv)
[^DJI.csv](https://github.com/filecoin-project/filecoin-plus-large-datasets/files/8830451/DJI.csv)
[AAPL.csv](https://github.com/filecoin-project/filecoin-plus-large-datasets/files/8830452/AAPL.csv)
[^IXIC.csv](https://github.com/filecoin-project/filecoin-plus-large-datasets/files/8830462/IXIC.csv)

Confirm that this is a public dataset that can be retrieved by anyone on the Network (i.e., no specific permissions or access rights are required to view the data).

Yes, we confirm!

What is the expected retrieval frequency for this data?

Data retrieval has no fixed frequency, depending on the needs of the applications or customers.

For how long do you plan to keep this dataset stored on Filecoin?

We plan to keep this dataset on Filecoin 2-3 years.

DataCap allocation plan

In which geographies (countries, regions) do you plan on making storage deals?

China and Asia-GCN.

How will you be distributing your data to storage providers? Is there an offline data transfer process?

Online transfer and offline copy.

How do you plan on choosing the storage providers with whom you will be making deals? This should include a plan to ensure the data is retrievable in the future both by you and others.

We plan to select multiple storage vendors (4-8) and request them open for retrieval.
1) SH and SZ stock analysis data in China.
2) Foreign exchange, Hong Kong stock, American stock analysis data in Asia -GCN.

How will you be distributing deals across storage providers?

No single SPs can make deals more than 20%.

Do you have the resources/funding to start making deals as soon as you receive DataCap? What support from the community would help you onboard onto Filecoin?

Yes, we have the resources/funding to begin making deals .
kevzak commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

⚠️ 7 storage providers sealed too much duplicate data - f01885260: 84.20%, f01885280: 60.88%, f01880897: 66.37%, f01880894: 74.02%, f01880896: 78.62%, f01879880: 79.33%, f01845679: 76.08%

⚠️ 1 storage providers have unknown IP location - f01845679

Deal Data Replication

⚠️ 100.00% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

⚠️ CID sharing has been observed. (Top 3)

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the full report.

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 6

Multisig Notary address

f02049625

Client address

f1dpgqn57cl5wqijiyv3256nids2ox3fms2mg3oay

DataCap allocation requested

4.72TiB

Id

7849113d-83e5-4b0e-a48a-bda1a8d45401

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f01858410

Client address

f1dpgqn57cl5wqijiyv3256nids2ox3fms2mg3oay

Rule to calculate the allocation request amount

400% of weekly dc amount requested

DataCap allocation requested

4.72TiB

Total DataCap granted for client so far

1.51805579662323e+52YiB

Datacap to be granted to reach the total amount requested by the client (5 PiB)

-1.83B

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
136199 21 1.63PiB 12.69 409.74TiB
kevzak commented 1 year ago

Notaries: Serious due diligence is needed before anymore signatures are considered. This application's CID report highlights significant duplicate data being stored across SPs. @Pearl918 would need to explain in detail what happened to justify continuation.

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

github-actions[bot] commented 1 year ago

This application has not seen any responses in the last 14 days, so for now it is being closed. Please feel free to contact the Fil+ Gov team to re-open the application if it is still being processed. Thank you!