filecoin-project / filecoin-plus-large-datasets

Hub for client applications for DataCap at a large scale
110 stars 62 forks source link

[DataCap Application]FilSwan Multi-chain Storage Product #278

Closed yoogakim closed 1 year ago

yoogakim commented 2 years ago

Large Dataset Notary Application

To apply for DataCap to onboard your dataset to Filecoin, please fill out the following.

Core Information

Project details

Share a brief history of your project and organization.

Filswan provides a decentralized edge infrastructure marketplace and software that moves data and processing as close as necessary on Web 3.0. FilSwan multi-chain storage solution gates log files, artifacts, backups, container images, photos and videos on the FIlecoin Network. 
FilSwan Multi-chain storage is a cutting-edge blockchain storage solution providing a data cache and presidency layer for your important files. The smart contract-based multi-chain storage gateway is integrated with oracle technology.  
You will get an IPFS link with a long-term CID for your website's daily usage and benefit from the long-term storage network provided by Filecoin.  
With Filswan multi-chain storage solution, users can: 
*Upload files to the IPFS/Filecoin network  
*Pay with stable coins  
*Swap major cryptocurrencies to Fil Token via the Polygon network at a low cost 
*Mint NFTs on OpenSea while storing data on the IPFS/Filecoin network 

What is the primary source of funding for this project?

FilSwan, Filecoin Foundation's dev grant, Chainlink-FIlecoin joint grant 

What other projects/ecosystem stakeholders is this project associated with?

Polygon with storage providers cross continents.  

Use-case details

Describe the data being stored onto Filecoin

log files, artifacts, backups, container images, photos and videos  

Where was the data in this dataset sourced from?

Users’ data, uploaded by the users. 

Can you share a sample of the data? A link to a file, an image, a table, etc., are good ways to do this.

https://calibration-ipfs.filswan.com/ipfs/QmcZsKyYkxTUWqG2ogbrZTjDK7za8sQ2TgboZD6V9sK4pf

Confirm that this is a public dataset that can be retrieved by anyone on the Network (i.e., no specific permissions or access rights are required to view the data).

yes

What is the expected retrieval frequency for this data?

It should be no more than a weekly base.

For how long do you plan to keep this dataset stored on Filecoin?

540 days, it should be a permanent as soon as the client willing to expend it. 

DataCap allocation plan

In which geographies (countries, regions) do you plan on making storage deals?

All continents.

How will you be distributing your data to storage providers? Is there an offline data transfer process?

We will distribute to storage providers using offline deals 

How do you plan on choosing the storage providers with whom you will be making deals? This should include a plan to ensure the data is retrievable in the future both by you and others.

We use the Filswans auto-bid system, users need to submit their requests, and the system will match storage providers and users automatically. We also build an api for providers to download the dataset query the dataset name, and retrieval via miner id and cid. 

How will you be distributing deals across storage providers?

We are using FilSwan client agent for batch sending deals.  https://github.com/filswan/swan-client

Do you have the resources/funding to start making deals as soon as you receive DataCap? What support from the community would help you onboard onto Filecoin?

Yes. Funding & marketing support from the community.
large-datacap-requests[bot] commented 2 years ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

large-datacap-requests[bot] commented 2 years ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

large-datacap-requests[bot] commented 2 years ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

large-datacap-requests[bot] commented 2 years ago

Thanks for your request! :exclamation: We have found some problems in the information provided. We could not find your Filecoin address in the information provided

Please, take a look at the request and edit the body of the issue providing all the required information.
large-datacap-requests[bot] commented 2 years ago

Thanks for your request! :exclamation: We have found some problems in the information provided. We could not find your Filecoin address in the information provided

Please, take a look at the request and edit the body of the issue providing all the required information.
large-datacap-requests[bot] commented 2 years ago

Thanks for your request! :exclamation: We have found some problems in the information provided. We could not find your Filecoin address in the information provided

Please, take a look at the request and edit the body of the issue providing all the required information.
large-datacap-requests[bot] commented 2 years ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

large-datacap-requests[bot] commented 2 years ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

IreneYoung commented 2 years ago

I have interest in the multi-chain storage project. We team also has similar plans to do so possibly after FVM launching. Hopefully there will be more sharing and communication

flyworker commented 2 years ago

We are kicking off the competition for this product, this is a project award by FilSwan Awarded Chainlink-Filecoin Joint Grant to Create Multi-Chain Decentralized Data Storage Payment Solution

https://filswan.medium.com/multi-chain-storage-testnet-competition-collect-points-and-share-a-200-fil-prize-pool-b6d637953b48

flyworker commented 2 years ago

just a follow up, anything we need to do for moving forward? @galen-mcandrew

kernelogic commented 2 years ago

I support this application because it is part of the ecosystem.

large-datacap-requests[bot] commented 2 years ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

Sunnyiscoming commented 2 years ago

image Hi, @flyworker Data sample can not be opened. Can you check it?

flyworker commented 2 years ago

@Sunnyiscoming sorry about the downtime, we were in a network update event, everything should be back online now

Sunnyiscoming commented 2 years ago

Yes, the file has been seen.

Sunnyiscoming commented 2 years ago

Describe the data being stored onto Filecoin log files, artifacts, backups, container images, photos and videos
Where was the data in this dataset sourced from? Users’ data, uploaded by the users.

Can you provide more abundant data samples to prove that you have 5PB data already?

AthSmith commented 2 years ago

For this kind of data container tool platform, not recommended to apply for LDN. No one guarantees the quality and value of the dataset, although we'd like to support such eco collaborators.

flyworker commented 2 years ago

For this kind of data container tool platform, not recommended to apply for LDN. No one guarantees the quality and value of the dataset, although we'd like to support such eco collaborators.

Are you really Protocal Labs employee? Your account looks like a spammer

kernelogic commented 2 years ago

Lol, even protocol is misspelled.

Sunnyiscoming commented 2 years ago

@flyworker Hi. Can you provide more abundant data samples to prove that you have 5PB data already?

dkkapur commented 2 years ago

@flyworker quick question for you - is it fair to assume this is similar to Estuary / Textile Bidbot in that users bring data that they upload to Filecoin via FilSwan? What's your mechanism for picking SPs and tracking reputation over time?

Sunnyiscoming commented 2 years ago

Any update here?

flyworker commented 2 years ago

@flyworker quick question for you - is it fair to assume this is similar to Estuary / Textile Bidbot in that users bring data that they upload to Filecoin via FilSwan? What's your mechanism for picking SPs and tracking reputation over time?

Yes you are correct, it is similar to Estuary+Textile Bidbot but using offline deal storage and enable user upload files and mint as NFT. more than 5000 owners mint over 100K NFT on the opensea testnet.

image

Miners are selected with a deal matching engine similar to bidbot but use filswan reputation module, details about the reputation can be found here: https://console.filswan.com/#/methodology currently there are more than 50 sps connected with filswan engine, and they are signed on filswan platform, we can see their information on https://console.filswan.com/#/dashboard

image
raghavrmadya commented 2 years ago

@flyworker , Thanks for confirming. Could you elaborate a bit more on the your mechanism for picking SPs and tracking reputation over time?

flyworker commented 2 years ago

Hi with filswan MCS, We already have the following miners accept our deals in the past 6 months: f010088 f02301 f0187709 f0461791 f01402814 f01072221 f0240185 f0143858 f01390330 f01222595 f03223 f01840390 f01859603 f01133080 f01858429 f01817860 f01883179

FIlSwan platform has a miner reputation tracking system:

Reputation Score = Time-based Reachability + Regional Weighted Adjusted Power + General Deals and Verified-Storage Provider Deals https://docs.filswan.com/filswan-platform/overview/reputation-system

A sample reputation score can be found here: https://console.filswan.com/#/miner/f0187709 Miners will get less/No deals if they cannot complete the sealing task.

When select the miners, the auction system will randomly select the miners based on Gaussian distribution, the higher ranking you have, the more chance you will get a deal. for miners below 30 points you won't be able to get a deal in 7 days. Failed in complete sealing will cause you lost score.

for details about the auction: https://docs.filswan.com/filswan-platform/overview/filswan-auction-system

The average deal send out is around 100 deals with 16~32GB sectors.

raghavrmadya commented 2 years ago

Datacap Request Trigger

Total DataCap requested

5 PiB

Expected weekly DataCap usage rate

100 TiB

Client address

f1o54sve7ede7im4caux3ug7lsyjmbue7ss3zzl6y

large-datacap-requests[bot] commented 2 years ago

DataCap Allocation requested

Multisig Notary address

f01858410

Client address

f1o54sve7ede7im4caux3ug7lsyjmbue7ss3zzl6y

DataCap allocation requested

50TiB

kernelogic commented 2 years ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzaceb4cb5tu725mv2rzghq762br4bsr24de6pisinbikarbaw5lpvqqu

Address

f1o54sve7ede7im4caux3ug7lsyjmbue7ss3zzl6y

Datacap Allocated

50.00TiB

Signer Address

f1yjhnsoga2ccnepb7t3p3ov5fzom3syhsuinxexa

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceb4cb5tu725mv2rzghq762br4bsr24de6pisinbikarbaw5lpvqqu

NiwanDao commented 2 years ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaced3u6jcaoojn367ihdhvnwl6waouuektwz7rzngmj2lyiijcr2pls

Address

f1o54sve7ede7im4caux3ug7lsyjmbue7ss3zzl6y

Datacap Allocated

50.00TiB

Signer Address

f1a2lia2cwwekeubwo4nppt4v4vebxs2frozarz3q

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaced3u6jcaoojn367ihdhvnwl6waouuektwz7rzngmj2lyiijcr2pls

yoogakim commented 2 years ago

finally get the account back

flyworker commented 2 years ago

it seems the auto trigger not working due to previous account issue.

haheihou5 commented 2 years ago

支持swan支持小脑斧

haheihou5 commented 2 years ago

快点通过,给swan发datacap,我们SP等着封装呢,快点快点

tangmizhong commented 2 years ago

Thanks for the filswan's work. As an SP we feel proud to support the filecoin ecosystem

jamesxuzm commented 2 years ago

谢谢filswan的订单让我的节点获得对filecoin 生态的更多参与

hxg8202 commented 2 years ago

Thanks for the filswan's work. As an SP we feel proud to support the filecoin ecosystem

large-datacap-requests[bot] commented 2 years ago

DataCap Allocation requested

Request number 2

Multisig Notary address

f01858410

Client address

f1o54sve7ede7im4caux3ug7lsyjmbue7ss3zzl6y

DataCap allocation requested

100TiB

large-datacap-requests[bot] commented 2 years ago

Stats & Info for DataCap Allocation

Multisig Notary address

f01858410

Client address

f1o54sve7ede7im4caux3ug7lsyjmbue7ss3zzl6y

Last two approvers

xingjitansuo & kernelogic

Rule to calculate the allocation request amount

100% of weekly dc amount requested

DataCap allocation requested

100TiB

Total DataCap granted for client so far

50TiB

Datacap to be granted to reach the total amount requested by the client (5PiB)

4.95PiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
2114 20 50TiB 72.73 8GiB
spark8899 commented 2 years ago

support swan

Joshua-zhao commented 2 years ago

The MCS from filswan has been very supportive for SPs and it has contributed a lot to filecoin ecosystem. hopefully this datacap application could be approved so as to benefit more SPs in filecoin ecosystem.

kernelogic commented 2 years ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzaceb24esor7cxugvd7j3i75mcsytitvbsh6gx2cvtncfw7aqbx2voee

Address

f1o54sve7ede7im4caux3ug7lsyjmbue7ss3zzl6y

Datacap Allocated

100.00TiB

Signer Address

f1yjhnsoga2ccnepb7t3p3ov5fzom3syhsuinxexa

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceb24esor7cxugvd7j3i75mcsytitvbsh6gx2cvtncfw7aqbx2voee

IreneYoung commented 2 years ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaceahn5kbpdikktft3g7jfmnxjsuvez2lu6yu6xohcaesaxqbx6ydd6

Address

f1o54sve7ede7im4caux3ug7lsyjmbue7ss3zzl6y

Datacap Allocated

100.00TiB

Signer Address

f1d4gmpqz3execjj2wvrxuuhvbms5mzh7t7yqrviq

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceahn5kbpdikktft3g7jfmnxjsuvez2lu6yu6xohcaesaxqbx6ydd6

large-datacap-requests[bot] commented 2 years ago

DataCap Allocation requested

Request number 3

Multisig Notary address

f01858410

Client address

f1o54sve7ede7im4caux3ug7lsyjmbue7ss3zzl6y

DataCap allocation requested

200TiB

large-datacap-requests[bot] commented 2 years ago

Stats & Info for DataCap Allocation

Multisig Notary address

f01858410

Client address

f1o54sve7ede7im4caux3ug7lsyjmbue7ss3zzl6y

Last two approvers

IreneYoung & kernelogic

Rule to calculate the allocation request amount

200% of weekly dc amount requested

DataCap allocation requested

200TiB

Total DataCap granted for client so far

50TiB

Datacap to be granted to reach the total amount requested by the client (5PiB)

4.95PiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
2114 20 100TiB 72.73 8GiB
cryptowhizzard commented 2 years ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacebqnfcmvgy5avrqgyfyiqryzvg3xppvxtmzvj5selj4c2ui2l4dsc

Address

f1o54sve7ede7im4caux3ug7lsyjmbue7ss3zzl6y

Datacap Allocated

200.00TiB

Signer Address

f1krmypm4uoxxf3g7okrwtrahlmpcph3y7rbqqgfa

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebqnfcmvgy5avrqgyfyiqryzvg3xppvxtmzvj5selj4c2ui2l4dsc

Joss-Hua commented 2 years ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaceak5nyye44643bzmcckgyfqpy5yui47jmlu7hporyuaelfxzqmkbm

Address

f1o54sve7ede7im4caux3ug7lsyjmbue7ss3zzl6y

Datacap Allocated

200.00TiB

Signer Address

f1tfg54zzscugttejv336vivknmsnzzmyudp3t7wi

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceak5nyye44643bzmcckgyfqpy5yui47jmlu7hporyuaelfxzqmkbm

large-datacap-requests[bot] commented 2 years ago

DataCap Allocation requested

Request number 4

Multisig Notary address

f01858410

Client address

f1o54sve7ede7im4caux3ug7lsyjmbue7ss3zzl6y

DataCap allocation requested

400TiB

large-datacap-requests[bot] commented 2 years ago

Stats & Info for DataCap Allocation

Multisig Notary address

f01858410

Client address

f1o54sve7ede7im4caux3ug7lsyjmbue7ss3zzl6y

Last two approvers

Joss-Hua & cryptowhizzard

Rule to calculate the allocation request amount

400% of weekly dc amount requested

DataCap allocation requested

400TiB

Total DataCap granted for client so far

250TiB

Datacap to be granted to reach the total amount requested by the client (5PiB)

4.75PiB

Stats

Number of deals Number of storage providers Previous DC Allocated Top provider Remaining DC
6024 28 200TiB 26.97 49.53TiB
IreneYoung commented 2 years ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacebkpke3batgm723pb66mrrb3qc6xa5gm56cyyeuofvomfnw7ecltw

Address

f1o54sve7ede7im4caux3ug7lsyjmbue7ss3zzl6y

Datacap Allocated

400.00TiB

Signer Address

f1d4gmpqz3execjj2wvrxuuhvbms5mzh7t7yqrviq

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebkpke3batgm723pb66mrrb3qc6xa5gm56cyyeuofvomfnw7ecltw