FILDCKabat commented 2 years ago

Large Dataset Notary Application

To apply for DataCap to onboard your dataset to Filecoin, please fill out the following.

Core Information

Organization Name: AGENCJA PROMOCJI MOTORYZACJI A.P.M.AUTO
Website / Social Media: https://motowizja.pl/
Total amount of DataCap being requested (between 500 TiB and 5 PiB): 5PiB
Weekly allocation of DataCap requested (usually between 1-100TiB): 100TiB
On-chain address for first allocation: f1ieughpsgkdynelb2tipfpuhdvcvltfvsuv54q2q
Type: Custom Notary
Identifier: E-fil

Please respond to the questions below by replacing the text saying "Please answer here". Include as much detail as you can in your answer.

Project details

Share a brief history of your project and organization.

AGENCJA PROMOCJI MOTORYZACJI A.P.M.AUTO makes motorsport movies for over 20 years, we have been doing motorsport content for different televisions and also for private competitors, as well as for YouTube's channels. For today we make realizations mainly for Motowizja televisions which is the biggest motorsport one in Poland. We would like to store our big archives.

What is the primary source of funding for this project?

own source

What other projects/ecosystem stakeholders is this project associated with?

none

Use-case details

Describe the data being stored onto Filecoin

nothing stored yet, we plan mainly movie files

Where was the data in this dataset sourced from?

own 20Y of work in movies branch

Can you share a sample of the data? A link to a file, an image, a table, etc., are good ways to do this.

(https://www.youtube.com/watch?v=qfoQqS2a4ck) this is an example of ready, short movie, but mainly we 30min programs and we like to store all uncut files, which are many times bigger.

Confirm that this is a public dataset that can be retrieved by anyone on the Network (i.e., no specific permissions or access rights are required to view the data).

NO, there is no problem to show what type of files we are going to store but we like to get files encrypted at the final stage.

What is the expected retrieval frequency for this data?

rare, it's a matter of keeping archive

For how long do you plan to keep this dataset stored on Filecoin?

5Years +

DataCap allocation plan

In which geographies (countries, regions) do you plan on making storage deals?

Poland, Estonia, USA

How will you be distributing your data to storage providers? Is there an offline data transfer process?

Yes, there is offline possibilities with SP from who we got a contact

How do you plan on choosing the storage providers with whom you will be making deals? This should include a plan to ensure the data is retrievable in the future both by you and others.

Kabat LTD (Poland and Estonia) and Piknik (USA)

How will you be distributing deals across storage providers?

Eight full replicas, 5 of 40 [PiB] of Datacap

Primary SP Partner:
FILDC (f01127678, f02097113, f02218935, f02239603)
Location: Europe, Poland:
-> Poznan: f01127678, f02770730
-> Lodz: f02239603, f02097113, f02817132, f02218935

Additional replica's partners:
DSS, Australia (f01919423, f01938357)
DCENT, Europe, Netherlands (f02033496, f01771403)
Hong Kong Morpho Technology Limited, Hong Kong (f01989888, f01926635, f01933917, f01885088, f02058048, f01999119)
Saturnia, Russian Federation (f08240, f0187709, f01222595)
Pride Holdings, UAE, Dubai (f01907556)
PiKNiK, USA, Las Vegas (f01851060)
Flux Storage Pte. Ltd, Europe, Germany (f01619524)
Web3 providers LLC, Europe, Bulgaria (f020378)
Stander, Europe, Latvia (f099608)

Random partners from BDE (max 2 full replicas):
Many SPs around the world except Russia & Ukraine
Partial replicas, size: 100-1000TiB
Max 2 copies per SP if stored on different miners / locations.

Do you have the resources/funding to start making deals as soon as you receive DataCap? What support from the community would help you onboard onto Filecoin?

Ready to work with just SPs

large-datacap-requests[bot] commented 2 years ago

Thanks for your request! :exclamation: We have found some problems in the information provided.

Address should start with either f1 or f3

Please, take a look at the request and edit the body of the issue providing all the required information.

large-datacap-requests[bot] commented 2 years ago

Thanks for your request! :exclamation: We have found some problems in the information provided.

Address should start with either f1 or f3

Please, take a look at the request and edit the body of the issue providing all the required information.

FILDCKabat commented 2 years ago

f01127678

large-datacap-requests[bot] commented 2 years ago

Thanks for your request! :exclamation: We have found some problems in the information provided.

Address should start with either f1 or f3

Please, take a look at the request and edit the body of the issue providing all the required information.

large-datacap-requests[bot] commented 2 years ago

Thanks for your request! :exclamation: We have found some problems in the information provided.

Address should start with either f1 or f3

Please, take a look at the request and edit the body of the issue providing all the required information.

large-datacap-requests[bot] commented 2 years ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

kevzak commented 2 years ago

@raghavrmadya FYI this will be a E-Fil+ application

raghavrmadya commented 2 years ago

Datacap Request Trigger

Total DataCap requested

5PiB

Expected weekly DataCap usage rate

100TiB

Client address

f1ieughpsgkdynelb2tipfpuhdvcvltfvsuv54q2q

large-datacap-requests[bot] commented 2 years ago

Hello @derricktan23 - @BlockMakeronline , please sign the datacap request

large-datacap-requests[bot] commented 2 years ago

DataCap Allocation requested

Multisig Notary address

f01940930

Client address

f1ieughpsgkdynelb2tipfpuhdvcvltfvsuv54q2q

DataCap allocation requested

50TiB

Id

f2ef7b1e-0040-433c-8079-f2eec3865665

large-datacap-requests[bot] commented 2 years ago

Hello @Fenbushi-Filecoin - @Fatman13 , please sign the datacap request

kevzak commented 2 years ago

Hello - I can confirm for the Fil+ team that @FILDCKabat has completed the Client Registration and Business Verification Check successfully. Notaries can proceed to review the application and the Exception Proposal https://github.com/filecoin-project/notary-governance/issues/622

kevzak commented 2 years ago

Per SP request, I updated the client on-chain address

Fatman13 commented 2 years ago

The application looks solid, plus the fact that @Don Kabat is a pretty active community member on Slack. Except there is one thing that may need to be confirmed. .

Describe the data being stored onto Filecoin

nothing stored yet, we plan mainly movie files

Do you have the ownership of the movies? i.e would SP get into any legal trouble for storing these movies (copy right music etc)? Other than that everything else looks fine. Sample looks cool too.

Currently I am not able to sign this LDN though as the backend only allows one LDN for me to sign (everything else is greyed out). I have DMed f+ team for help on that matter.

Fenbushi-Filecoin commented 2 years ago

Hi @kevzak , I don't see the application on my backend.

Also, for the application, the # of SPs seems to be not enough. Need more details on the distribution plan.

kevzak commented 2 years ago

@FILDCKabat see question about copyright and comment above about more Data Storage details. Please let us know. Thanks!

FILDCKabat commented 2 years ago

Full copyright is owned by our customer, company: "AGENCJA PROMOCJI MOTORYZACJI A.P.M.AUTO". All data is going to be encrypted at the customer side. We are going to create cars. So from our and customer's point of view there is no issue between us.

kevzak commented 2 years ago

@Fatman13 @Fenbushi-Filecoin we made an update, you should be able to see the application now. Let me know.

FILDCKabat commented 2 years ago

Data Storage Plan

Five full replicas, total of 25 PiB of Datacap

Primary SP Partner: FILDC, receiving 5 PiB of Datacap for one full replica, Location: Lodz, Poland, Europe

1 additional SP from EU / North America / Australia receiving 5 PiB, for the 2nd full replica

Many SPs around the world except Russia and Ukraine Partial replicas, size: 100-1000TiB Max 2 copies per SP if stored on different miners / locations. Unsealed copy is not needed.

Fatman13 commented 2 years ago

Hello, team, for me its still the old situation where everything is greyed out with only the top one clickable... Could you please take a look into this issue? Thank you!

kevzak commented 2 years ago

@fabriziogianni7 @panges2 can you help @Fatman13 ? Thanks

FILDCKabat commented 2 years ago

Per SP request, I updated the client on-chain address

we have to remember that correct client's address is f1ieughpsgkdynelb2tipfpuhdvcvltfvsuv54q2q

kevzak commented 1 year ago

@Fatman13 I realize you are having issues with notary review screen, but are you able to review this application and make a decision? They are awaiting approval. Thank you.

Fatman13 commented 1 year ago

Yes, I approve this application. Sorry for the delay. Still working on fixing the signing issue.

donkabat commented 1 year ago

@Fatman13 hi, there is still wrong client's address on your screenshot. As we and @kevzak wrote, we asked to change from FILDC miner's address: f1cuoouxsmyeigcqbbviebgvdd4xdkapy42jgo4vy to client's address: f1ieughpsgkdynelb2tipfpuhdvcvltfvsuv54q2q (ID f01955887)

When first Allocation should be done? What do you think?

kevzak commented 1 year ago

Hi @donkabat we are realizing there is a larger issue with @Fatman13 s account right now. Apologies for delays.

Let's ask either @derricktan23 or @BlockMakeronline, who were originally tagged on this application above (see link https://github.com/filecoin-project/filecoin-plus-large-datasets/issues/928#issuecomment-1309990634) to review and trigger the approval. Thank you all.

dkkapur commented 1 year ago

I see that the ledger is verified. @panges2 can we request you to check in with @Fatman13 and maybe screenshare on a call and see what's happening?

kevzak commented 1 year ago

FYI @dkkapur https://github.com/filecoin-project/filecoin-plus-large-datasets/issues/928#issuecomment-1316882635 was sent to the wrong client address. the client onchain address was updated, but the notary view was apparently not also updated?

donkabat commented 1 year ago

FYI @dkkapur #928 (comment) was sent to the wrong client address. the client onchain address was updated, but the notary view was apparently not also updated?

yes, I informed @kevzak just after I saw first Allocation Request here: https://github.com/filecoin-project/filecoin-plus-large-datasets/issues/928#issuecomment-1310001085 I hope that it's not a big problem to fix it now... If you need something from FILDC just ping @FILDCKabat ;) thx!

kevzak commented 1 year ago

looking for feedback from @fabriziogianni7 here on the on-chain address

donkabat commented 1 year ago

Hi everybody! When can we expect that the address will be fixed and we can start datacap allocation?

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Multisig Notary address

f01940930

Client address

f1ieughpsgkdynelb2tipfpuhdvcvltfvsuv54q2q

DataCap allocation requested

50TiB

Id

d92597f9-bb50-49e6-9955-0439573bf9ae

kevzak commented 1 year ago

Hi @Fenbushi-Filecoin @tom-originstorage can you resign the first allocation request here? We had to update the client on-chain address, it should be correct now. Thanks! cc @donkabat

Fatman13 commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacecmoaggklwp2ddeturk6gt74te6soyvkqjyho4ukgqwmpfpmtqhyy

Address

f1ieughpsgkdynelb2tipfpuhdvcvltfvsuv54q2q

Datacap Allocated

50.00TiB

Signer Address

f1j3u7crhjzwb2cj5mq7vodlt4o66yoyci7lhcauy

Id

d92597f9-bb50-49e6-9955-0439573bf9ae

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacecmoaggklwp2ddeturk6gt74te6soyvkqjyho4ukgqwmpfpmtqhyy

Fenbushi-Filecoin commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaceahkwszswzcuwkvc37gewazv4qh76yfqohywzghbnjmzjqqlzb3zo

Address

f1ieughpsgkdynelb2tipfpuhdvcvltfvsuv54q2q

Datacap Allocated

50.00TiB

Signer Address

f1yqydpmqb5en262jpottko2kd65msajax7fi4rmq

Id

d92597f9-bb50-49e6-9955-0439573bf9ae

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceahkwszswzcuwkvc37gewazv4qh76yfqohywzghbnjmzjqqlzb3zo

kevzak commented 1 year ago

@donkabat we have a few new LDN questions we're asking, we'd like to learn about your project:

How do you plan to make deals to your storage providers?

Boost Client
Lotus Client
Bidbot
Singularity
I dont know yet
Other/custom tool

kevzak commented 1 year ago

How do you plan to prepare the dataset?

IPFS
lotus
singularity
graphsplit
I don't know
other/custom tool (if this, can you please add more detail?)

kevzak commented 1 year ago

How will you be distributing your data to storage providers? cloud storage (S3)

HTTP or FTP Server
IPFS
Shipping Hard Drives
Lotus Built-in data transfer
I don't know yet
others

donkabat commented 1 year ago

How do you plan to make deals to your storage providers?

Boost Client

How do you plan to prepare the dataset?

Singularity

How will you be distributing your data to storage providers?

HTTP or FTP Server Shipping Hard Drives Boost online deals

donkabat commented 1 year ago

Hi, when bot will make new DataCap Allocation requested? We're going to start deals next week but 50TiB ran out after 2 days.

DSS-AL commented 1 year ago

Hi @FILDCKabat we're an enterprise scale storage provider in Sydney Australia, we'd be more than happy to discuss storing a full replica here if you're still seeking SPs to support this project. If so please reach out on Filecoin Slack @andrew leece

kevzak commented 1 year ago

@donkabat I'm following with with Fil+ team now to confirm next steps for subsequent allocations.

filplus-checker commented 1 year ago

DataCap and CID Checker Report[^1]

Organization: AGENCJA PROMOCJI MOTORYZACJI A.P.M.AUTO
Client: f1ieughpsgkdynelb2tipfpuhdvcvltfvsuv54q2q
Storage Provider Distribution

The below table shows the distribution of storage providers that have stored data for this client.

If this is the first time a provider takes verified deal, it will be marked as new.

For most of the datacap application, below restrictions should apply.

Storage provider should not exceed 25% of total datacap.
Storage provider should not be storing duplicate data for more than 20%.
Storage provider should have published its public IP address.
All storage providers should be located in different regions.

⚠️ f01885088 has sealed 99.65% of total datacap.

Provider	Location	Total Deals Sealed	Percentage	Unique Data	Duplicate Deals
f01885088	Hong Kong, Central and Western, HK	26.39 TiB	99.65%	26.39 TiB	0.00%
f01127678	Łódź, Łódź Voivodeship, PL	96.00 GiB	0.35%	96.00 GiB	0.00%

Provider Distribution

Deal Data Replication

The below table shows how each many unique data are replicated across storage providers.

No more than 25% of unique data are stored with less than 4 providers.

⚠️ 100.00% of deals are for data replicated across less than 4 storage providers.

Unique Data Size	Total Deals Made	Number of Providers	Deal Percentage
26.48 TiB	26.48 TiB	1	100.00%

Replication Distribution

Deal Data Shared with other Clients

The below table shows how many unique data are shared with other clients. Usually different applications owns different data and should not resolve to the same CID.

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 3

Multisig Notary address

f01940930

Client address

f1ieughpsgkdynelb2tipfpuhdvcvltfvsuv54q2q

DataCap allocation requested

200TiB

Id

5706dda4-cbe0-4a87-9cd1-b174902b3f3c

large-datacap-requests[bot] commented 1 year ago

Hello @jamerduhgamer - @davidthoms , please sign the datacap request

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f01940930

Client address

f1ieughpsgkdynelb2tipfpuhdvcvltfvsuv54q2q

Last two approvers

Fenbushi-Filecoin & Fatman13

Rule to calculate the allocation request amount

200% of weekly dc amount requested

DataCap allocation requested

200TiB

Total DataCap granted for client so far

50TiB

Datacap to be granted to reach the total amount requested by the client (5PiB)

4.95PiB

Stats

Number of deals	Number of storage providers	Previous DC Allocated	Top provider	Remaining DC
1156	2	50TiB	98.10	7.40TiB

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report[^1]

Organization: AGENCJA PROMOCJI MOTORYZACJI A.P.M.AUTO
Client: f1ieughpsgkdynelb2tipfpuhdvcvltfvsuv54q2q
Storage Provider Distribution

The below table shows the distribution of storage providers that have stored data for this client.

If this is the first time a provider takes verified deal, it will be marked as new.

For most of the datacap application, below restrictions should apply.

Storage provider should not exceed 30% of total datacap.
Storage provider should not be storing duplicate data for more than 20%.
Storage provider should have published its public IP address.
All storage providers should be located in different regions.

⚠️ f01885088 has sealed 99.72% of total datacap.

Provider	Location	Total Deals Sealed	Percentage	Unique Data	Duplicate Deals
f01885088	Hong Kong, Central and Western, HK	32.98 TiB	99.72%	31.30 TiB	5.12%
f01127678	Ursynów, Mazovia, PL	96.00 GiB	0.28%	96.00 GiB	0.00%

Provider Distribution

Deal Data Replication

The below table shows how each many unique data are replicated across storage providers.

No more than 30% of unique data are stored with less than 4 providers.

⚠️ 100.00% of deals are for data replicated across less than 4 storage providers.

Unique Data Size	Total Deals Made	Number of Providers	Deal Percentage
31.39 TiB	33.08 TiB	1	100.00%

Replication Distribution

Deal Data Shared with other Clients

The below table shows how many unique data are shared with other clients. Usually different applications owns different data and should not resolve to the same CID.

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

jamerduhgamer commented 1 year ago

Hello all, just say the ping to sign the datacap. I believe PiKNiK is going to be one of the SPs in the USA that store a replica of this dataset. If so, should I still sign the datacap? Usually this is not allowed but I am asking since this is an enterprise FIL+ LDN there may be an exception I don't know about.

kevzak commented 1 year ago

Yes, in that case, do not sign @jamerduhgamer. The same rules apply, E-Fil will not be an exception. Thank you

donkabat commented 1 year ago

@kevzak What can we do now? By tomorrow ,we will have been waiting for a week for sign the datacap request... :(
@davidthoms? @jamerduhgamer do you know who will replace you?

filecoin-project / filecoin-plus-large-datasets

[DataCap Application] <KABAT> - <Motowizja> #928

Large Dataset Notary Application

Core Information

Project details

Use-case details

DataCap allocation plan

Datacap Request Trigger

Total DataCap requested

Expected weekly DataCap usage rate

Client address

DataCap Allocation requested

Multisig Notary address

Client address

DataCap allocation requested

Id

DataCap Allocation requested

Multisig Notary address

Client address

DataCap allocation requested

Id

Request Proposed

Message sent to Filecoin Network

Address

Datacap Allocated

Signer Address

Id

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacecmoaggklwp2ddeturk6gt74te6soyvkqjyho4ukgqwmpfpmtqhyy

Request Approved

Message sent to Filecoin Network

Address

Datacap Allocated

Signer Address

Id

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceahkwszswzcuwkvc37gewazv4qh76yfqohywzghbnjmzjqqlzb3zo

DataCap and CID Checker Report[^1]

Storage Provider Distribution

Deal Data Replication

Deal Data Shared with other Clients

DataCap Allocation requested

Request number 3

Multisig Notary address

Client address

DataCap allocation requested

Id

Stats & Info for DataCap Allocation

Multisig Notary address

Client address

Last two approvers

Rule to calculate the allocation request amount

DataCap allocation requested

Total DataCap granted for client so far

Datacap to be granted to reach the total amount requested by the client (5PiB)

Stats

DataCap and CID Checker Report[^1]

Storage Provider Distribution

Deal Data Replication

Deal Data Shared with other Clients