search

filecoin-project / filecoin-plus-large-datasets

Hub for client applications for DataCap at a large scale
110 stars 62 forks source link

[DataCap Allocation] - NFT Printer Dataset #1251

Closed CourteousBin closed 1 year ago

CourteousBin commented 1 year ago

name: Large Dataset Notary application about: Clients should use this application form to request a DataCap allocation via a LDN for a dataset title: "NFT Printer Dataset" labels: 'application, Phase: Diligence' assignees: ''

Large Dataset Notary Application

To apply for DataCap to onboard your dataset to Filecoin, please fill out the following.

Core Information

Please respond to the questions below by replacing the text saying "Please answer here". Include as much detail as you can in your answer.

Project details

Share a brief history of your project and organization.

NFT Printer is a unique platform that provides authentication for digital NFT assets as well as its physical counterpart. Contrary to traditional NFT services, we cover up several areas of the ecosystem and welcome all actors on the market to be a part of our community.

Our service is based on the idea of reproducing your digital NFT asset as a pshysical object, and turn it into a trackable item with its data stored in the blockchain. It is a secure, easy and unique way of NFT services that you won’t find anywhere else.

What is the primary source of funding for this project?

Income of the company and investment of shareholders.

What other projects/ecosystem stakeholders is this project associated with?

We indepentdently run the project without any other stakeholder.

Use-case details

Describe the data being stored onto Filecoin

nft picture/video

Where was the data in this dataset sourced from?

customer provided

Can you share a sample of the data? A link to a file, an image, a table, etc., are good ways to do this. 

https://www.nft-printer.net/#/go-print/400
https://www.nft-printer.net/#/go-print/429
https://www.nft-printer.net/#/go-print/477

Confirm that this is a public dataset that can be retrieved by anyone on the Network (i.e., no specific permissions or access rights are required to view the data).

Yes

What is the expected retrieval frequency for this data?

Current retrieval frequency is expected to be twice a year.

For how long do you plan to keep this dataset stored on Filecoin?

About 3 years.

DataCap allocation plan

In which geographies (countries, regions) do you plan on making storage deals?

Europe \ North America

How will you be distributing your data to storage providers? Is there an offline data transfer process?

We will distribute our data to storage providers through offline data transfer.

How do you plan on choosing the storage providers with whom you will be making deals? This should include a plan to ensure the data is retrievable in the future both by you and others.

Look for SPs on slack and community, make sure they are different entities.
And who could prove that they have ability to make data safe and can finish the sealing tasks.

How will you be distributing deals across storage providers?

No single SPs can make deals more than 25%.

Do you have the resources/funding to start making deals as soon as you receive DataCap? What support from the community would help you onboard onto Filecoin?

Of course, the funds are enough. We are happy to continue to make deals with more SPs in the future.
filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report[^1]

Storage Provider Distribution

The below table shows the distribution of storage providers that have stored data for this client.

If this is the first time a provider takes verified deal, it will be marked as new.

For most of the datacap application, below restrictions should apply.

⚠️ 21.25% of total deal sealed by f07830 are duplicate data.

Provider Location Total Deals Sealed Percentage Unique Data Duplicate Deals
f07786new Singapore, Singapore, SG
Akamai Technologies, Inc.		 7.59 TiB 9.70% 7.59 TiB 0.00%
f01915133 Hong Kong, Central and Western, HK
ANYUN INTERNET TECHNOLOGY (HK) CO.,LIMITED		 12.86 TiB 16.43% 11.27 TiB 12.39%
f01915188 Shenzhen, Guangdong, CN
CHINANET-BACKBONE		 12.09 TiB 15.44% 10.55 TiB 12.67%
f01915135 Hong Kong, Central and Western, HK
Diyixian.com Limited		 13.09 TiB 16.73% 11.78 TiB 10.02%
f07830 San Jose, California, US
Tencent Building, Kejizhongyi Avenue		 13.09 TiB 16.72% 10.30 TiB 21.25%
f020522 Frankfurt am Main, Hesse, DE
Tencent Building, Kejizhongyi Avenue		 12.54 TiB 16.03% 12.54 TiB 0.00%
f03325 Ōi, Saitama, JP
The Constant Company, LLC		 7.00 TiB 8.95% 6.85 TiB 2.23%

Provider Distribution

Deal Data Replication

The below table shows how each many unique data are replicated across storage providers.

⚠️ 100.00% of deals are for data replicated across less than 4 storage providers.

Unique Data Size Total Deals Made Number of Providers Deal Percentage
23.59 TiB 26.40 TiB 1 33.73%
13.86 TiB 32.09 TiB 2 40.99%
6.53 TiB 19.78 TiB 3 25.27%

Replication Distribution

Deal Data Shared with other Clients

The below table shows how many unique data are shared with other clients. Usually different applications owns different data and should not resolve to the same CID.

However, this could be possible if all below clients use same software to prepare for the exact same dataset or they belong to a series of LDN applications for the same dataset.

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

cryptowhizzard commented 1 year ago

Hello,

I tried retrieval on your deals but could not get any on the providers here. Can you please contact your SP's and get this fixed?

client-f01986859.csv

CourteousBin commented 1 year ago

@cryptowhizzard Thank you for your feedback.

We have been concerned about this issue and have informed SP to provide the retrieval function.

cryptowhizzard commented 1 year ago

Hi,

Let us know when you have fixed the issue. We cannot assign new datacap requests until this is fixed.

phantom-rabbit commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report[^1]

Storage Provider Distribution

The below table shows the distribution of storage providers that have stored data for this client.

If this is the first time a provider takes verified deal, it will be marked as new.

For most of the datacap application, below restrictions should apply.

⚠️ 26.15% of total deal sealed by f07786 are duplicate data.

Provider Location Total Deals Sealed Percentage Unique Data Duplicate Deals
f07786new Singapore, Singapore, SG
Akamai Technologies, Inc.		 28.79 TiB 15.86% 21.26 TiB 26.15%
f01915188 Shenzhen, Guangdong, CN
CHINANET-BACKBONE		 12.09 TiB 6.66% 10.55 TiB 12.67%
f01915133 Hong Kong, Central and Western, HK
Cloudie Limited		 33.63 TiB 18.53% 32.03 TiB 4.74%
f01915135 Hong Kong, Central and Western, HK
Diyixian.com Limited		 13.09 TiB 7.22% 11.78 TiB 10.02%
f020522 Frankfurt am Main, Hesse, DE
Tencent Building, Kejizhongyi Avenue		 32.98 TiB 18.17% 32.95 TiB 0.09%
f03325 Tokyo, Tokyo, JP
The Constant Company, LLC		 27.93 TiB 15.39% 27.17 TiB 2.70%
f07830 San Jose, California, US
Zenlayer Inc		 32.97 TiB 18.17% 30.13 TiB 8.63%

Provider Distribution

Deal Data Replication

The below table shows how each many unique data are replicated across storage providers.

⚠️ 44.99% of deals are for data replicated across less than 4 storage providers.

Unique Data Size Total Deals Made Number of Providers Deal Percentage
23.56 TiB 26.37 TiB 1 14.53%
13.89 TiB 32.16 TiB 2 17.72%
7.56 TiB 23.12 TiB 3 12.74%
6.79 TiB 27.60 TiB 4 15.21%
12.94 TiB 72.22 TiB 5 39.80%

Replication Distribution

Deal Data Shared with other Clients

The below table shows how many unique data are shared with other clients. Usually different applications owns different data and should not resolve to the same CID.

However, this could be possible if all below clients use same software to prepare for the exact same dataset or they belong to a series of LDN applications for the same dataset.

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

cryptowhizzard commented 1 year ago

You can use the command line : lotus client retrieve --provider < SP > filename.car

cryptowhizzard commented 1 year ago

I checked for you. All retrieval requests are rejected.

ecv 0 B, Paid 0 FIL, Open (New), 0s [1674816302356116589|0] Recv 0 B, Paid 0 FIL, DealProposed (WaitForAcceptance), 9ms [1674816302356116589|0] Recv 0 B, Paid 0 FIL, Open (New), 0s [1674816302356116590|0] Recv 0 B, Paid 0 FIL, DealRejected (RetryLegacy), 473ms [1674816302356116589|0] Recv 0 B, Paid 0 FIL, DealProposed (WaitForAcceptance), 21ms [1674816302356116590|0] Recv 0 B, Paid 0 FIL, DealProposed (WaitForAcceptanceLegacy), 482ms [1674816302356116589|0] Recv 0 B, Paid 0 FIL, Open (New), 0s [1674816302356116591|0] Recv 0 B, Paid 0 FIL, DealProposed (WaitForAcceptance), 11ms [1674816302356116591|0] Recv 0 B, Paid 0 FIL, DealRejected (RetryLegacy), 655ms [1674816302356116590|0] Recv 0 B, Paid 0 FIL, DealProposed (WaitForAcceptanceLegacy), 670ms [1674816302356116590|0] Recv 0 B, Paid 0 FIL, DealRejected (Rejecting), 1.134s [1674816302356116589|0] Recv 0 B, Paid 0 FIL, BlockstoreFinalized (Rejected), 1.134s [1674816302356116589|0] ERROR: Retrieval Proposal Rejected: deal rejected: false

Recv 0 B, Paid 0 FIL, DealRejected (RetryLegacy), 455ms [1674816302356116591|0] Recv 0 B, Paid 0 FIL, DealProposed (WaitForAcceptanceLegacy), 461ms [1674816302356116591|0] Recv 0 B, Paid 0 FIL, DealRejected (Rejecting), 1.292s [1674816302356116590|0] Recv 0 B, Paid 0 FIL, BlockstoreFinalized (Rejected), 1.292s [1674816302356116590|0] ERROR: Retrieval Proposal Rejected: deal rejected: false

Recv 0 B, Paid 0 FIL, DealRejected (Rejecting), 1.233s [1674816302356116591|0] Recv 0 B, Paid 0 FIL, BlockstoreFinalized (Rejected), 1.233s [1674816302356116591|0] ERROR: Retrieval Proposal Rejected: deal rejected: false

CourteousBin commented 1 year ago

@cryptowhizzard The retrieval function has been repaired. Please approve the next round of DC asap

Recv 19.99 GiB, Paid 0 FIL, BlocksReceived (Ongoing), 33m39.771s Recv 19.99 GiB, Paid 0 FIL, BlocksReceived (Ongoing), 33m39.862s Recv 19.99 GiB, Paid 0 FIL, BlocksReceived (Ongoing), 33m39.966s Recv 19.99 GiB, Paid 0 FIL, BlocksReceived (Ongoing), 33m40.054s Recv 19.99 GiB, Paid 0 FIL, BlocksReceived (Ongoing), 33m40.157s Recv 20 GiB, Paid 0 FIL, BlocksReceived (Ongoing), 33m40.244s Recv 20 GiB, Paid 0 FIL, BlocksReceived (Ongoing), 33m40.335s Recv 20 GiB, Paid 0 FIL, BlocksReceived (Ongoing), 33m40.425s Recv 20 GiB, Paid 0 FIL, BlocksReceived (Ongoing), 33m40.509s Recv 20 GiB, Paid 0 FIL, BlocksReceived (Ongoing), 33m40.595s Recv 20 GiB, Paid 0 FIL, BlocksReceived (Ongoing), 33m40.687s Recv 20 GiB, Paid 0 FIL, Complete (CheckComplete), 33m40.768s Recv 20 GiB, Paid 0 FIL, WaitForLastBlocks (WaitingForLastBlocks), 33m40.768s Recv 20 GiB, Paid 0 FIL, BlocksReceived (WaitingForLastBlocks), 33m40.772s Recv 20 GiB, Paid 0 FIL, AllBlocksReceived (FinalizingBlockstore), 33m40.772s Recv 20 GiB, Paid 0 FIL, BlockstoreFinalized (Completed), 33m40.802s Success

lvschouwen commented 1 year ago

Trying to retrieve from MinerID f020522 takes a very long time and seems to hang on after "PaymentChannelSkip".

# lotus client retrieve --provider f020522 mAXCg5AIgTKbp6JvpzqNDRJedS34uUrlQ7g/mTXgdTL9xMPe6HfM tmp
Recv 0 B, Paid 0 FIL, Open (New), 0s [1676276718026808211|0]
Recv 0 B, Paid 0 FIL, DealProposed (WaitForAcceptance), 2ms [1676276718026808211|0]
Recv 0 B, Paid 0 FIL, DealAccepted (Accepted), 765ms [1676276718026808211|0]
Recv 0 B, Paid 0 FIL, PaymentChannelSkip (Ongoing), 765ms [1676276718026808211|0]

Same goes for f07786

# lotus client retrieve --provider f07786 mAXCg5AIgc5h9J33cIYIqVcpYFi5aM45M+d4VTvBvO+jnFFG20Bk tmp
Recv 0 B, Paid 0 FIL, Open (New), 0s [1676276718026808212|0]
Recv 0 B, Paid 0 FIL, DealProposed (WaitForAcceptance), 1ms [1676276718026808212|0]
Recv 0 B, Paid 0 FIL, DealAccepted (Accepted), 321ms [1676276718026808212|0]
Recv 0 B, Paid 0 FIL, PaymentChannelSkip (Ongoing), 322ms [1676276718026808212|0]

@CourteousBin Can you tell me which deal your tried to retrieve that did work?

CourteousBin commented 1 year ago

@lvschouwen

lotus client retrieve --miner f01915133 --from f1qwhdzl4c5mredljdi22npsqd4aoh4b5ehqacdfa bafykbzacech4qxstbspow5opzufzgsonaav66hnefd66eaxua3nsfh7badh5k /data/nft101.tar 2>&1 | tee -a nftretrieve.log

lvschouwen commented 1 year ago

@CourteousBin Can't connect to the miner.

# lotus net connect f01915133
f01915133 -> {12D3KooWPeGXZ1k2b9QLdRhqUtdEkpWftHD8Et6qLfx7rwK9Wf8D: [/ip4/122.10.123.251/tcp/6789]}
# telnet 122.10.123.251 6789
Trying 122.10.123.251...
telnet: Unable to connect to remote host: Connection refused

Cannot verify that data is retrievable at this point.

CourteousBin commented 1 year ago

@lvschouwen Try again

lvschouwen commented 1 year ago

Ok ... now lets wait for the data to come ... Also still waiting on f020522 and f07786 to start sending data

# lotus client retrieve --miner f01915133 bafykbzacech4qxstbspow5opzufzgsonaav66hnefd66eaxua3nsfh7badh5k tmp
Recv 0 B, Paid 0 FIL, Open (New), 0s [1676276718026808216|0]
Recv 0 B, Paid 0 FIL, DealProposed (WaitForAcceptance), 2ms [1676276718026808216|0]
Recv 0 B, Paid 0 FIL, DealAccepted (Accepted), 244ms [1676276718026808216|0]
Recv 0 B, Paid 0 FIL, PaymentChannelSkip (Ongoing), 244ms [1676276718026808216|0]
CourteousBin commented 1 year ago

@lvschouwen What is the outcome of the survey and will we be able to apply for the next round of quotas?

cryptowhizzard commented 1 year ago

The outcome is that i still have trouble doing a retrieval from your side.

Feb 15 08:03:45 proposals dealscanner-f01986859-f01915133-21159093: ERROR: offer error: retrieval query offer errored: failed to fetch piece to retrieve from: getting pieces for cid bafykbzacecpg4nnltmydaftkoyrptvkfibpwgvguejs2soclgxjjcr4wdqbci: getting pieces containing block bafykbzacecpg4nnltmydaftkoyrptvkfibpwgvguejs2soclgxjjcr4wdqbci: failed to lookup index for mh a0e402209e6e35ab9b3030166a7622f9d545405f6354d42265a9384b35d29147961c0224, err: datastore: key not found Feb 15 08:03:45 proposals dealscanner-f01986859-f01915133-21159093: message repeated 2 times: [ ERROR: offer error: retrieval query offer errored: failed to fetch piece to retrieve from: getting pieces for cid bafykbzacecpg4nnltmydaftkoyrptvkfibpwgvguejs2soclgxjjcr4wdqbci: getting pieces containing block bafykbzacecpg4nnltmydaftkoyrptvkfibpwgvguejs2soclgxjjcr4wdqbci: failed to lookup index for mh a0e402209e6e35ab9b3030166a7622f9d545405f6354d42265a9384b35d29147961c0224, err: datastore: key not found] Feb 15 08:03:45 proposals dealscanner-f01986859-f01915133-21159093: Feb 15 08:03:45 proposals dealscanner-f01986859-f01915133-21159093: message repeated 2 times: [ ] Feb 15 08:03:45 proposals dealscanner-f01986859-f01915133-21159093: ERROR: offer error: retrieval query offer errored: failed to fetch piece to retrieve from: getting pieces for cid bafykbzacecpg4nnltmydaftkoyrptvkfibpwgvguejs2soclgxjjcr4wdqbci: getting pieces containing block bafykbzacecpg4nnltmydaftkoyrptvkfibpwgvguejs2soclgxjjcr4wdqbci: failed to lookup index for mh a0e402209e6e35ab9b3030166a7622f9d545405f6354d42265a9384b35d29147961c0224, err: datastore: key not found Feb 15 08:03:45 proposals dealscanner-f01986859-f01915133-21159093: Feb 15 08:03:45 proposals dealscanner-f01986859-f01915133-22358766: ERROR: offer error: retrieval query offer errored: failed to fetch piece to retrieve from: getting pieces for cid bafykbzacebgzvttnnimledbdya5iuvdvozvk2rhithaxxfdfmr7syta7vshe2: getting pieces containing block bafykbzacebgzvttnnimledbdya5iuvdvozvk2rhithaxxfdfmr7syta7vshe2: failed to lookup index for mh a0e402204d9ace6d6a18b20c23c03a8a5475766aad44e899c17b9465647f2c4c1fac8e4d, err: datastore: key not found Feb 15 08:03:45 proposals dealscanner-f01986859-f01915133-22358766: Feb 15 08:03:45 proposals dealscanner-f01986859-f01915133-21159093: ERROR: offer error: retrieval query offer errored: failed to fetch piece to retrieve from: getting pieces for cid bafykbzacecpg4nnltmydaftkoyrptvkfibpwgvguejs2soclgxjjcr4wdqbci: getting pieces containing block bafykbzacecpg4nnltmydaftkoyrptvkfibpwgvguejs2soclgxjjcr4wdqbci: failed to lookup index for mh a0e402209e6e35ab9b3030166a7622f9d545405f6354d42265a9384b35d29147961c0224, err: datastore: key not found

Can you fix?

CourteousBin commented 1 year ago

@cryptowhizzard Hello Notary, can’t wait to tell you that after several weeks of communication with SP, the search function of f01915133 has been fixed, please confirm so that the next round of allocation can be approved, thank you for your support.

cryptowhizzard commented 1 year ago

I am really sorry, but i still get the same error. Perhaps another notary can try retrieval?

Mar 9 14:21:47 proposals dealscanner-f01986859-f01915133: Recv 0 B, Paid 0 FIL, Open (New), 1ms [1678031108993914614|0] Mar 9 14:21:47 proposals dealscanner-f01986859-f01915133: Recv 0 B, Paid 0 FIL, Open (New), 0s [1678031108993914615|0] Mar 9 14:21:47 proposals dealscanner-f01986859-f01915133: Recv 0 B, Paid 0 FIL, DealProposed (WaitForAcceptance), 5ms [1678031108993914614|0] Mar 9 14:21:47 proposals dealscanner-f01986859-f01915133: Recv 0 B, Paid 0 FIL, DealProposed (WaitForAcceptance), 4ms [1678031108993914615|0] Mar 9 14:21:48 proposals dealscanner-f01986859-f01915133-21159093: Recv 0 B, Paid 0 FIL, Open (New), 0s [1678020242222140012|0] Mar 9 14:21:48 proposals dealscanner-f01986859-f01915133-21159093: Recv 0 B, Paid 0 FIL, DealProposed (WaitForAcceptance), 43ms [1678020242222140012|0] Mar 9 14:21:48 proposals dealscanner-f01986859-f01915133-21159093: Recv 0 B, Paid 0 FIL, DealAccepted (Accepted), 340ms [1678020242222140012|0] Mar 9 14:21:48 proposals dealscanner-f01986859-f01915133-21159093: Recv 0 B, Paid 0 FIL, PaymentChannelSkip (Ongoing), 340ms [1678020242222140012|0] Mar 9 14:21:48 proposals dealscanner-f01986859-f01915133-22358766: Recv 0 B, Paid 0 FIL, Open (New), 0s [1678020242222140013|0] Mar 9 14:21:48 proposals dealscanner-f01986859-f01915133-22358766: Recv 0 B, Paid 0 FIL, DealProposed (WaitForAcceptance), 28ms [1678020242222140013|0] Mar 9 14:21:49 proposals dealscanner-f01986859-f07830: Recv 0 B, Paid 0 FIL, Open (New), 0s [1678031108993914616|0] Mar 9 14:21:49 proposals dealscanner-f01986859-f07830: Recv 0 B, Paid 0 FIL, DealProposed (WaitForAcceptance), 3ms [1678031108993914616|0] Mar 9 14:21:49 proposals dealscanner-f01986859-f01915133-22358766: Recv 0 B, Paid 0 FIL, DealAccepted (Accepted), 301ms [1678020242222140013|0] Mar 9 14:21:49 proposals dealscanner-f01986859-f01915133-22358766: Recv 0 B, Paid 0 FIL, PaymentChannelSkip (Ongoing), 302ms [1678020242222140013|0] Mar 9 14:21:49 proposals dealscanner-f01986859-f07830: Error: Failed to retrieve content with candidate miner f07830: data transfer failed: deal errored: getting pieces for cid bafykbzacecm2cef6iojrvqniloah7eukkeqecmbjuadbjojkvljtm2iaizago: getting pieces containing block bafykbzacecm2cef6iojrvqniloah7eukkeqecmbjuadbjojkvljtm2iaizago: failed to lookup index for mh a0e4022099a110be43931ac1a85b807f928a5120413029a00614b92aaad3366900464067, err: datastore: key not found Mar 9 14:21:49 proposals dealscanner-f01986859-f01915133-21159093: Recv 0 B, Paid 0 FIL, Open (New), 0s [1678020242222140014|0] Mar 9 14:21:49 proposals dealscanner-f01986859-f01915133-21159093: Recv 0 B, Paid 0 FIL, DealProposed (WaitForAcceptance), 22ms [1678020242222140014|0] Mar 9 14:21:49 proposals dealscanner-f01986859-f01915133-21159093: 2023-03-09T14:21:49.374+0100#011WARN#011rpc#011go-jsonrpc@v0.2.1/client.go:392#011rpc output message buffer#011{"n": 11} Mar 9 14:21:49 proposals dealscanner-f01986859-f01915133-21159093: 2023-03-09T14:21:49.374+0100#011WARN#011rpc#011go-jsonrpc@v0.2.1/client.go:392#011rpc output message buffer#011{"n": 11} Mar 9 14:21:49 proposals dealscanner-f01986859-f01915133-21159093: Recv 0 B, Paid 0 FIL, DealAccepted (Accepted), 278ms [1678020242222140014|0] Mar 9 14:21:49 proposals dealscanner-f01986859-f01915133-21159093: 2023-03-09T14:21:49.374+0100#011WARN#011rpc#011go-jsonrpc@v0.2.1/client.go:392#011rpc output message buffer#011{"n": 11} Mar 9 14:21:49 proposals dealscanner-f01986859-f01915133-21159093: 2023-03-09T14:21:49.375+0100#011WARN#011rpc#011go-jsonrpc@v0.2.1/client.go:392#011rpc output message buffer#011{"n": 12} Mar 9 14:21:49 proposals dealscanner-f01986859-f01915133-21159093: Recv 0 B, Paid 0 FIL, PaymentChannelSkip (Ongoing), 279ms [1678020242222140014|0] Mar 9 14:21:49 proposals dealscanner-f01986859-f01915133-21159093: 2023-03-09T14:21:49.375+0100#011WARN#011rpc#011go-jsonrpc@v0.2.1/client.go:392#011rpc output message buffer#011{"n": 12} Mar 9 14:21:49 proposals dealscanner-f01986859-f01915133-21159093: 2023-03-09T14:21:49.375+0100#011WARN#011rpc#011go-jsonrpc@v0.2.1/client.go:392#011rpc output message buffer#011{"n": 12} Mar 9 14:21:49 proposals dealscanner-f01986859-f07830: Recv 0 B, Paid 0 FIL, DealAccepted (Accepted), 366ms [1678031108993914616|0] Mar 9 14:21:49 proposals dealscanner-f01986859-f07830: Recv 0 B, Paid 0 FIL, PaymentChannelSkip (Ongoing), 366ms [1678031108993914616|0] Mar 9 14:21:49 proposals dealscanner-f01986859-f07830: Recv 0 B, Paid 0 FIL, ProviderCancelled (Cancelling), 395ms [1678031108993914616|0] Mar 9 14:21:49 proposals dealscanner-f01986859-f07830: Recv 0 B, Paid 0 FIL, CancelComplete (Cancelled), 395ms [1678031108993914616|0] Mar 9 14:21:49 proposals dealscanner-f01986859-f07830: ERROR: Retrieval Proposal Cancelled: Provider cancelled retrieval Mar 9 14:21:49 proposals dealscanner-f01986859-f07830: Mar 9 14:21:49 proposals dealscanner-f01986859-f01915133-21159093: Recv 0 B, Paid 0 FIL, Open (New), 0s [1678020242222140015|0] Mar 9 14:21:49 proposals dealscanner-f01986859-f01915133-21159093: 2023-03-09T14:21:49.453+0100#011WARN#011rpc#011go-jsonrpc@v0.2.1/client.go:392#011rpc output message buffer#011{"n": 13} Mar 9 14:21:49 proposals dealscanner-f01986859-f01915133-21159093: 2023-03-09T14:21:49.453+0100#011WARN#011rpc#011go-jsonrpc@v0.2.1/client.go:392#011rpc output message buffer#011{"n": 13} Mar 9 14:21:49 proposals dealscanner-f01986859-f03325-22439907: ERROR: offer error: retrieval query offer errored: failed to fetch piece to retrieve from: getting pieces for cid bafykbzacebrhhjf3lsoasiuvtd54s3caohtof2bnrei7h6ziiq3znt35va2vk: getting pieces containing block bafykbzacebrhhjf3lsoasiuvtd54s3caohtof2bnrei7h6ziiq3znt35va2vk: failed to lookup index for mh a0e402206273a4bb5c9c09229598fbc96c4071e6e2e82d8911f3fb28443796cf7da83555, err: datastore: key not found Mar 9 14:21:49 proposals dealscanner-f01986859-f03325-22439907: Mar 9 14:21:49 proposals dealscanner-f01986859-f01915133-21159093: 2023-03-09T14:21:49.476+0100#011WARN#011rpc#011go-jsonrpc@v0.2.1/client.go:392#011rpc output message buffer#011{"n": 14} Mar 9 14:21:49 proposals dealscanner-f01986859-f01915133-21159093: Recv 0 B, Paid 0 FIL, DealProposed (WaitForAcceptance), 22ms [1678020242222140015|0] Mar 9 14:21:49 proposals dealscanner-f01986859-f01915133-21159093: 2023-03-09T14:21:49.476+0100#011WARN#011rpc#011go-jsonrpc@v0.2.1/client.go:392#011rpc output message buffer#011{"n": 14} Mar 9 14:21:49 proposals dealscanner-f01986859-f01915133-21159093: 2023-03-09T14:21:49.752+0100#011WARN#011rpc#011go-jsonrpc@v0.2.1/client.go:392#011rpc output message buffer#011{"n": 15} Mar 9 14:21:49 proposals dealscanner-f01986859-f01915133-21159093: Recv 0 B, Paid 0 FIL, DealAccepted (Accepted), 298ms [1678020242222140015|0] Mar 9 14:21:49 proposals dealscanner-f01986859-f01915133-21159093: 2023-03-09T14:21:49.753+0100#011WARN#011rpc#011go-jsonrpc@v0.2.1/client.go:392#011rpc output message buffer#011{"n": 15} Mar 9 14:21:49 proposals dealscanner-f01986859-f01915133-21159093: 2023-03-09T14:21:49.753+0100#011WARN#011rpc#011go-jsonrpc@v0.2.1/client.go:392#011rpc output message buffer#011{"n": 16} Mar 9 14:21:49 proposals dealscanner-f01986859-f01915133-21159093: Recv 0 B, Paid 0 FIL, PaymentChannelSkip (Ongoing), 299ms [1678020242222140015|0] Mar 9 14:21:49 proposals dealscanner-f01986859-f01915133-21159093: 2023-03-09T14:21:49.753+0100#011WARN#011rpc#011go-jsonrpc@v0.2.1/client.go:392#011rpc output message buffer#011{"n": 16} Mar 9 14:21:49 proposals dealscanner-f01986859-f01915133-21159093: 2023-03-09T14:21:49.802+0100#011WARN#011rpc#011go-jsonrpc@v0.2.1/client.go:392#011rpc output message buffer#011{"n": 17} Mar 9 14:21:49 proposals dealscanner-f01986859-f01915133-21159093: Recv 0 B, Paid 0 FIL, Open (New), 0s [1678020242222140016|0] Mar 9 14:21:49 proposals dealscanner-f01986859-f01915133-21159093: Recv 0 B, Paid 0 FIL, DealProposed (WaitForAcceptance), 22ms [1678020242222140016|0] Mar 9 14:21:49 proposals dealscanner-f01986859-f01915133-21159093: 2023-03-09T14:21:49.824+0100#011WARN#011rpc#011go-jsonrpc@v0.2.1/client.go:392#011rpc output message buffer#011{"n": 18} Mar 9 14:21:50 proposals dealscanner-f01986859-f01915133-21159093: Recv 0 B, Paid 0 FIL, DealAccepted (Accepted), 286ms [1678020242222140016|0] Mar 9 14:21:50 proposals dealscanner-f01986859-f01915133-21159093: 2023-03-09T14:21:50.088+0100#011WARN#011rpc#011go-jsonrpc@v0.2.1/client.go:392#011rpc output message buffer#011{"n": 19} Mar 9 14:21:50 proposals dealscanner-f01986859-f01915133-21159093: Recv 0 B, Paid 0 FIL, PaymentChannelSkip (Ongoing), 286ms [1678020242222140016|0] Mar 9 14:21:50 proposals dealscanner-f01986859-f01915133-21159093: 2023-03-09T14:21:50.089+0100#011WARN#011rpc#011go-jsonrpc@v0.2.1/client.go:392#011rpc output message buffer#011{"n": 20} Mar 9 14:21:50 proposals dealscanner-f01986859-f01915133-21159093: Recv 0 B, Paid 0 FIL, Open (New), 0s [1678020242222140017|0] Mar 9 14:21:50 proposals dealscanner-f01986859-f01915133-21159093: Recv 0 B, Paid 0 FIL, DealProposed (WaitForAcceptance), 29ms [1678020242222140017|0] Mar 9 14:21:50 proposals dealscanner-f01986859-f01915133-21159093: Recv 0 B, Paid 0 FIL, DealAccepted (Accepted), 300ms [1678020242222140017|0] Mar 9 14:21:50 proposals dealscanner-f01986859-f01915133-21159093: Recv 0 B, Paid 0 FIL, PaymentChannelSkip (Ongoing), 301ms [1678020242222140017|0]

Joss-Hua commented 1 year ago

image can't connect to these sps.

CourteousBin commented 1 year ago

@Joss-Hua Hello Notary, the test normal, you can try again image

image

CourteousBin commented 1 year ago

@Joss-Hua Hello notary, could you please verify again?

Joss-Hua commented 1 year ago 
LOTUS_PATH=./.lotus/ ./lotus client query-ask f07786
2023-03-13T14:02:31.939+0800    ERROR   rpc go-jsonrpc@v0.1.8/websocket.go:667  Connection timeout  {"remote": "47.115.22.33:1234"}
2023-03-13T14:02:31.940+0800    WARN    rpc go-jsonrpc@v0.1.8/websocket.go:678  failed to write close message: write tcp 192.168.19.112:57403->47.115.22.33:1234: use of closed network connection
2023-03-13T14:02:31.940+0800    WARN    rpc go-jsonrpc@v0.1.8/websocket.go:681  websocket close error   {"error": "close tcp 192.168.19.112:57403->47.115.22.33:1234: use of closed network connection"}
ERROR: %!s(PANIC=Error method: runtime error: invalid memory address or nil pointer dereference)

LOTUS_PATH=./.lotus/ ./lotus client query-ask f01915135
ERROR: %!s(PANIC=Error method: runtime error: invalid memory address or nil pointer dereference)

f07830 is ready, here is the rest.

CourteousBin commented 1 year ago

HI @Joss-Hua, after we tried, the query-ask is returned normally, and the static address and port are also listed in the figure, please check whether your node network is reachable

image image
Joss-Hua commented 1 year ago

cool, will support this round. Keep watching and hope that will get better to better.

Joss-Hua commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzaceaemkimm2y3ndp5b2dpzphmxfgb355r25fzqg6gilgbonmewf2c6a

Address

f1qwhdzl4c5mredljdi22npsqd4aoh4b5ehqacdfa

Datacap Allocated

212.00TiB

Signer Address

f1tfg54zzscugttejv336vivknmsnzzmyudp3t7wi

Id

f79bf203-6047-45d4-a519-03704ea71470

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceaemkimm2y3ndp5b2dpzphmxfgb355r25fzqg6gilgbonmewf2c6a

CourteousBin commented 1 year ago

Hi, @simonkim0515 @Kevin-FF-USA @Alex11801 @IreneYoung @yuwenhui Dear Notaries! Please take a look at our program. Our allocation has been requested already. Are there any honorable Notaries who can sign this for us?Really need your support to move on! Thanks a lot!

GaryGJG commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

⚠️ 1 storage providers sealed too much duplicate data - f07786: 26.15%

Deal Data Replication

⚠️ 44.99% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the full report.

maxvint commented 1 year ago

The distribution of SPs looks good, and retrieval also works very well. Please notice the deal data replication percentage in the filplus-checker-report mentioned. Happy to support this round, keep going.

CourteousBin commented 1 year ago

@yuwenhui Thank you for your support, we will maintain compliant distribution and support fast retrieval.

maxvint commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacedpvgw5wnwyqmol7peqk4r35xcpo2aw4rnowjr5u57di7wiag44u2

Address

f1qwhdzl4c5mredljdi22npsqd4aoh4b5ehqacdfa

Datacap Allocated

212.00TiB

Signer Address

f1ui5iy4mmkxjbw7752omiwp2ols2fzv4thrayagi

Id

f79bf203-6047-45d4-a519-03704ea71470

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedpvgw5wnwyqmol7peqk4r35xcpo2aw4rnowjr5u57di7wiag44u2

laurarenpanda commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaceaxcugohoxw4ieumf6cl7vuq3k3olut7xpgpa5uippiw6icuyvg7e

Address

f1qwhdzl4c5mredljdi22npsqd4aoh4b5ehqacdfa

Datacap Allocated

212.00TiB

Signer Address

f1bp3tzp536edm7dodldceekzbsx7zcy7hdfg6uzq

Id

f79bf203-6047-45d4-a519-03704ea71470

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceaxcugohoxw4ieumf6cl7vuq3k3olut7xpgpa5uippiw6icuyvg7e

phantom-rabbit commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

⚠️ 1 storage providers sealed too much duplicate data - f07786: 22.18%

Deal Data Replication

⚠️ 50.20% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the full report.

phantom-rabbit commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

⚠️ 37.54% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the full report.

phantom-rabbit commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

⚠️ 32.43% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the full report.

phantom-rabbit commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

⚠️ 38.43% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the full report.

phantom-rabbit commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the full report.

Normalnoise commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the full report.

Normalnoise commented 1 year ago

@CourteousBin cid checker has tell me your storage is healthy, but I think you need to send deals to more storage providers and reduce the duplicate data in the single SP. I suggest you can using swan-provider to find more SPs around the world. if you have some questions, you can visit the #fil-filswan channel in the slack. Based on the current situation, I am willing to support you in this round

lvschouwen commented 1 year ago

@CourteousBin cid checker has tell me your storage is healthy, but I think you need to send deals to more storage providers and reduce the duplicate data in the single SP. I suggest you can using swan-provider to find more SPs around the world. if you have some questions, you can visit the #fil-filswan channel in the slack. Based on the current situation, I am willing to support you in this round

Is this a commercial? Because I have yet to see your proposal message.

CourteousBin commented 1 year ago

@Normalnoise Thank you for your suggestion. We will look for other SPs around the world and check their retrieval capabilities to allocate our DC.

Normalnoise commented 1 year ago

@CourteousBin cid checker has tell me your storage is healthy, but I think you need to send deals to more storage providers and reduce the duplicate data in the single SP. I suggest you can using swan-provider to find more SPs around the world. if you have some questions, you can visit the #fil-filswan channel in the slack. Based on the current situation, I am willing to support you in this round

Is this a commercial? Because I have yet to see your proposal message.

@lvschouwen no, as a notary, I just suggest @CourteousBin distribute data to more SPs to make the data more stable. @CourteousBin sorry, I met some bugs about the sign message I have reported it to Philippe, maybe signing will be delay

image

CourteousBin commented 1 year ago

@Normalnoise

Thank you for your feedback, we will check where the problem lies.

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 5

Multisig Notary address

f02049625

Client address

f1qwhdzl4c5mredljdi22npsqd4aoh4b5ehqacdfa

DataCap allocation requested

132.5TiB

Id

bf7612ac-0c35-442b-919d-c9f310fc4e9e