Closed nicelove666 closed 11 months ago
DD: retrieval rate is increasing, as client said. LGTM. As far as I know, different sp can indeed use the same port. Nodes under the same port may belong to the same person, but in most cases, the same port can be shared by multiple companies. This is a common phenomenon in Chinese IDC. Therefore, it is not sufficient to prove that these nodes belong to the same sp based on the port alone. In addition, this client contacted me through Slack and described the plan, and I will continue to follow up on whether it is carried out according to what the client disclosed. Will be marked in the notary background.
Your Datacap Allocation Request has been proposed by the Notary
bafy2bzacebpcel5ncpu6zkxaqn7zji36v3wur5x2yzt3agzgk34ujqscan6wy
Address
f1y5mkyvzsfxsapuecbbs4hrrmio2te6ajdqpgedq
Datacap Allocated
1.00PiB
Signer Address
f12tk3adljauwnd3hjbigpfxb7b7gdlj63p6afwtq
Id
b01c22b9-65cf-4527-922a-892e76148d44
You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebpcel5ncpu6zkxaqn7zji36v3wur5x2yzt3agzgk34ujqscan6wy
Your Datacap Allocation Request has been proposed by the Notary
bafy2bzacecbjwopa54fzqncudhgypdflunfhc32dyrvljpid6y6j5ack4cez2
Address
f1y5mkyvzsfxsapuecbbs4hrrmio2te6ajdqpgedq
Datacap Allocated
1.00PiB
Signer Address
f1n5wlrrhoxpkgwij25xrtt7w7g2k3fhbthmdn6ri
Id
b01c22b9-65cf-4527-922a-892e76148d44
You can check the status of the message here: https://filfox.info/en/message/bafy2bzacecbjwopa54fzqncudhgypdflunfhc32dyrvljpid6y6j5ack4cez2
checker:manualTrigger
✔️ Storage provider distribution looks healthy.
✔️ Data replication looks healthy.
✔️ No CID sharing has been observed.
[^1]: To manually trigger this report, add a comment with text checker:manualTrigger
[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger
[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...
Click here to view the CID Checker report. Click here to view the Retrieval report.
Clients contact us and promise they will improve the retrieval rate, I will support this round
Your Datacap Allocation Request has been approved by the Notary
bafy2bzacecsll5j6bf73bz74hi7f665duoghvru76b722a6xonizsv6k6odrk
Address
f1y5mkyvzsfxsapuecbbs4hrrmio2te6ajdqpgedq
Datacap Allocated
1.00PiB
Signer Address
f1xrnysd4gimg64d4l6qi7ulzwwq22c6vfg6lpw3i
Id
b01c22b9-65cf-4527-922a-892e76148d44
You can check the status of the message here: https://filfox.info/en/message/bafy2bzacecsll5j6bf73bz74hi7f665duoghvru76b722a6xonizsv6k6odrk
This application has been signed while there is ongoing dispute here.
https://github.com/filecoin-project/notary-governance/issues/917
f02049625
f1y5mkyvzsfxsapuecbbs4hrrmio2te6ajdqpgedq
1PiB
596c23c0-4e24-4ab5-ac79-70ebb6639af3
f02049625
f1y5mkyvzsfxsapuecbbs4hrrmio2te6ajdqpgedq
200% weekly > 1PiB, requesting 1PiB
1PiB
465661.3YiB
465661.3YiB
Number of deals | Number of storage providers | Previous DC Allocated | Top provider | Remaining DC |
---|---|---|---|---|
30889 | 11 | 512TiB | 17.79 | 123.81TiB |
I am trying to download some of your data but it is not succesful. What is going on?
lotus client retrieve --provider=f02194766 bafykbzaceaqb3d456nv2au7u6z7ncwsms6spmayzjbc6plmymzyn6uybqxq2e -o 2050-f02227756-baga6ea4seaqkfvwyj4kv25k7iosjtthkyjdattv4afn5mefyrcuueajwr6ypkhi^C You have new mail in /var/mail/root root@proposals:~/downloads# lotus state get-deal 46914916 { "Proposal": { "PieceCID": { "/": "baga6ea4seaqpldz4glwpd6g2ncrjepjgu7idc5zs4dkpf4k2xibqqq6qcxz5cha" }, "PieceSize": 68719476736, "VerifiedDeal": true, "Client": "f02224404", "Provider": "f02230977", "Label": "bafykbzacecckpsinz4lve2wkjjn2iumtrjtlsri3n7s5hnv6twppydtaceakq", "StartEpoch": 3108200, "EndEpoch": 4116200, "StoragePricePerEpoch": "0", "ProviderCollateral": "20386236246409368", "ClientCollateral": "0" }, "State": { "SectorStartEpoch": 3052668, "LastUpdatedEpoch": -1, "SlashEpoch": -1, "VerifiedClaim": 29227031 } } root@proposals:~/downloads# lotus client retrieve --provider f02230977 bafykbzacecckpsinz4lve2wkjjn2iumtrjtlsri3n7s5hnv6twppydtaceakq bafykbzacecckpsinz4lve2wkjjn2iumtrjtlsri3n7s5hnv6twppydtaceakq.car ERROR: offer error: retrieval query offer errored: failed to fetch piece to retrieve from: getting pieces for cid bafykbzacecckpsinz4lve2wkjjn2iumtrjtlsri3n7s5hnv6twppydtaceakq: getting pieces containing block bafykbzacecckpsinz4lve2wkjjn2iumtrjtlsri3n7s5hnv6twppydtaceakq: failed to lookup index for mh a0e4022084a7c90dcf17526aca4a5ba451938a66b9451b6fe5d3b6be9d9efc0e601100a8, err: datastore: key not found
Second attempt:
root@proposals:~/downloads# lotus client retrieve --provider f02230977 bafykbzacecckpsinz4lve2wkjjn2iumtrjtlsri3n7s5hnv6twppydtaceakq bafykbzacecckpsinz4lve2wkjjn2iumtrjtlsri3n7s5hnv6twppydtaceakq.car ERROR: offer error: retrieval query offer errored: failed to fetch piece to retrieve from: getting pieces for cid bafykbzacecckpsinz4lve2wkjjn2iumtrjtlsri3n7s5hnv6twppydtaceakq: getting pieces containing block bafykbzacecckpsinz4lve2wkjjn2iumtrjtlsri3n7s5hnv6twppydtaceakq: failed to lookup index for mh a0e4022084a7c90dcf17526aca4a5ba451938a66b9451b6fe5d3b6be9d9efc0e601100a8, err: datastore: key not found
root@proposals:~/downloads# lotus state get-deal 46914913 { "Proposal": { "PieceCID": { "/": "baga6ea4seaqnxnu4bfwh6jnsj7ol46mr3gyi2qhnithcwovn2cppbemrt37ogly" }, "PieceSize": 68719476736, "VerifiedDeal": true, "Client": "f02224404", "Provider": "f02230977", "Label": "bafykbzacectqvbk4xx4hqheliy6mkhccty4opzhzhtxu3srgtkd3owzxqgdpy", "StartEpoch": 3108197, "EndEpoch": 4116197, "StoragePricePerEpoch": "0", "ProviderCollateral": "20386236246409368", "ClientCollateral": "0" }, "State": { "SectorStartEpoch": 3054308, "LastUpdatedEpoch": -1, "SlashEpoch": -1, "VerifiedClaim": 29227028 } } You have new mail in /var/mail/root root@proposals:~/downloads# lotus client retrieve --provider f02230977 bafykbzacectqvbk4xx4hqheliy6mkhccty4opzhzhtxu3srgtkd3owzxqgdpy bafykbzacectqvbk4xx4hqheliy6mkhccty4opzhzhtxu3srgtkd3owzxqgdpy.car ERROR: offer error: retrieval query offer errored: failed to fetch piece to retrieve from: getting pieces for cid bafykbzacectqvbk4xx4hqheliy6mkhccty4opzhzhtxu3srgtkd3owzxqgdpy: getting pieces containing block bafykbzacectqvbk4xx4hqheliy6mkhccty4opzhzhtxu3srgtkd3owzxqgdpy: failed to lookup index for mh a0e40220a70a855cbdf8781c8b463cc51c429e38e7e4f93cef4dca269a87b75b378186fc, err: datastore: key not found
Third attempt:
root@proposals:~/downloads# lotus state get-deal 46911128 { "Proposal": { "PieceCID": { "/": "baga6ea4seaqk76xr65v3s6mfndjwfu5y4k3qil4i2qfn5a2ibb5sgczhuy7tkga" }, "PieceSize": 68719476736, "VerifiedDeal": true, "Client": "f02224404", "Provider": "f02230983", "Label": "bafykbzacebb5ikaonopnoex4cc2so54w6uzylt6w2z46l76ixu33qkjwpao76", "StartEpoch": 3108001, "EndEpoch": 4116001, "StoragePricePerEpoch": "0", "ProviderCollateral": "20386267662041523", "ClientCollateral": "0" }, "State": { "SectorStartEpoch": 3053340, "LastUpdatedEpoch": -1, "SlashEpoch": -1, "VerifiedClaim": 29223243 } } root@proposals:~/downloads# lotus client retrieve --provider f02230983 bafykbzacebb5ikaonopnoex4cc2so54w6uzylt6w2z46l76ixu33qkjwpao76 bafykbzacebb5ikaonopnoex4cc2so54w6uzylt6w2z46l76ixu33qkjwpao76.car ERROR: offer error: retrieval query offer errored: failed to fetch piece to retrieve from: getting pieces for cid bafykbzacebb5ikaonopnoex4cc2so54w6uzylt6w2z46l76ixu33qkjwpao76: getting pieces containing block bafykbzacebb5ikaonopnoex4cc2so54w6uzylt6w2z46l76ixu33qkjwpao76: failed to lookup index for mh a0e4022043d4280e6b9ed712fc10b5277796f53385cfd6d679e5ffc8bd37b82936781dff, err: datastore: key not found
@nicelove666 Please contact Sp to confirm why it cannot be retrieved, and deal with it immediately to cooperate with @cryptowhiizzard
@cryptowhizzard @zcfil Thank you for the reminder, we are contacting these SP repairs, we will work hard to fix them
In addition, this SP is not the SP we cooperated with. There may be some misunderstandings here. Of course, the other two SPs are indeed cooperated with us. After receiving your message, we have notified them immediately, thank you again.
checker:manualTrigger
⚠️ 1 storage providers have unknown IP location - f02246008
✔️ Data replication looks healthy.
✔️ No CID sharing has been observed.
[^1]: To manually trigger this report, add a comment with text checker:manualTrigger
[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger
[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...
Click here to view the CID Checker report. Click here to view the Retrieval report.
Hello, f02246008 (100%), f02200472 (100%), f01435542 (40%) We tested the retrieval success rate is very high, it is recommended to retrieve and view the data from here, the 2 problematic nodes you pointed out, we have checked the problem of the network, our network engineer returned to his hometown today, I have asked him to come back as soon as possible, we will try to fix it tomorrow, at the latest, we will fix it this Monday, we will fix it in the We will let you know later, thanks for your patience.
checker:manualTrigger
✔️ Storage provider distribution looks healthy.
✔️ Data replication looks healthy.
✔️ No CID sharing has been observed.
[^1]: To manually trigger this report, add a comment with text checker:manualTrigger
[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger
[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...
Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.
Currently, SP retrieval has been fixed. However, in the past few days, SP has continued to receive a large number of malicious searches, which caused boost to be mounted, and thus made the search unavailable. I don't know who is maliciously harming SP, please stop doing evil.
checker:manualTrigger
✔️ Storage provider distribution looks healthy.
✔️ Data replication looks healthy.
✔️ No CID sharing has been observed.
[^1]: To manually trigger this report, add a comment with text checker:manualTrigger
[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger
[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...
Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.
@nicelove666 "Currently, SP retrieval has been fixed." Can you show us results, we love to check the data for its validity. Thank you!
Please note this is a Microsoft IP adres. If you are bothered then send them a e-mail to abuse@microsoft.com or the email adres in the Ripe database for network abuse.
目前,SP检索已得到修复。 但连日来,SP持续收到大量恶意搜索,导致boost被挂载,从而导致搜索不可用。不知道是谁在恶意伤害SP,请停止作恶。
I believe you have seen my message.
May I ask who is always maliciously downloading data, causing multiple sp boosts to continue to crash, resulting in a relatively low retrieval rate?
Please be kind to SP.
Finally, in the event of malicious attacks, our retrieval rate is still improving.
Again, you need to contact microsoft and ask them. This IP address ( 20.10.1.43 ) is a Microsoft IP.
I don't want to attack anyone, I want everyone to be nice, however, sp has given us feedback many times that whenever they fix the retrievals, they receive tons of malicious retrievals that aren't for 1 hour or 6 hours lasts for 1 day, 2 days, or 3 days. This often causes boost to hang up, orders are unstable, network crashes, and luck values are affected. Every time this happens, it can be solved only if the SP closes the search entry, but once the SP opens the search, a large number of malicious attacks will continue to appear.
Please be kind !The current community rule is that the retrieval rate is greater than 1%.Even with a large number of malicious attacks, we can still maintain a retrieval rate of about 15%, so the attack is meaningless. Please be kind!
I don't want to attack anyone, I want everyone to be nice, however, sp has given us feedback many times that whenever they fix the retrievals, they receive tons of malicious retrievals that aren't for 1 hour or 6 hours lasts for 1 day, 2 days, or 3 days. This often causes boost to hang up, orders are unstable, network crashes, and luck values are affected. Every time this happens, it can be solved only if the SP closes the search entry, but once the SP opens the search, a large number of malicious attacks will continue to appear.
Please be kind !The current community rule is that the retrieval rate is greater than 1%.Even with a large number of malicious attacks, we can still maintain a retrieval rate of about 15%, so the attack is meaningless. Please be kind!
We have miscommunication going on. I will try more clear language.
Please also talk to the boost team on slack in #boost-help so they know what is going on and can improve the code to make boost more resilent against these attacks.
Nice weekend!
checker:manualTrigger
✔️ Storage provider distribution looks healthy.
✔️ Data replication looks healthy.
✔️ No CID sharing has been observed.
[^1]: To manually trigger this report, add a comment with text checker:manualTrigger
[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger
[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...
Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.
CID check report shows this application is healthy, maybe the retrieval success rate should be improved
我不想攻击任何人,我希望每个人都友善,但是,sp多次给我们反馈,每当他们修复检索时,他们都会收到大量恶意检索,这些恶意检索不是持续1小时或6小时的1天、2天或3天。这往往会导致boost挂掉、订单不稳定、网络崩溃、运气值受到影响。每次出现这种情况,只有SP关闭搜索入口才能解决,但一旦SP打开搜索,大量的恶意攻击就会不断出现。 请善待!目前的社区规则是检索率大于1%。即使有大量的恶意攻击,我们仍然可以保持15%左右的检索率,所以攻击是没有意义的。请善待!
我们之间存在沟通不畅的情况。我会尝试更清晰的语言。
- 我会将检索标记为“确定”,因为我知道你们正在努力,并会在一个月后回来查看。
- 我们几天才检索一次。您需要给微软发电子邮件,让这个机器人离线,并在防火墙中阻止它的 IP。
还请在 #boost-help 中与 slack 上的 boost 团队交谈,以便他们知道发生了什么,并可以改进代码,使 boost 更能抵御这些攻击。
愉快的周末!
Thank you for telling me the solution. In the next round, you can see the improvement of the retrieval success rate. Can you support me, have a good day.
checker:manualTrigger
✔️ Storage provider distribution looks healthy.
✔️ Data replication looks healthy.
✔️ No CID sharing has been observed.
[^1]: To manually trigger this report, add a comment with text checker:manualTrigger
[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger
[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...
Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.
Your Datacap Allocation Request has been approved by the Notary
bafy2bzaceaw65n26j5evzegsivoklwlt5s6krhcoveithtq2qgtpohicm7gxy
Address
f1y5mkyvzsfxsapuecbbs4hrrmio2te6ajdqpgedq
Datacap Allocated
1.00PiB
Signer Address
f1c5non5yf35avgcpsqvxu4yj54yyvxorwyjochqq
Id
596c23c0-4e24-4ab5-ac79-70ebb6639af3
You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceaw65n26j5evzegsivoklwlt5s6krhcoveithtq2qgtpohicm7gxy
Comments from a DC seller, we are inclined to ignore.
f02049625
f1y5mkyvzsfxsapuecbbs4hrrmio2te6ajdqpgedq
2PiB
ae0bed38-71c6-4815-86de-fc155e492911
f02049625
f1y5mkyvzsfxsapuecbbs4hrrmio2te6ajdqpgedq
400% weekly > 2PiB, requesting 2PiB
2PiB
9.313225746154786e+36YiB
9.313225746154786e+36YiB
Number of deals | Number of storage providers | Previous DC Allocated | Top provider | Remaining DC |
---|---|---|---|---|
48767 | 17 | 1PiB | 15.16 | 227.46TiB |
checker:manualTrigger
✔️ Storage provider distribution looks healthy.
✔️ Data replication looks healthy.
✔️ No CID sharing has been observed.
[^1]: To manually trigger this report, add a comment with text checker:manualTrigger
[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger
[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...
Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.
The dispute has been closed during T&T zoom call. Report also LGTM.
Your Datacap Allocation Request has been proposed by the Notary
bafy2bzacedh4dcsr7b7vu5tgrvornuwedneimmyek7qp4wcj6b5imphxiwsm6
Address
f1y5mkyvzsfxsapuecbbs4hrrmio2te6ajdqpgedq
Datacap Allocated
2.00PiB
Signer Address
f1yjhnsoga2ccnepb7t3p3ov5fzom3syhsuinxexa
Id
ae0bed38-71c6-4815-86de-fc155e492911
You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedh4dcsr7b7vu5tgrvornuwedneimmyek7qp4wcj6b5imphxiwsm6
Report looks good to me. Will support this round.
Your Datacap Allocation Request has been approved by the Notary
bafy2bzacealtavwtl64x6ems4fjtotxxc5qasb3f6w7jzfhu5m3iz462wctmq
Address
f1y5mkyvzsfxsapuecbbs4hrrmio2te6ajdqpgedq
Datacap Allocated
2.00PiB
Signer Address
f1xzff5xup63o5sygr2swp4zvcajg54lotliimdty
Id
ae0bed38-71c6-4815-86de-fc155e492911
You can check the status of the message here: https://filfox.info/en/message/bafy2bzacealtavwtl64x6ems4fjtotxxc5qasb3f6w7jzfhu5m3iz462wctmq
checker:manualTrigger
⚠️ 1 storage providers sealed too much duplicate data - f02217602: 23.18%
✔️ Data replication looks healthy.
✔️ No CID sharing has been observed.
[^1]: To manually trigger this report, add a comment with text checker:manualTrigger
[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger
[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...
Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.
⚠️ 1 storage providers sealed too much duplicate data - f02217602: 45.31%
✔️ Data replication looks healthy.
✔️ No CID sharing has been observed.
[^1]: To manually trigger this report, add a comment with text checker:manualTrigger
[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger
[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...
Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.
f02049625
f1y5mkyvzsfxsapuecbbs4hrrmio2te6ajdqpgedq
2PiB
f0b0971a-2b87-466e-ac36-0c1644cc568e
Data Owner Name
National Radio Astronomy Observatory
What is your role related to the dataset
Data Preparer
Data Owner Country/Region
United States
Data Owner Industry
Life Science / Healthcare
Website
https://data.nrao.edu/portal/#/
Social Media
Total amount of DataCap being requested
27PiB
Expected size of single dataset (one copy)
1P
Number of replicas to store
10
Weekly allocation of DataCap requested
1PiB
On-chain address for first allocation
f1y5mkyvzsfxsapuecbbs4hrrmio2te6ajdqpgedq
Data Type of Application
Public, Open Dataset (Research/Non-Profit)
Custom multisig
Identifier
No response
Share a brief history of your project and organization
Is this project associated with other projects/ecosystem stakeholders?
No
If answered yes, what are the other projects/ecosystem stakeholders
No response
Describe the data being stored onto Filecoin
Where was the data currently stored in this dataset sourced from
AWS Cloud
If you answered "Other" in the previous question, enter the details here
No response
How do you plan to prepare the dataset
IPFS, lotus, singularity
If you answered "other/custom tool" in the previous question, enter the details here
No response
Please share a sample of the data
Confirm that this is a public dataset that can be retrieved by anyone on the Network
If you chose not to confirm, what was the reason
No response
What is the expected retrieval frequency for this data
Yearly
For how long do you plan to keep this dataset stored on Filecoin
1.5 to 2 years
In which geographies do you plan on making storage deals
Greater China, Asia other than Greater China, Africa, North America, Europe
How will you be distributing your data to storage providers
Cloud storage (i.e. S3), HTTP or FTP server, IPFS, Shipping hard drives, Lotus built-in data transfer
How do you plan to choose storage providers
Slack, Filmine, Big Data Exchange
If you answered "Others" in the previous question, what is the tool or platform you plan to use
No response
If you already have a list of storage providers to work with, fill out their names and provider IDs below
How do you plan to make deals to your storage providers
Boost client, Lotus client, Droplet client, Singularity
If you answered "Others/custom tool" in the previous question, enter the details here
No response
Can you confirm that you will follow the Fil+ guideline
Yes