search

filecoin-project / notary-governance

114 stars 58 forks source link

Abusive Behavior by Notaries in the Filecoin+ Program: Urgent Action Required #882

Closed herrehesse closed 3 months ago

herrehesse commented 1 year ago

Issue Description

It has come to our attention that several notaries participating in the Filecoin+ program have been engaging in abusive behavior. These notaries have been signing applications that do not adhere to the program's rules and guidelines, such as failing to enable the retrieval of stored files. Shockingly, none of the listed notaries below have demonstrated due diligence by thoroughly checking the files or asking appropriate questions to the applicants.

Based on the available evidence, it appears that these notaries are colluding with clients and miners to simply gain datacap for growth, at the expense of program integrity. This behaviour is unacceptable and demands immediate intervention.

Proposed Solution(s)

We hereby request the prompt removal of the following notaries from the program within 14 days, unless they can adequately explain themselves and provide a compelling justification for their actions. During this period, it is essential that they publicly explain their abusive conduct as notaries, clarify why they neglected their responsibilities, and disclose how they mistakenly allocated multiple PiBs of datacap to clients who exploited the system. Furthermore, we strongly recommend putting them on hold immediately to prevent them from signing any additional datacap requests.

Notaries in question:

The aforementioned notaries' actions have undermined the trust and fairness of the Filecoin+ program. It is crucial to restore confidence by addressing these issues and taking appropriate action against those responsible.

Continued Investigation(s)

Furthermore, we strongly encourage all members of the community to conduct a comprehensive investigation into the applications, clients, and notary behaviors over the past few months. We firmly believe that the instances of abusive behavior we have uncovered are merely the beginning, and there may be more concerning incidents yet to be discovered. The following are the initial applications that have exhibited detected abusive behavior:

Transparency, accountability, and adherence to established guidelines are vital to the success of the Filecoin+ program. We appreciate the community's cooperation in promptly rectifying these concerns and ensuring a more equitable and secure environment for all participants.

Carohere commented 1 year ago

These notaries have been signing applications that do not adhere to the program's rules and guidelines, such as failing to enable the retrieval of stored files.

Retrieval is up to service providers. I don't see why notaries should be blamed for this.

herrehesse commented 1 year ago

The responsibility of retrieval falls upon the storage providers, while the duty of conducting thorough due diligence before signing any application lies with the individual notaries. It is an integral part of their professional responsibilities. Failing to perform this essential task may give the impression that these notaries are engaging in collusion with clients and miners solely for the purpose of acquiring datacap, disregarding the integrity of the program.

Official guidelines below:

Notaries are selected to serve as fiduciaries for the Filecoin Network and are responsible for allocating DataCap to clients with valuable storage use cases. The base responsibilities of notaries include:

Carohere commented 1 year ago

Disagree. You are not considering non-technical notaries. My due diligence is based on data size, distribution plan, back and forth communication with the client, etc. to determine the validity of the client & application. Let's say the client says the application will support retrieval but later turns out it doesn't - this is not something that could be confirmed at the time I provided my signature. Plus retrievability of open data is still under discussion. I don't think sp abuse should be attributed to notaries.

Btw for any concerns you may have, it's best to raise them via the dispute tracker. The link to notion is posted in T&T WG so most conversations are going on Slack. People would have a hard time following up on GitHub.

herrehesse commented 1 year ago

In my sincere opinion, the responsibilities of a notary encompass technical aspects. I strongly believe that notaries should possess the ability to retrieve files in order to effectively fulfill their professional duties. Also, I have identified several issues with your current operational approach:

Added them to the dispute tracker and love to chat about it during the T&T call.

raghavrmadya commented 1 year ago

Acknowledging the points made here. Requesting all notaries mentioned to respond to the claims above. Failure to respond may lead to the recommendation of removal from T&T WG

kernelogic commented 1 year ago

As a non E-FIL+ LDNs, I believe data needs to be retrievable. Also more importantly, data should not be artificially enlarged to fill the sectors (i.e. share files between cars, enlarge a low res 15 sec video to 2GB, enlarge a low res picture to 700MB, etc).

cryptowhizzard commented 1 year ago

Disagree. You are not considering non-technical notaries. My due diligence is based on data size, distribution plan, back and forth communication with the client, etc. to determine the validity of the client & application. Let's say the client says the application will support retrieval but later turns out it doesn't - this is not something that could be confirmed at the time I provided my signature. Plus retrievability of open data is still under discussion. I don't think sp abuse should be attributed to notaries.

Btw for any concerns you may have, it's best to raise them via the dispute tracker. The link to notion is posted in T&T WG so most conversations are going on Slack. People would have a hard time following up on GitHub.

As stated yesterday in the T&T call my most frustrating issue is that when I show that a dataset is not retrievable in contrast to their stated promise other notary’s still sign off future tranches. If we go down this path then this whole FIL+ thing is a charade and we can just go ahead and do no due diligence at all and sign everything.

If a notary is not equipped to test retrieval i don’t see a problem with that as long as he/she/them aligns with the notary who is. It is a matter of respect.

Secondly i have asked clients to provide me with car files they are storing where retrieval is or was not possible as second solution to build trust. This file + CID (that matches the file) enables me to verify if the file is authentic as well enables me to unpack and verify what is in it. Failure to do so by the client only means one thing and that’s they are hiding they are not storing what they say they are. Still other notary’s go ahead and sign these.

cryptowhizzard commented 1 year ago

Disagree. You are not considering non-technical notaries. My due diligence is based on data size, distribution plan, back and forth communication with the client, etc. to determine the validity of the client & application. Let's say the client says the application will support retrieval but later turns out it doesn't - this is not something that could be confirmed at the time I provided my signature. Plus retrievability of open data is still under discussion. I don't think sp abuse should be attributed to notaries.

Btw for any concerns you may have, it's best to raise them via the dispute tracker. The link to notion is posted in T&T WG so most conversations are going on Slack. People would have a hard time following up on GitHub.

I forgot:

retrievability is not under discussion. It is a clear program rule to have the data readily retrievable. Without retrieval there is no way to check if a client is really storing the data he/she/them says is.

TakiChain commented 1 year ago

Thanks for the reminder RG. We have responded directly in applications. Please see https://github.com/filecoin-project/filecoin-plus-large-datasets/issues/1938#issuecomment-1539625434

AthSmith commented 1 year ago

Hi @raghavrmadya The reason we are reluctant to respond to such post soon is that we don't really accept this challenge way they are using, it tends to create intense confrontation. All of us have too much work to do.

At this stage, We think it is significant to maintain the notary's decision-making independence, which will help the filecoin network to be more inclusive.

We communicate carefully with each applicant and also refer to the CID reports. These applicants are honest and positive, and the reports are healthy, so we are willing to support them. As for the inability of some SP nodes to be retrieved online, I would like to trust that it will be fundamentally changed with the opening of the official incentive network.

Of course, if the community feels the need to include the scores of retrieval in the CID report, I also think that is a good idea. It would be easier for us to make decisions in the future.

BobbyChoii commented 1 year ago

The retrievability of the LDN application data and how to perform the retrieval are not explained during the notary's onboarding phase. Imho unsuccessful miner retrieval should not be interpreted as a notary abusing the system. I have included details of my due diligence on the applications mentioned above in Dispute Disclosure.xls. Any feedback is welcome.

Casey-PG commented 1 year ago

Thank you for pointing out the unretrievable data problem @herrehesse. As we shared before, as a media company we do our best to judge from the description of the application, the information shared by the client combined with the Filplus large dataset project concept. But we have to admit the fact that our technical field has certain limitations. In order to reduce misunderstandings and increase the level of consensus, we are working on the technical retrieval section to enlarge our understanding. However, at the moment the feedback can not be the same from different technical bases.

We have requested the governance group or the foundation to start data retrieval education courses for the notary public. We also noticed the discussion about the retrievability of Fil+ open data. We strongly believe tooling from the PL side will be a great help to erase internal friction among notaries and make the value of real datasets more easily recognized.

Best regards, PangodGroup Team

Bennyyangpu commented 1 year ago

These datasets have visible proof of the data size and are useful for the network. These clients justified their reasoning for the amount they are applying. I find it difficult to respond to accusations against us when the available evidence is subjective... Is there somewhere I could find statistics on current retrievable LDN requests? @raghavrmadya as the lead of the T&T WG I would like to learn your take on this.

cryptowhizzard commented 1 year ago

Let me give you an example and summary of what you and @Casey-PG and @BobbyChoii and @AthSmith are saying and some facts.

The fact is that there are rules in the FIL+ program. The data must be open and readily retrievable. This has been since day one. One of the obvious reasons for this you can read down below.

The second fact is that despite notary's advising not to sign, there are still other notary's disrespectfully overriding others who are investing time by checking retrievals + the data.

Now for the interesting part:

Suppose Applicant A submits a LDN request and say that they will store the DANDI dataset. In the LDN they state that they will follow the rules and make the dataset public and retrievable. Now, given the responses above, if the checkbot looks alright, notary's sign. No KYC, no further questions and no retrieval and checking of the data. We all wash our hands in innocence.......

Let me ask you, what will happen if this applicant stores Childporn and we don't care to check? Without proper retrieval we can't check the data? What will this mean for the reputation of Filecoin when this get's out in the open??

What will happen if we start storing sensitive information like human rights violations against the Uyghurs or warcrimes in Ukraine on Chinese SP's? Do you guys still think this is funny by then and will you accept an excuse like 'Ohhhh sorry, we have a technical difficulty with one of our SP's, next time will be better' ??????

I really don't think so and ask you to reconsider your answers.

Again -> Just check the data that is stored. You can do it with retrieval and look in the files you retrieve. If this is impossible then make an exception and ask a random .car file from the data preparer and check what is in it + verify commP. If you don't know how to do it ask someone else to do it for you and let them explain how to do it while you hold of from signing.

Because, deliberately failing to do our job makes us look bad in the long run and ruin all our reputation. I don't want to be a part of that and i am sorry if you guys find this offending.

BobbyChoii commented 1 year ago

Just want to be clear that I agree 100% with the need for retrieval. In the long run, this must be an important feature to be implemented in Filecoin. But the current retrieval part has two obstacles 1) the actual amount of data retrieved, success rate and frequency is unknown 2) an effective method of retrieval at scale. Individual behavior is not sufficient to verify the volume of Filplus data, simple retrieval tools are a must.

herrehesse commented 1 year ago

In light of these persistent signing issues, it is crucial that appropriate actions be taken. We propose implementing measures such as removal, indefinite pausing, or temporary bans on notaries who repeatedly engage in such practices. It is imperative to explicitly state that notaries must possess the capability to retrieve and test files effectively, as this is essential for them to fulfil their responsibilities adequately.

It is our firm belief that those who cannot meet these standards should not pursue the role of a notary.

Additionally, we would like to emphasise the significance of quality notaries in the Filecoin+ program. If we aim to enter a phase focused on enhancing quality, it is paramount to start with the foundation of capable notaries. To this end, we propose that Notary Application Round 5 becomes a pivotal moment to admit only those notaries who demonstrate the necessary skills and commitment. Furthermore, we recommend limiting the number of notaries to ensure a more manageable and efficient system. Additionally, we should promptly identify and remove idle notaries (2-3 weeks) to maintain an active and engaged cohort.

By implementing these actions, we can reinforce the importance of accountability, reliability, and professionalism within the Filecoin+ program. Our collective efforts will ensure a more robust and trustworthy ecosystem that fosters collaboration and success.

Let us move forward together in a professional and sophisticated manner, focusing on the enhancement of quality in the Filecoin+ program.

caoyoungyoung commented 1 year ago

When the retrieval bot will be available for the public? Maybe the bot will solve the dilema. Cid report will help notaries a lot and add transparency.

Casey-PG commented 1 year ago

The universalization of retrieval needs to be achieved through official tool. Once the retrieval bot is live, notaries can use the data it provides as a reference together with CID reports. We believe this will be a new milestone on Filecoin.

We understand and appreciate that retrieval is an important component of the network. However, there is no mandatory requirement for retrieval capability in the R4 notary election. @Kevin-FF-USA @raghavrmadya We strongly urge that before the next round of notary applications begins, an official retrieval course be formed for interested candidates to learn. We believe this would be useful to help improve the quality phase and reduce community friction.

Best regards, PangodGroup Team

raghavrmadya commented 1 year ago

Been following this discussion closely and taking time took time to form an unbiased opinion. Retrieval bot is live. The discussion, for some reason, has become highly indexed on retrievals but there are other issues that @herrehesse has pointed out that have yet to be addressed.

Taking the suggestion of the retrieval course and working on it.

We will discuss this issue as well as #892 in the T&T WG call. Kindly engage there

BobbyChoii commented 1 year ago

The discussion, for some reason, has become highly indexed on retrievals but there are other issues that @herrehesse has pointed out that have yet to be addressed.

Whether or not the notary is capable of conducting retrieval is the main reason this dispute was brought up. I have previously explained my signature https://github.com/filecoin-project/notary-governance/issues/882#issuecomment-1552707599. If there is any confusion or any detail needed please tag me

herrehesse commented 1 year ago

Regardless of the opinions or beliefs of notaries, it is essential for Filecoin Plus data to be easily retrievable, as it is one of the fundamental requirements. If this requirement is not met but notaries still sign off on the data, they will face disputes and be asked to provide an explanation.

Responding with statements such as "There is no mandatory requirement for retrieval capability in the R4 notary election" is meaningless and holds no value because the rules clearly state that files must be readily retrievable. This implies that part of the notaries' due diligence is to verify this aspect.

If notaries fail to perform this duty, they are logically at risk of losing their position.

Casey-PG commented 1 year ago

Responding with statements such as "There is no mandatory requirement for retrieval capability in the R4 notary election" is meaningless and holds no value because the rules clearly state that files must be readily retrievable. This implies that part of the notaries' due diligence is to verify this aspect.

If notaries fail to perform this duty, they are logically at risk of losing their position.

I'm very offended by your words, what do you mean by "meaningless"? I'm simply laying out the fact that there is no requirement for retrieval ability in R4 notary election rules. If you think the election requirements are inappropriate, please propose improvements and don't use your personal logic to damage our reputation at will.

I would also like to remind you that retrieval is still not at consensus. https://filecoinproject.slack.com/archives/C01DLAPKDGX/p1688152842582719?thread_ts=1688029823.430619&cid=C01DLAPKDGX We are closely following the discussion on retrieval and notaries have no control over sp retrieval. This is an issue between the client and the sp. We are not the same as Dcent, we are not sp, we have no control over the storage.

We have responded many times in the past two months. It's just too bizarre for us to keep self-proving without substantial evidence or written rules. This ineffective communication left us weary. We sincerely expect the retrieval rules to be placed soon to reduce the cost of communication.

Best, PangodGroup Team