Proposal: Qualifiers for immediate notary removal

[raghavrmadya]

Issue

Over the past few months, it has become clear that a prolonged dispute-based approach i.e. sharing evidence of potential abuse of DC back and forth has NOT been proven to be the best approach to catch abusive notaries. While there have been several attempts to address potential notary abuse, eg #934 #901 #902 #907, the inherent room for subjectivity given to notaries as outlined in FIP-003 has made the notary group too powerful

As outlined in FIP-003, "Notaries are selected to act as fiduciaries for the Filecoin network. Notaries are entrusted with DataCap to be spent in a manner that aligns with the Principles.

The responsibilities of the Notaries are as follows:

1. Allocate DataCap to clients in order to subsidize reliable and useful storage on the network.

2. Verify that clients receive a DataCap commensurate with the level of trust that is warranted based on the information provided.

3. Ensure that in the allocation of the DataCap no party is given excessive trust in any form that might jeopardize the network.

4. Follow operational guidelines, keep a record of decision flow, and respond to any requests for audits of their allocation decisions.

While we have invested and made significant improvements in 1. and 2. over the last year which includes improvements to the notary front-end, KYC experiments and launching DD tools such as the starboard dashboard, we have seen many instances where 3. and 4. have been points of contention and have not been properly monitored.

Attempts have been made by the governance team to keep notaries accountable for responsibilities 1. and 2. , for example, #948 and #927, any attempts to address potential violations of responsibilities 3. and 4. have been met with a blame game.

Proposed solution

As such, this proposal proposes to establish some objective instances where a notary shall be removed if he/she is unable to provide counter-evidence within 1 week of a potential violation.

1. If an active V4 notary signs an application currently under diligence by another active notary where the latter has explicitly informed the community of ongoing diligence, the signing notary must be removed.

2. If a notary fails to provide a record of decision flow, and response to any requests for audits of their allocation decisions within 1 week, the said notary must be removed.

3. If a direct relationship exists b/w a client and a notary, as outlined in #825, the active notary must be removed from the multi-sig.

Thoughts and suggestions on the aforementioned and additional instances that must qualify for immediate removal of notaries to address power concentration are welcome.

Proposed timeline Discussion - Aug 9th - Aug 15th 2023 Decision on proposal - August 16th 2023

[Filplus-Watchdog]

https://github.com/filecoin-project/filecoin-plus-large-datasets/issues/2084 @kernelogic @woshidama323 @NDLABS-Leo

[kernelogic]

It looks like @Filplus-Watchdog submitted a dispute in #2084 and I missed it.

I apologize for failing to recognize a dispute but judging from the answer from client that they wouldn't work with said SP. I am not sure why you still opened a dispute. Did you not see the response or was it not satisfactory to you?

[Filplus-Watchdog]

Failure to see is itself a dereliction of duty. If all rules could be easily interpreted or excused to be broken at will, it would be better to not have them at all.

[kernelogic]

@Filplus-Watchdog Are you an active notary? If not, your dispute doesn't really fall within this scope.

If an active V4 notary signs an application currently under diligence by another active notary where the latter has explicitly informed the community of ongoing diligence, the signing notary must be removed.

[kernelogic]

Also, if this going to get implemented, I have the following suggestions:

1. It should be effective after the date this issue is passed. No clawback.
2. We need some kind of time SLA on the dispute resolution - they can't be sitting in the tracker for weeks without resolution. For example have a bot to auto close as dismissed if no resolution is made within a certain timeframe, equivalent to someone going to court to fight a traffic ticket but the police doesn't show up.
3. Dispute on the same LDN should not repeat within a certain timeframe. i.e. after a dispute on #1234 is dismissed, the same person cannot dispute it again for a certain timeframe (so that it will not be put back on the tracker again immediately blocked for progressing).

[woshidama323]

I have reviewed all the info from the comments when signing and it seems that the owner of LDN has given his plan. I haven't seen any further disputes. I don't understand why I can't sign it

Totally agree with @kernelogic 's suggestions, Just like I mentioned in discussion 933

1. How do we define disputes?
2. Any rules about disputes?
3. Who can judge if the dispute is reasonable or not?
4. If the dispute is reasonable then LDN would be removed But what if a dispute is unreasonable, Is any solutions for compensating SP or LDN owner? I don't think this is fair for the whole community

@raghavrmadya. Wish you could help to consider that unreasonable disputes will cause loss to SP or LDN owner as this would break their plan and reputations

BTW, I am not sure who is the owner of Filplus-Watchdog, Just want to confirm if this from the FILPlus gov team ?

[NDLABS-Leo]

filecoin-项目/filecoin-plus-large-datasets#2084 @kernelogic @woshidama323 @NDLABS-Leo

@Filplus-Watchdog Please clearly state your demands, provide a link, and then mention us. Additionally, who are you? Representing which organization?

[Janicelu1]

Hi RG, any subjective determination will lead to all kinds of friction, and I don't think your proposal will help solve the problem, as they still contain a great deal of subjective intent.

As a practical matter, dispute resolution requires voting tools, not relying on endless debate and evidence provision. Most of the controversies are at this stage now: any person has the right to submit a dispute freely, but no one is authorized to rule on the validity of evidence. So it's a mess.

Regarding the rules for voting, we should discuss them.

[hcgun]

filecoin-project/filecoin-plus-large-datasets#2084 @kernelogic @woshidama323 @NDLABS-Leo

@Filplus-Watchdog I don't think just because you silently submit a dispute, people can't sign it.Anyone can submit a dispute silently.

[Dave-lgtm]

Also curious to know if everyone can submit a dispute , and any notaries aren't allowed to sign until it's resolved?

PS: We're finding too many applications stalled in the dispute queue.

[1ane-1]

Proposal：We should set up a reasonable scope of disputes to avoid too many unproof disputes, leading to waste of everyone's time to handle disputes.

[joshua-ne]

Proposal：We should set up a reasonable scope of disputes to avoid too many unproof disputes, leading to waste of everyone's time to handle disputes.

I agree with this

[joshua-ne]

Meanwhile, judging if a notary is misbehaving is not as easy as judging if an LDN is behaving well or badly. So before taking action on notaries, I think it is easier to set quantified standards to judge if an LDN is abusive. With that in hand, it would be much easier to quantify notaries' behavior (i.e. to see how many, or what's the percentage of the 'bad' LDNs the notary has signed)

[joshua-ne]

The quantification of an LDN can be done by bots and if an LDN is marked as bad or abusive, the bots can automatically put a tag on that LDN so no notary can sign it anymore before the tag is removed by a proper explanation from the client (this process may need some judgment from the notary community, in the form of voting or some other ways; and it also depends on the reason why it is tagged)

[spaceT9]

The abusive behavior should be defined more clearly and accurately

[Chris00618]

The best way to resolve disputes is to vote, and I don't think anyone wants to be internally consumed in endless disputes. Even if the results of the vote don't agree with what individuals think, that's the consensus and you need to abide by it.

[raghavrmadya]

A vote does not help us build in the direction of automated notaries/smart contract-enabled notaries. I personally have learned this through past attempts to resolve disputes using a voting mechanism.

I'm going to side with the comments above that are calling for a "reasonable scope of disputes".

That is EXACTLY what this proposal is attempting to do. All below are reasonable scope of disputes that do not require a vote. If it is not clear by now, voting is just another form of delaying consensus and we need to start to define consequences for notary actions that are not disputable.

If an active V4 notary signs an application currently under diligence by another active notary where the latter has explicitly informed the community of ongoing diligence, the signing notary must be removed.

On what basis will you dispute this and on what basis will you vote on this?

If a notary fails to provide a record of decision flow, and response to any requests for audits of their allocation decisions within 1 week, the said notary must be removed.

This is clearly a violation of FIP-003 as well as the due diligence plan self outlined by notaries in their application. Again, opening the scope for voting and dispute on this one leads us nowhere because it will be a never-ending game of bickering.

If a direct relationship exists b/w a client and a notary, as outlined in Addressing collusion b/w client and notary #825, the active notary must be removed from the multi-sig.

I agree that this point is more subjective as it can take time to prove such a relationship. But once proven, what is the reasoning for having a vote?

[raghavrmadya]

I have reviewed all the info from the comments when signing and it seems that the owner of LDN has given his plan. I haven't seen any further disputes. I don't understand why I can't sign it

Totally agree with @kernelogic 's suggestions, Just like I mentioned in discussion 933

1. How do we define disputes?
2. Any rules about disputes?
3. Who can judge if the dispute is reasonable or not?
4. If the dispute is reasonable then LDN would be removed But what if a dispute is unreasonable, Is any solutions for compensating SP or LDN owner? I don't think this is fair for the whole community

@raghavrmadya. Wish you could help to consider that unreasonable disputes will cause loss to SP or LDN owner as this would break their plan and reputations

BTW, I am not sure who is the owner of Filplus-Watchdog, Just want to confirm if this from the FILPlus gov team ?

Good question. Disputes are defined ACTIVELY as the community engages in deal-making based on the principles defined in FIP-003. Unfortunately, no one had a magical ball when they wrote the FIP and as this ecosystem and program is a living organism, something that wasn't a qualifier for dispute 6 months ago, in the absence of a CID checker bot for example, can qualify for a dispute today.

[raghavrmadya]

Also, if this going to get implemented, I have the following suggestions:

1. It should be effective after the date this issue is passed. No clawback.
2. We need some kind of time SLA on the dispute resolution - they can't be sitting in the tracker for weeks without resolution. For example have a bot to auto close as dismissed if no resolution is made within a certain timeframe, equivalent to someone going to court to fight a traffic ticket but the police doesn't show up.
3. Dispute on the same LDN should not repeat within a certain timeframe. i.e. after a dispute on #1234 is dismissed, the same person cannot dispute it again for a certain timeframe (so that it will not be put back on the tracker again and immediately blocked for progressing).

Agreed on 1.

Agreed. Effective SLA is 14 days. Appreciate you holding me and gov team accountable. Perhaphs it is reasonable to propose that if a dispute isn't resolved in 14 days, issues like #895 are void?

Agreed as well. Happy to support a proposal if you can propose one

[raghavrmadya]

Also curious to know if everyone can submit a dispute , and any notaries aren't allowed to sign until it's resolved?

PS: We're finding too many applications stalled in the dispute queue.

Yes. #895

[raghavrmadya]

Just to clarify, a lot of comments here are broad strokes and do not address the three qualifiers mentioned in the proposal. If you disagree, kindly provide input on WHY you think a specific qualifier is unreasonable. Jut saying "no" or making philosophical comments of what you think will work or what is merely your opinion you are welcome to make a proposal based on it.

It is only reasonable that comments made directly the listed qualifiers, either in support or disagreement (both with reasoning), be taken into consideration when assess if the proposal is passed.

[kernelogic]

Agreed. Effective SLA is 14 days. Appreciate you holding me and gov team accountable. Perhaps it is reasonable to propose that if a dispute isn't resolved in 14 days, issues like #895 are void?

Correct, after 14 days no resolution = dismissal, #895 condition is void, notary can resume signing.

Agreed as well. Happy to support a proposal if you can propose one

After a dispute is dismissed. The same person cannot dispute on the same LDN before the next tranche is approved.

Other DD ideas see my v5 selection recommendation.

[Chris00618]

That is EXACTLY what this proposal is attempting to do. All below are reasonable scope of disputes that do not require a vote. If it is not clear by now, voting is just another form of delaying consensus and we need to start to define consequences for notary actions that are not disputable.

Okay, if you still feel that voting only reduces efficiency, then we can continue the discussion:

If an active V4 notary signs an application currently under diligence by another active notary where the latter has explicitly informed the community of ongoing diligence, the signing notary must be removed.

A、how to define "active"? how about inactive? then what? B、what's the SOP if a V4 notary signs an application currently under gaslighting by another notary or some active/inactive members using their personal rules? C、how to distinguish the difference between a gaslighting nuisance or a normal DD? D、how to distinguish the difference between the personal rules and the consensus rules?( for example: the SPs you work with have some bad record before, you have some bad application record before, part of your SPs are using VPN, your decentralized nodes does not span continents, the bandwidth of your nodes is too slow, other people can download successfully but I can't...etc)

If a notary fails to provide a record of decision flow, and response to any requests for audits of their allocation decisions within 1 week, the said notary must be removed.

if the notary gave the response within one week , no voting? are you gonna be a judge? @raghavrmadya. If in the future you're no longer impartial because of the interests involved, what should we do? if the notary didn't response, but the applicant or other active members did that, then what?

Of course, the 14-day expiration date reduces these negative effects due to rough governance. But we must not minimize their destructive intent.

[NDLABS-Leo]

Also curious to know if everyone can submit a dispute , and any notaries aren't allowed to sign until it's resolved? PS: We're finding too many applications stalled in the dispute queue.

Yes. #895

@raghavrmadya If this is the case, it means that rkh's existence is merely nominal. Because as long as someone is willing, they can create numerous alternate accounts and extend the 14-day deadline indefinitely. In that scenario, the LDNs would lose their purpose of being signed.

[raghavrmadya]

That is EXACTLY what this proposal is attempting to do. All below are reasonable scope of disputes that do not require a vote. If it is not clear by now, voting is just another form of delaying consensus and we need to start to define consequences for notary actions that are not disputable.

Okay, if you still feel that voting only reduces efficiency, then we can continue the discussion:

If an active V4 notary signs an application currently under diligence by another active notary where the latter has explicitly informed the community of ongoing diligence, the signing notary must be removed.

A、how to define "active"? how about inactive? then what? B、what's the SOP if a V4 notary signs an application currently under gaslighting by another notary or some active/inactive members using their personal rules? C、how to distinguish the difference between a gaslighting nuisance or a normal DD? D、how to distinguish the difference between the personal rules and the consensus rules?( for example: the SPs you work with have some bad record before, you have some bad application record before, part of your SPs are using VPN, your decentralized nodes does not span continents, the bandwidth of your nodes is too slow, other people can download successfully but I can't...etc)

If a notary fails to provide a record of decision flow, and response to any requests for audits of their allocation decisions within 1 week, the said notary must be removed.

if the notary gave the response within one week , no voting? are you gonna be a judge? @raghavrmadya. If in the future you're no longer impartial because of the interests involved, what should we do? if the notary didn't response, but the applicant or other active members did that, then what?

Of course, the 14-day expiration date reduces these negative effects due to rough governance. But we must not minimize their destructive intent.

Active is defined here - https://notaries.datacapstats.io/

[raghavrmadya]

That is EXACTLY what this proposal is attempting to do. All below are reasonable scope of disputes that do not require a vote. If it is not clear by now, voting is just another form of delaying consensus and we need to start to define consequences for notary actions that are not disputable.

Okay, if you still feel that voting only reduces efficiency, then we can continue the discussion:

If an active V4 notary signs an application currently under diligence by another active notary where the latter has explicitly informed the community of ongoing diligence, the signing notary must be removed.

A、how to define "active"? how about inactive? then what? B、what's the SOP if a V4 notary signs an application currently under gaslighting by another notary or some active/inactive members using their personal rules? C、how to distinguish the difference between a gaslighting nuisance or a normal DD? D、how to distinguish the difference between the personal rules and the consensus rules?( for example: the SPs you work with have some bad record before, you have some bad application record before, part of your SPs are using VPN, your decentralized nodes does not span continents, the bandwidth of your nodes is too slow, other people can download successfully but I can't...etc)

If a notary fails to provide a record of decision flow, and response to any requests for audits of their allocation decisions within 1 week, the said notary must be removed.

if the notary gave the response within one week , no voting? are you gonna be a judge? @raghavrmadya. If in the future you're no longer impartial because of the interests involved, what should we do? if the notary didn't response, but the applicant or other active members did that, then what?

Of course, the 14-day expiration date reduces these negative effects due to rough governance. But we must not minimize their destructive intent.

B, C, D are not in this scope IMO. One can easily follow notary DD in an LDN. It's not my job to distinguish between a gaslighting nuisance or a normal DD

[raghavrmadya]

if the notary gave the response in a week and the other notary does not have follow-ups, that notary decision-flow is valid.

if the notary didn't respond, but the applicant or other active members did that, then what? - How can an applicant or community member respond to notary decision on singing?

cc @Chris00618

[NDLABS-Leo]

https://github.com/filecoin-project/filecoin-plus-large-datasets/issues/2055 https://github.com/filecoin-project/filecoin-plus-large-datasets/issues/2084

@raghavrmadya An unknown GitHub account submitted an irrational Dispute Tracker entry for ND's application and then disappeared. In this situation, will we be unable to proceed with our project? Do we have to wait for the full 14 days?

[NDLABS-Leo]

@kernelogic @raghavrmadya @Chris00618 @1ane-1 Additionally, I believe the Dispute Tracker is not meant to be a tool used maliciously to obstruct others' storage projects. The Dispute Tracker is designed to address situations where there is a dispute that requires the involvement of rg for resolution. However, it has now turned into a scenario where there are no discussion records in LDN, yet an unknown GitHub account casually submits a Dispute Tracker entry. This clearly isn't reasonable. This not only causes significant time wastage for the applicants but also reduces the efficiency of rg's operations as there's a need to clean up numerous irrelevant entries.

Therefore, I propose: the submission of a Dispute Tracker should require the precondition that there is a dispute present in LDN, with at least a discussion history between the questioner and the one being questioned, to be considered a valid Dispute Tracker entry.

[Chris00618]

That is EXACTLY what this proposal is attempting to do. All below are reasonable scope of disputes that do not require a vote. If it is not clear by now, voting is just another form of delaying consensus and we need to start to define consequences for notary actions that are not disputable.

Okay, if you still feel that voting only reduces efficiency, then we can continue the discussion:

If an active V4 notary signs an application currently under diligence by another active notary where the latter has explicitly informed the community of ongoing diligence, the signing notary must be removed.

A、how to define "active"? how about inactive? then what? B、what's the SOP if a V4 notary signs an application currently under gaslighting by another notary or some active/inactive members using their personal rules? C、how to distinguish the difference between a gaslighting nuisance or a normal DD? D、how to distinguish the difference between the personal rules and the consensus rules?( for example: the SPs you work with have some bad record before, you have some bad application record before, part of your SPs are using VPN, your decentralized nodes does not span continents, the bandwidth of your nodes is too slow, other people can download successfully but I can't...etc)

If a notary fails to provide a record of decision flow, and response to any requests for audits of their allocation decisions within 1 week, the said notary must be removed.

if the notary gave the response within one week , no voting? are you gonna be a judge? @raghavrmadya. If in the future you're no longer impartial because of the interests involved, what should we do? if the notary didn't response, but the applicant or other active members did that, then what? Of course, the 14-day expiration date reduces these negative effects due to rough governance. But we must not minimize their destructive intent.

B, C, D are not in this scope IMO. One can easily follow notary DD in an LDN. It's not my job to distinguish between a gaslighting nuisance or a normal DD

How can you expect notaries to carry out your proposal accurately when your own answer is so equivocate?

[Dave-lgtm]

All suspicious behaviors claimed by some people should not be defined as "abuse" unless there's irrefutable evidence. We don't wish the endless "gaslighting" behaviors destroy the entire fil+ community.

In addition, we need to make it clear that the community will not tolerate the behaviors on the part of the SPs, not the clients or the notaries. And, these results need to be clearly reflected in the report, not subjective judgments by the notary or the community.

If we can't govern well, we might as well cancel Fil+.

[raghavrmadya]

Thanks everyone for your comments and feedback. Seeing no absolute consensus, this proposal is did not pass the community review and requires further iterations